Skip to main content
Glama

screen-mcp

一个 MCP (Model Context Protocol) 服务器,让 Claude Agent / Claude Code 可以按需截取用户屏幕,并维护一个最近若干帧的滑动窗口作为多模态问答的上下文。

设计灵感来自豆包等工具的"共享屏幕"功能 —— 低频轮询循环持续维护上下文,加上按需截图路径处理一次性提问;并对采集到的画面执行严格的不落盘、不缓存策略。

功能特性

  • 6 个 MCP toolstart_capturestop_capturecapture_nowset_pollinglist_windowsanalyze_screen

  • Anthropic Messages API 兼容视觉后端:默认走官方 API,也可指向国内大模型代理(见使用国内模型代理

  • 多帧上下文:基于感知哈希 (perceptual hash) 去重,维护最多 20 帧不同的滑动窗口;analyze_screen 把最近 N 帧一起发给视觉模型

  • 单 session 不变量:每个进程最多一个采集会话 —— mss / DXGI 等采集设备无法安全共享

  • Windows 原生单窗口采集:通过 PrintWindow + PW_RENDERFULLCONTENT;全屏采集在所有平台走 mss

  • 数据本地化:画面在内存里 WebP 编码后经 TLS 发给视觉 API,不写盘、不缓存

Related MCP server: Window Screenshooter MCP Server

平台支持

平台

fullscreen

window(单应用)

Linux

✅ (mss)

❌ 返回 unsupported_platform

macOS

✅ (mss)

❌ 返回 unsupported_platform

Windows

✅ (mss)

✅ (PrintWindow)

MCP 传输层(stdio)在所有平台都能跑;只有 mode='window' 采集路径是 Windows 专属。

安装

推荐用 uv 创建虚拟环境并安装,避免污染全局 Python。

# 创建虚拟环境(如果还没有)
uv venv .venv

# 以 editable 模式安装(含 dev 和 windows 依赖)
uv pip install -e ".[dev,windows]"

传统 pip 也可以(前提是已经在虚拟环境里):

# Linux / macOS(开发环境)
pip install -e ".[dev]"

# Windows(目标运行时,会额外装 pywin32)
pip install -e ".[dev,windows]"

第一次运行会读取 .env 文件 —— 见 配置 章节。

配置

.env.example 复制成 .env,填入 Anthropic API key:

cp .env.example .env
# 编辑 .env,设置 ANTHROPIC_API_KEY

所有可配置项(含默认值):

环境变量

默认值

用途

VISION_PROVIDER

anthropic

目前只实现了 anthropic(兼容所有 Anthropic Messages API 端点)

ANTHROPIC_API_KEY

(必填)

API 密钥

ANTHROPIC_MODEL

(必填)

调用的模型名(官方 API 用 claude-sonnet-4-6,代理用代理方指定的字符串)

ANTHROPIC_BASE_URL

(空)

留空走官方 API;填了就走该 URL 下的 /v1/messages

DEFAULT_POLLING_INTERVAL

3.0

轮询模式下相邻两次采集的间隔(秒)

MAX_FRAME_BUFFER

20

滑动窗口大小

PHASH_DEDUPE_THRESHOLD

6

汉明距阈值,低于此值视为重复帧丢弃

PHASH_DEDUPE_LOOKBACK

3

与最近多少帧做 phash 比较

WEBP_QUALITY

75

存储帧的 WebP 压缩质量(1-100)

CAPTURE_MAX_EDGE

1564

发给视觉 API 前的长边像素上限(Anthropic 推荐值)

⚠️ 如果你的 shell 已经导出了 ANTHROPIC_MODEL(比如给 Claude Code 用的),把这个值复制到 .env 里,让本项目的模型选择显式可见。

使用国内模型代理

如果你的 Claude Code 走的是国产大模型代理(代理对外暴露 Anthropic Messages API,即 /v1/messages),直接在 .env 里把 ANTHROPIC_BASE_URL 指向代理地址:

# .env
ANTHROPIC_BASE_URL=https://your-proxy.example.com/anthropic
ANTHROPIC_MODEL=your-model-name          # 代理方指定的模型字符串
ANTHROPIC_API_KEY=your-proxy-key

vision/anthropic.py 用的是官方 anthropic Python SDK,它原生支持 base_url 参数,不需要换 SDK。多 image content blocks(WebP base64)+ 文本 prompt 的请求格式遵循 Anthropic Messages API 规范 —— 任何兼容该规范的代理都可以直接对接。

💡 如果你的 shell 已经导出了 ANTHROPIC_AUTH_TOKEN(Claude Code 的命名),本项目会自动把它当作 ANTHROPIC_API_KEY 用,不需要把 token 复制到 .env。明确设了 ANTHROPIC_API_KEY 的话它优先。

接入 Claude Code

方式一:.mcp.json(推荐)

在项目根目录创建 .mcp.json

{
  "mcpServers": {
    "screen-mcp": {
      "command": "C:\\Users\\xzs\\Desktop\\mcp_test\\.venv\\Scripts\\screen-mcp.exe"
    }
  }
}

然后在 ~/.claude/settings.json 里批准这个 server:

{
  "enabledMcpjsonServers": ["screen-mcp"]
}

方式二:claude mcp add 命令

# 在项目目录下
claude mcp add screen-mcp -- .venv/Scripts/screen-mcp.exe

之后在 Claude Code 会话里,6 个 tool 就以 start_capturestop_capturecapture_nowset_pollinglist_windowsanalyze_screen 的名字可用。

端到端示例

一个典型的 agent 交互流程:

用户: 列出我打开的窗口。

Agent: (调用 list_windows) —— 我看到 Notepad (hwnd 0x1a2b3c) 和 Visual Studio Code (hwnd 0x4d5e6f)。

用户: 对 Notepad 窗口开 3 秒一次的轮询采集。

Agent: (调用 start_capture("window", "Notepad"),然后 set_polling(enabled=True, interval_seconds=3))

用户: 我刚才打了什么?

Agent: (调用 analyze_screen("我刚才打了什么?", lookback_frames=3)) 你写的是:"回家路上买点牛奶。"

MCP tool 参考

start_capture(mode: 'fullscreen'|'window', target?: str)

启动采集会话。返回 {session_id, mode, target, state}

  • mode='window' 必须传 target(窗口标题子串或十六进制 hwnd)

  • 错误:会话已 active 时抛 SessionError;在 Linux/macOS 上请求 mode='window'UnsupportedPlatformError

stop_capture()

结束当前会话。返回 {stopped, state}

capture_now()

按需截一次图。返回帧的元信息(不含图片字节,避免 MCP 消息体爆炸): {frame_id, captured_at, width, height, phash, format, metadata}

set_polling(enabled: bool, interval_seconds: float = 3.0)

开关后台轮询循环。返回 {polling, interval}

list_windows()

枚举可见的顶层窗口。Windows 上返回 [{hwnd, id, title, pid, bbox}],Linux/macOS 上返回 []

analyze_screen(query: str, lookback_frames: int = 3)

把最近若干帧发给视觉模型。返回 {text, frame_ids, region_count, regions, model, tokens_used?}

开发

# 跑所有测试
pytest tests/ -v

# 跑某个模块的测试
pytest tests/test_session.py -v

# 启动 server(stdio 模式,会等 stdin)
python -m screen_mcp.server

Linux 显示器说明

在无显示器的 Linux 主机上,tests/test_capture.pytests/test_capture_linux.pymock 掉 mss 库,让测试套件在没 X server 的情况下也能跑。要跑真实的 mss 采集路径

# Ubuntu / Debian
sudo apt install xvfb
xvfb-run -a pytest tests/test_capture_linux.py -v -k real_mss

test_real_mss_capture_under_xvfb 测试在没装 Xvfb 时会自动 skip。

已知问题与修复

mss.shot(output=BytesIO) 的陷阱 — 早期版本里 MssBackend.capture_frameio.BytesIO() 当成 output 参数传给 sct.shot(),导致 '_io.BytesIO' object has no attribute 'format' 错误。

原因:mss.shot()output 参数期望的是文件名模板字符串(如 "{mon}.png"),不是 file-like 对象;mss 内部会对它调用 .format(),而 BytesIO 没有这个方法。

修复方案:先 sct.shot(mon=1) 拿到返回的文件名,读出 bytes 再删掉临时文件:

filename = sct.shot(mon=1)
try:
    with open(filename, "rb") as f:
        return f.read()
finally:
    os.remove(filename)

项目结构

src/screen_mcp/
├── server.py             # FastMCP 入口
├── tools.py              # 6 个 tool 的实现
├── session.py            # 单例 session + 状态机
├── frame.py              # Frame + pHash 去重 buffer
├── config.py             # pydantic-settings 配置
├── capture/
│   ├── base.py           # CaptureBackend Protocol + Target
│   ├── mss_backend.py    # 跨平台全屏
│   └── windows_backend.py# Windows PrintWindow 单窗口
└── vision/
    ├── base.py           # VisionProvider Protocol
    └── anthropic.py      # AnthropicVisionProvider(默认实现,支持 base_url 代理)

License

TBD.

Available Tools

6 tools
analyze_screenA

Send recent frames to the configured vision provider along with the user's question, and return the model's answer. lookback_frames controls how many of the most recent frames to include (default 3).

ParametersJSON Schema
NameRequiredDescriptionDefault
queryYes
lookback_framesNo

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations provided, so description carries full burden. It discloses that the tool sends frames to a vision provider and returns an answer, but does not mention whether it is read-only, if it has side effects, or if it requires prior setup (e.g., capturing frames). The behavior is mostly clear but lacks specifics on state modification.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with purpose, no redundancy. Every word provides value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema and no annotations, description should cover more context. It lacks details on prerequisite conditions (e.g., vision provider configured, frames captured), return format, error handling, and potential costs. Leaves gaps for an AI agent to infer.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so description must compensate. It explains lookback_frames as controlling number of frames (default 3) and query as the user's question. This adds meaning over schema titles, but query could be more specific (e.g., format, constraints). Adequate but not excellent.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states the tool sends recent frames to a vision provider with the user's question and returns the answer. This distinguishes it from sibling tools like capture_now (capturing frames) and list_windows (listing windows).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Description implies usage for analyzing screen content via a vision provider, and mentions the lookback_frames parameter for controlling frame count. However, it does not explicitly state when to use this tool versus alternatives like capture_now (which captures without analysis) or when not to use it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

capture_nowA

Take a single screenshot on demand. Returns frame metadata (id, size, phash, timestamp) — NOT the image bytes. Use analyze_screen to actually read the content.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Discloses return contents (metadata fields) and explicitly states what is NOT returned (image bytes), which is crucial for correct use. No contradictions with annotations (none provided).

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, no redundant words, critical information (action, return, alternative) presented efficiently. Every sentence earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Covers all necessary information: action, return (with specific metadata fields), and clear alternative for image bytes. No gaps given the tool's simplicity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Tool has no parameters; schema coverage is 100%, so no additional parameter semantics are needed. Baseline 4 as per rules.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states the specific action ('Take a single screenshot on demand') and clearly distinguishes it from sibling tool analyze_screen by noting what it does not return and where to get the image.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides explicit guidance to use analyze_screen for image content, differentiating the primary use case. Lacks guidance for other sibling tools but suffices for the most relevant alternative.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_windowsA

List visible top-level windows. On non-Windows platforms returns [].

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A3.9/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Without annotations, the description carries the burden of behavioral disclosure. It mentions returning an empty list on non-Windows platforms, which is a useful behavior. However, it does not explain what 'visible' means or whether the tool requires specific permissions.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence with no wasted words. It front-loads the action and provides essential information efficiently.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description covers the basic purpose and cross-platform behavior, but it lacks details on what 'visible' entails or how windows are identified. With an output schema present, completeness is adequate but not thorough.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

There are no parameters, and the schema coverage is 100%. The description does not need to add parameter semantics because none exist. According to guidelines, baseline for 0 parameters is 4.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'List' and the resource 'visible top-level windows', and it distinguishes from sibling tools like capture_now and analyze_screen by specifying a distinct action.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description does not explicitly state when to use this tool versus alternatives, nor does it provide prerequisites or exclusions. The cross-platform note is helpful but not sufficient for usage guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

set_pollingB

Enable or disable a low-frequency background polling loop that keeps the frame buffer fresh. Default interval is 3 seconds.

ParametersJSON Schema
NameRequiredDescriptionDefault
enabledYes
interval_secondsNo

TDQS

B3.1/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description should disclose safety and side effects. It only mentions the action and default interval but does not state whether the operation is safe, reversible, or what happens to ongoing captures.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Single sentence, front-loaded with the action, no extraneous words. Efficient and to the point.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 2-parameter tool with no output schema and no annotations, the description is incomplete. It lacks details on behavior when enabled/disabled, relation to other tools, and any return values.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It provides the default for interval_seconds but no additional meaning for the enabled parameter or the range/units of interval_seconds.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool enables or disables a low-frequency background polling loop to keep the frame buffer fresh. The verb 'Enable or disable' and resource 'polling loop' are specific, and the context distinguishes it from sibling capture tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance on when to enable or disable polling or when to prefer this over alternatives. Lacks explicit context for usage decisions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

start_captureA

Begin a screen-capture session. mode='fullscreen' works on every platform; mode='window' is Windows-only and requires target to be either a window-title substring or a hex hwnd (e.g. 'Notepad' or '0x1a2b3c').

ParametersJSON Schema
NameRequiredDescriptionDefault
modeYes
targetNo

TDQS

A3.7/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It discloses platform dependency and target format but lacks information on side effects, permissions, or whether the capture is synchronous. Important behavioral traits are missing.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, each adding unique value. The first sentence states purpose, the second explains parameter details and platform behavior. No redundant or unnecessary information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description covers parameters and platform specifics, but lacks details on return value, error handling, session lifecycle, and whether multiple sessions can run concurrently. For a session-based tool with no output schema, this is incomplete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 0% schema coverage, the description compensates well by explaining 'mode' enum values and 'target' types with examples. It adds meaning beyond the schema, though it could clarify that 'target' is only relevant for 'window' mode.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'Begin a screen-capture session' with a specific verb and resource. It distinguishes from sibling tools like 'capture_now' and 'stop_capture' by implying a session-based operation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides good guidance on mode and platform constraints (fullscreen universal, window Windows-only with target). However, it does not explicitly compare to sibling tools or specify when to use this tool versus alternatives like 'capture_now'.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

stop_captureA

End the current capture session.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

A3.6/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description bears full responsibility for behavioral disclosure. It only states the end action but does not mention any side effects, reversibility, or safety (e.g., whether it can be called multiple times).

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence of five words, conveying the exact purpose with no wasted text. It is front-loaded and efficient.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple action with no parameters and no output schema, the description is mostly complete. However, it could mention if the session is fully saved or if any cleanup occurs, but the minimalism is acceptable given the tool's simplicity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters, and the schema coverage is 100% (trivially). The description adds no parameter information, but there are none to document. Baseline of 4 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('End') and the resource ('the current capture session'), distinguishing it from siblings like 'start_capture' and 'capture_now'. It is specific and unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance on when to use this tool versus alternatives, such as when to stop vs. when to start or analyze. Implied usage from context but no explicit exclusions or when-not scenarios.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 6 tool updatesv0.1.0
    • First observedanalyze_screen
    • First observedcapture_now
    • First observedlist_windows
    • First observedset_polling
    • First observedstart_capture
    • First observedstop_capture

TDQS

A3.9/5.0

Scored across 6 tools

Disambiguation5/5

Each tool has a clear, distinct purpose: capture management, single capture, analysis, window listing, and polling control. No overlap in functionality.

Naming Consistency5/5

All tools follow a consistent verb_noun snake_case pattern (e.g., start_capture, analyze_screen), making intent predictable.

Tool Count5/5

6 tools is well-scoped for a screen capture and analysis server, covering core operations without excess or deficiency.

Completeness4/5

Covers essential lifecycle (start/stop, single capture, analysis, window listing), but lacks frame retrieval by ID and capture region configuration, which are minor gaps.

Maintenance

ActivityInactive
ResponsivenessNo issues

Related MCP Connectors

Related MCP Servers