Skip to main content
Glama

Vision Primitives MCP — 给纯文本 LLM 装上眼睛的视觉原语 MCP 服务器

简体中文 | English

让纯文本模型(DeepSeek / Codex / 任意 MCP 客户端)通过 30 个 MCP 工具获得完整视觉能力:描述 → 定位(坐标)→ OCR → 标注 → 裁切/放大 → 异常扫描 → UI 结构化 → 多轮推理 → 电脑操控。视觉后端可切换(MiMo V2.5 云端 / LM Studio 本地 Qwen2.5-VL 等),单文件 Python,核心仅依赖 Pillow。

项目定位:通用视觉推理桥接层——文本模型 + 任意 VLM 的通用视觉工作流,本地隐私 + 单文件轻量。高泛化优先:核心能力(定位/切块/放大/读取)全部是纯 VLM + PIL,专用检测器(YOLO/CRAFT)只是可插拔加速器,有则加速、无则照跑。与 UI-TARS / CogAgent 等端到端 GUI 模型竞争(它们有专门训练);核心差异化是无 grounding 模型的兜底定位任何模型可用的多轮视觉推理

纯文本模型(推理与决策)
    │  MCP 协议(stdio, JSON-RPC 2.0)
    ▼
vision_primitives_mcp.py(单文件,30 工具,186 项测试)
    │  OpenAI 兼容 API
    ▼
MiMo V2.5(云端)/ Qwen2.5-VL-7B(本地 LM Studio)/ 任意视觉 VLM

实测基准(2026-08-02,程序化 ground truth)

测试图(900×600,元素位置为已知真值):红圆中心 (150,140)、绿三角中心 (740,417):

测试基准图

定位对比(三模型 × 多模式)

定位实测对比

Qwen2.5-VL 对比

定位矩阵(同一基准图,像素级验证):

模型

locate 红圆

locate 绿三角

som 红圆

som 绿三角

单次调用

ui_parse

ui_refine

MiMo V2.5(云端)

10-64px(波动)

79-97px(波动)

33-82px(波动)

13-123px(波动)

15-25s

21.5s

MiMo + som-cv(兜底管线)

0px

4px

10-12s

Qwen3-VL-8B(本地)

90px

164px

77px

123px

10-30s

29.4s

>357s

GLM-4v-flash(智谱云)

89px

84px

191px

145px

1.5-3.3s

Qwen2.5-VL-7B(本地)

28px

27px

15px

123px

1.3-1.7s

12.4s

12.6s

能力矩阵(describe / OCR / 兜底定位 / 文本锚定 / 多轮推理):

模型

describe

OCR(prompt 修复后)

som-cv(颜色目标)

ui_locate 文本锚定

scratch 论文推理

MiMo V2.5(云端)

4.7s ✓

8.9s,4/4 块

0px(3.2s)

8.6s ✓

45.7s 3轮(漏答图趋势)

Qwen2.5-VL-7B(本地)

1.1s ✓

3.5-4.6s,4/4 块

0px

12.8s ✓

9.6s 1轮全对

GLM-4v-flash(智谱云)

1.5s ✓

2.0s,3/4 块(max_tokens≤1024 限制)

53px(格子选偏)

Qwen3-VL-8B(本地)

~10s

23s,4/4 块

0px

消融:定位误差根因是"分辨率稀释"——输入 0.5x→208px / 2x→70px / 目标单独裁切→0-3px。整图定位差不是模型能力问题,是每个对象分到的视觉 token 太少;"粗定位 → 裁切 → 精定位"的管线是正解som_locate 递归、final="cv"text_zoom 网格均基于此)。

ZeroBench 评测(视觉推理基准,持续追踪)

ZeroBench("Impossible" 基准,发布时 SOTA 0%)评测工作流见 benchmarks/zerobench/。基线(MiMo V2.5 直答 + pass@5 采样,2026-08-02):

数据集

题数

命中

numeric 命中率

主问题

100(numeric 84)

2

2.4%(超发布时 SOTA 0%)

子问题集

334(numeric 257)

183

71.2%

结论:主问题考多步组合视觉推理(模型能力瓶颈);子问题集是工具链主场——拆解后的单步问题 MiMo 直答 + 采样达 71%,验证"难题拆解 + 采样"是弱模型打硬基准的可行路径。每次模型/工具升级在此基线上重跑对比。

Related MCP server: SiliconFlow-Vision-MCP

工具一览(30 个)

分类

工具

作用

基础视觉

describe_image / analyze_image

描述 / 结构化分析(含坐标原语)

定位

locate_object

坐标输出定位,refine 两阶段精修

som_locate

SoM 编号网格递归定位(final: box/number/cv 三模式)

cursor_locate

移动光标 + 视觉反馈循环定位(GUI-Cursor 范式,建议云端强模型)

cv_locate

CV 兜底:颜色分割/模板匹配,像素级(numpy 加速 285x)

文本

ocr_image

逐文本块 OCR,带 bbox

text_detect

CRAFT 文本区域检测(本地 onnx,可选加速器)

text_zoom

程序化切块放大读取(高泛化默认路径):网格逐块放大 VLM 精读

UI 结构化

ui_parse / ui_locate / ui_refine

YOLO 检测 + 文本锚定 + VLM 语义编辑检测框

图像处理

annotate_image / crop_image / zoom_region

标注 / 裁切 / 放大

高级推理

scratch_think

视觉草稿纸:跨轮层栈 + 自适应裁切放大(图文论文理解)

compare_images / compare_infer / reason_graph / annotate_infer

多图对比 / 联合推理 / 图形推理协议 / 虚拟标注

扫描

scan_anomalies

自动异常扫描:切块候选 → 高清逐点验证(PCB 实战)

电脑控制

screen_capture / screen_info / screen_click / screen_move / screen_drag / screen_scroll / screen_type / screen_key

截屏 + 鼠标键盘(Windows-only,安全开关默认关)

诊断

vision_health

后端配置与连通性

配套脚本 paper_reader.py:arXiv/URL/PDF/图片 → 多屏截图 → 分屏 scratch_think → 结构化摘要。

核心方法论

  1. 对抗分辨率稀释:一切定位精度的基础。som_locate 编号递归(模型只选格子)、final="cv"(VLM 收敛 + CV 像素级)、text_zoom 网格切块(插图小字)——三个工具共享"局部放大"原理,只是块的产生方式不同(模型决策 / 检测器 / 网格)

  2. 文本锚定:UI 操作指令几乎总带文字——OCR 定位文字 → 控件框。比让模型猜坐标可靠一个量级

  3. 检测器可插拔:YOLO(icon_detect)/ CRAFT(text)都是"有则加速、无则照跑"——高泛化路径(纯 VLM)始终可用

  4. 视觉草稿纸(scratch_think):无 grounding 模型缺"视觉工作记忆"——层栈把中间状态渲染回图,模型每轮可见可编辑;坐标链保证裁切放大后一切可映射回原图

  5. 几何判定程序化:重叠合并、坐标换算、收敛检测这类确定性问题交给代码,VLM 只做语义判断(实测 VLM 合并会把相邻独立元素误并)

快速上手

# ~/.codex/config.toml
[mcp_servers.vision-primitives]
command = "python"
args = ['/path/to/vision-primitives-mcp/vision_primitives_mcp.py']
startup_timeout_sec = 60

[mcp_servers.vision-primitives.env]
VISION_API_BASE = "https://api.xiaomimimo.com/v1"      # 或本地 http://127.0.0.1:1234/v1
VISION_API_KEY = "你的MiMo密钥"                          # 本地 LM Studio 用任意占位
VISION_MODEL = "mimo-v2.5"                              # 或 qwen/qwen2.5-vl-7b
VISION_OUTPUT_DIR = '/path/to/vision-primitives-mcp/generated'

模型选型:定位/审查/多轮推理首选 Qwen2.5-VL-7B(grounding 专才 + 非思考型,1.3-1.7s/次);MiMo 通用但定位波动大,需 VISION_SAMPLES=3;Qwen3-VL-8B 仅描述/OCR 场景考虑;GLM-4v-flash(智谱)速度优秀(1.5-3.3s)适合 describe/OCR/粗定位,定位精度弱(84-191px),且 max_tokens 上限 1024(需 VISION_MAX_TOKENS=1024

可选增强(全部"有则加速、无则照跑"):

增强

放置

收益

numpy

pip install numpy

模板匹配 285x 加速

YOLO UI 检测

models/icon_detect.pt下载,39.7MB)

ui_parse 像素级 UI 元素检测

CRAFT 文本检测

models/craft_text.onnx下载,79MB)

text_detect/text_zoom 检测框加速

playwright

pip install playwright

paper_reader 截图

已知边界(诚实声明)

  • 整图定位受分辨率稀释(20-164px),用 som 递归 / final="cv" / text_zoom 对抗

  • 极小小字(<10px):text_zoom 网格放大可读(实测 False Stop/Class Settings),但个别字符仍可能误读——模型识别极限

  • cursor_locate 本地模型不可用(483/100px),仅云端强模型;ui_refine VLM 审查建议云端(本地 8B 数分钟)

  • OCR 对 prompt 长度敏感(已修复,保持简短指令);表格数据首选 DOM 提取,视觉 OCR 兜底(90-120s)

  • 平台:屏幕控制 8 工具 Windows-only;安全:URL 图片有 SSRF 防护(私网拦截)与解压炸弹限制(URL 源 50MP)

  • 延迟:MiMo 15-25s/次、本地 Qwen2.5-VL 1.3-1.7s/次、多轮推理按轮倍增

版本历史(精简)

  • v1.14.1(2026-08-02):ZeroBench 评测工作流(benchmarks/zerobench):主问题 2.4% / 子问题 71.2% numeric 基线,断点续跑 + 自动判分

  • v1.14(2026-08-02):text_detect(CRAFT 可选)+ text_zoom(程序化切块,高泛化默认路径,实测读出 <10px 图内标注);30 工具 / 186 测试

  • v1.13(2026-08-02):scratch_think 视觉草稿纸(层栈 + 自适应 zoom + 坐标链);paper_reader.py 论文阅读工作流;OCR prompt 修复(召回 1→4 块);extract_json 截断容错

  • v1.12(2026-08-02):ui_refine 检测框语义编辑(VLM 删除误检/标注 + 程序化几何合并)

  • v1.11(2026-08-02):ui_parse / ui_locate(UI 结构化 + 文本锚定)+ YOLO 可选检测 + SSRF/解压炸弹防护 + numpy 模板匹配

  • v1.10(2026-08-02):som_locate(SoM 编号递归,final=box/number/cv)+ cursor_locate + cv_locate

  • v1.9(2026-08-02):Computer Use(8 个屏幕控制工具,安全开关)

  • v1.7-1.8:refine 两阶段精修(70→20px)、reason_graph、annotate_infer、坐标格式实验(像素最稳)

  • v1.1-1.6:scan_anomalies(PCB 实战)、compare_images、中文 OCR 修复

  • v1.9.1 / v1.13.3 修复记录:MCP 响应帧 jsonrpc/id(严格客户端兼容)、长 JSON 截断容错

测试与安全

python test\run_tests.py   # 186 项 mock 测试,不依赖真实 key
python test\e2e_mimo.py    # 真实端到端(需要 VISION_API_KEY)
  • 输入图片只读,上传仅发往配置的后端;out_path 强制限定输出目录

  • URL 图片:SSRF 防护(私网/链路本地/元数据拦截,回环放行)+ 解压后 50MP 限制

  • API key 仅存本地配置;屏幕控制类工具默认拒绝(VISION_ALLOW_SCREEN_CONTROL=1 启用)

仓库github.com/zouyuanqing/vision-primitives-mcp

Available Tools

27 tools
analyze_imageB

结构化分析:返回 description + visual_primitives(box/point 坐标与标签)。

ParametersJSON Schema
NameRequiredDescriptionDefault
imageYes
formatNoprimitives 字段风格,默认 generic
questionNo可选的关注点

TDQS

B3.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description discloses the output structure (description + visual_primitives with coordinates and labels), which is a key behavioral trait. However, with no annotations available, it does not cover input requirements, limitations, or failure modes, leaving some ambiguity about expected behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence that packs essential output information without any redundancy. It is perfectly concise for the purpose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Despite the concise output summary, the description lacks usage context, input semantics, and any detail about the visual_primitives structure. With no output schema and no annotations, the description carries a heavy burden and falls short of being complete for a 3-parameter tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema already describes the 'format' and 'question' parameters, but the required 'image' parameter lacks any description, and the tool description doesn't compensate. With 67% schema coverage, the baseline is 3, but the missing image semantics are a noticeable gap.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states the tool performs structured analysis and returns description plus visual_primitives with box/point coordinates and labels. This clearly identifies the tool's output and distinguishes it from siblings like describe_image or ocr_image, though it doesn't explicitly name alternatives.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives no guidance on when to use this tool versus alternatives. It only states what it returns, leaving the agent to infer usage context. There is no mention of scenarios, exclusions, or sibling comparisons.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

annotate_imageA

在图片上画矩形框/圆点/标签(圈画标记),保存标注图并返回路径。

ParametersJSON Schema
NameRequiredDescriptionDefault
imageYes
itemsYes标注项数组或单个对象:[{label, box 或 point, color}],box=[x1,y1,x2,y2],point=[x,y]
styleNo{line_width, font_size, color}
coordsNo坐标单位,默认 pixel
out_pathNo输出路径(必须位于输出目录内),默认自动命名

TDQS

A3.7/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full disclosure burden. It does state the key behavior: save the annotated image and return the path. However, it does not mention whether the original image is modified, how coordinates are validated, or possible side effects like file overwriting. This is adequate but lacks depth.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single concise sentence that states the action and the result. It is front-loaded with the core function ('在图片上画矩形框/圆点/标签') and ends with the output behavior ('保存标注图并返回路径'). No filler or redundant information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the moderate complexity (5 parameters, nested objects, no annotations, no output schema), the description provides the essential output (returns path) and mentions the saving behavior. However, it lacks broader context such as when to use the tool, coordinate system handling, or limitations. The schema covers parameter details, but the description alone is thin for a tool of this complexity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema provides descriptions for 4 of 5 parameters (80% coverage), so the baseline is 3. The description does not add any parameter semantics beyond what the schema already offers. It only mentions the types of annotations (boxes/dots/labels) which aligns with the 'items' parameter but adds no new detail.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's function: drawing rectangles/dots/labels on an image, saving the annotated image, and returning the path. This distinguishes it from sibling tools like crop_image, ocr_image, and analyze_image, which have different purposes. The verb '画' (draw) and resource '图片' (image) are specific and unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No explicit when-to-use or when-not-to-use guidance is given. The usage is implied from the description: use this tool when you need to add visual markers to an image. However, no alternatives or exclusions are mentioned, so it only meets the 'implied usage' level.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

annotate_inferA

虚拟标注 + 增强图形推理:把框/点/连线/箭头/圆等标注(不修改原图)注入视觉模型,引导空间关系推理。mode=virtual 用坐标文本注入;mode=overlay 生成半透明叠加图。

ParametersJSON Schema
NameRequiredDescriptionDefault
modeNovirtual=坐标文本注入(默认,原图零修改);overlay=半透明叠加图
alphaNooverlay 模式叠加透明度 (0,1],默认 0.35
imageYes
itemsYes标注数组或单个对象:[{id?, type: box|point|line|arrow|circle|polygon|bubble, label, color, coords, box/point/from/to/center/radius/points/text/direction}]
detailNo细节程度
questionYes推理问题,如:框A中的元件是什么?A到B的连线代表什么连接关系?
auto_boxesNo自动框选:字符串或目标数组(如 '所有按钮'),内部 locate 后生成紫色框参与推理
correctionsNo多轮修正操作数组:[{op: add|remove|move|resize|set, id, delta/to/box/point/...}],基于 items 修正后推理

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the transparency burden. It discloses a key behavior: '不修改原图' (does not modify the original image), and clarifies how modes work. However, it does not describe the output/return format or any other side effects, which is a gap given there is no output schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely concise, consisting of two sentences that convey the core purpose, modes, and key safety behavior. Every phrase adds relevant information without redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity (8 parameters, no output schema, no annotations), the description provides a solid overview, covers modes, clarifies non-destructive behavior, and gives examples for items and question. However, it omits explicit return-value semantics, which would improve completeness, but is largely adequate due to strong schema coverage.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema already has 88% coverage, providing baseline. The description adds value by explaining the modes (virtual vs overlay) and listing annotation types (框/点/连线/箭头/圆) that map to the items parameter, beyond what the schema states. It also gives an example of the question parameter.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: '虚拟标注 + 增强图形推理' (virtual annotation + enhanced graphical reasoning), with specific actions like injecting annotations into the vision model to guide spatial reasoning. It distinguishes itself from sibling tools like annotate_image by adding the reasoning component, and details two modes (virtual and overlay).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage for spatial relationship reasoning and explains mode selection (virtual vs overlay), but it does not explicitly state when to use this tool over alternatives or mention exclusions. No comparison with siblings like annotate_image or describe_image is provided.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

compare_imagesA

多图对比分析(2-4 张):A/B 截图对比、设计稿一致性、多帧分析,返回逐项对比结果。

ParametersJSON Schema
NameRequiredDescriptionDefault
detailNo细节程度,默认 balanced
imagesYes2-4 张本地图片路径或 http(s) URL
questionNo对比重点,如:UI 有什么变化

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of disclosure. It states the return behavior ('返回逐项对比结果') and the scope of 2-4 images, but it does not mention any side effects, permissions, or limitations, which are less critical for a read-only analysis tool but still relevant.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single concise sentence in Chinese, front-loaded with the core purpose and immediately followed by concrete use cases and return type. Every phrase adds value, with no redundancy or filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with no output schema and no annotations, the description provides sufficient context: it explains what the tool does, how many images to use, common scenarios, and the nature of the output. It lacks explicit output structure but is adequate for an agent to select and invoke the tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the input schema already describes all parameters (detail, images, question). The description does not add significant parameter-level detail beyond the schema, so the baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states a specific verb and resource: '多图对比分析' (multi-image comparison analysis), with concrete use cases (A/B screenshot comparison, design mockup consistency, multi-frame analysis). It distinguishes from sibling tools like describe_image/analyze_image by emphasizing multiple images and comparison-specific tasks.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives example use cases that imply when to use the tool (A/B testing, design consistency, multi-frame analysis), but it does not explicitly state when not to use it or mention alternatives. The usage context is clear but not prescribed with exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

compare_inferA

多图联合推理(2-4 张):每张图可带独立标注(items_per_image),联合对比/推理关系(差异、因果、时序、整体结论)。

ParametersJSON Schema
NameRequiredDescriptionDefault
modeNo默认 virtual
alphaNooverlay 透明度,默认 0.35
imagesYes
questionYes联合推理问题
items_per_imageNo{图索引(0开始): 标注数组},每图可选

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden. It discloses core behavior: multi-image comparison and reasoning, per-image annotations, and relation categories, but it does not describe output format, potential limitations, or how mode/alpha parameters influence behavior, leaving notable uncertainty for the agent.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, dense sentence that front-loads the purpose, then specifies input count, per-image annotations, and reasoning categories. It contains no filler or redundant elaboration.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool has no output schema, optional mode/alpha parameters, and a nested items_per_image object, so the description needs to cover return behavior and parameter effects. It adequately conveys the core purpose and relation types but omits output format and mode/alpha semantics, making it moderately complete but not fully sufficient for all invocation contexts.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is high (80%) and the schema already documents items_per_image as an optional per-image mapping, mode defaults, and alpha. The description adds little beyond restating that images can carry independent annotations, so it does not meaningfully expand parameter understanding beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly identifies the tool as performing multi-image joint reasoning on 2-4 images, with optional per-image annotations and enumerated relation types (differences, causality, timing, overall conclusion). It is specific enough about scope, though it does not explicitly contrast it with the similarly named sibling tool compare_images.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage for multi-image reasoning scenarios and mentions the supported relationship types, but it provides no explicit guidance on when to prefer this tool over alternatives like compare_images or reason_graph, and no exclusions or conditions are stated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

crop_imageA

按坐标裁切图片(可边缘外扩),保存并返回路径与新尺寸。

ParametersJSON Schema
NameRequiredDescriptionDefault
boxYes[x1,y1,x2,y2]
imageYes
coordsNo坐标单位,默认 pixel
out_pathNo
expand_pxNo四边外扩像素数,默认 0

TDQS

A3.7/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description must carry the behavioral disclosure burden. It discloses key behaviors: it saves the cropped image and returns the path and new dimensions, and supports edge expansion. However, it does not describe the return format or what happens if out_path is omitted, and it leaves coordinate system defaults to the schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence that states the core action first ('按坐标裁切图片') and adds supplementary details. Every word adds value, with no redundancy or filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool has 5 parameters, no output schema, and no annotations. The description adequately conveys the main operation and result, but it lacks details on return value structure and parameter semantics for image/out_path. Given the complexity, it is minimally viable but not fully complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 60%, and the description adds some meaning beyond the schema by explaining the purpose of coordinates and edge expansion, and mentioning output path. It does not clarify the 'image' parameter or 'out_path' parameter in detail, but it partially compensates for the coverage gap.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool crops an image by coordinates, with optional edge expansion, and saves/returns the result. This is a specific verb+resource combination that is distinct from sibling tools like ocr_image or describe_image, which have different purposes.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage (crop an image when needed) but provides no explicit guidance on when to use this tool versus alternatives, nor any exclusions. It does not mention scenarios like 'use this instead of zoom_region' or when not to use it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

cursor_locateA

移动光标 + 视觉反馈循环定位:渲染光标位置,模型输出目标相对光标的偏移(dx/dy),逐步逼近目标中心。对相对偏移的估计比绝对坐标更准(参考 GUI-Cursor 交互式搜索范式)。

ParametersJSON Schema
NameRequiredDescriptionDefault
imageYes
startNo光标初始位置(归一化 0-1 比例),默认 [0.5, 0.5] 即图中心
coordsNo返回坐标单位:pixel(默认)或 norm(0-1000 归一化)
targetYes要定位的目标,如:蓝色提交按钮 / 红色圆形 / 报错文字
out_pathNo保存最终光标位置的图(必须位于输出目录内)
max_stepsNo最大移动轮数,默认 6,范围 1-20
step_ratioNo单轮最大移动比例(相对图宽高),默认 0.25,范围 0.05-0.6

TDQS

A4.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries full transparency burden. It explains the iterative feedback loop and why it is used, but does not disclose potential side effects (e.g., whether the actual screen cursor is moved), failure behaviors, or what happens if `max_steps` is exceeded. This is moderate but incomplete.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences: the first states the core mechanism, the second justifies its advantage. No wasted words, front-loaded with the tool's purpose and method.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a moderately complex tool with 7 params and no output schema, the description explains the distinct interactive approach. It lacks explicit mention of return format, but the schema's `coords` parameter indirectly covers this. Overall, it is nearly complete for a specialized locator.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema covers 86% of parameters, so the baseline is 3. The description adds meaning by explaining the relative-offset mechanism, which helps interpret `max_steps` and `step_ratio`. It does not detail individual parameters, but the core behavior clarifies their purpose.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's function: moving the cursor with a visual feedback loop to iteratively locate a target by estimating relative offsets (dx/dy). This distinguishes it from sibling tools like `locate_object` or `som_locate` by emphasizing the interactive refinement mechanism.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It provides a clear rationale for when this tool is preferred: when relative offset estimation is more accurate than absolute coordinates, citing the GUI-Cursor interactive search paradigm. However, it does not explicitly mention when not to use it or compare with alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

cv_locateA

传统 CV 精定位(备选方案):颜色分割 + 连通域质心(像素级,零依赖,实测 0-4px)或模板匹配。适用简单目标(纯色 UI 元素、几何图形、固定模板);泛化有限,通用目标请用 locate_object / som_locate。

ParametersJSON Schema
NameRequiredDescriptionDefault
colorNo颜色名(red/green/blue/yellow/orange/purple/cyan/white/black/gray) 或 [r,g,b] 或 [r1,g1,b1,r2,g2,b2],与 template 至少其一
imageYes
coordsNo返回坐标单位:pixel(默认)或 norm(0-1000 归一化)
targetYes目标描述
min_areaNo最小色块面积过滤,默认 0
templateNo模板图路径(模板匹配模式),与 color 至少其一;建议在 VLM 粗定位后的局部图上使用

TDQS

A4.6/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries full burden. It discloses internal mechanisms (color segmentation, centroid, template matching), zero dependencies, and measured accuracy (0-4px). However, it does not mention output format or potential failure modes, leaving a minor transparency gap.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with core function, then use cases and alternatives. No wasted words, every sentence earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No output schema, but the description covers purpose, method, accuracy, use cases, and alternatives. It does not explicitly state the return format, though the schema's coords parameter hints at it. Given moderate complexity and good schema coverage, it is mostly complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is high (83%), so baseline is 3. The description adds value by clarifying that color and template are alternative modes (matching the schema's 'at least one' requirement) and suggesting template use on VLM coarse-localized local regions, which is not in the schema field descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it performs traditional CV precise localization via color segmentation + connected component centroid or template matching. It explicitly positions itself as an alternative to locate_object/som_locate, distinguishing it from siblings with specific methods and scope.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly states suitability for simple targets (solid colors, geometric shapes, fixed templates) and limitations of generalization, then directly instructs to use locate_object/som_locate for general targets. This provides clear when-to-use and alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

describe_imageB

用视觉模型描述图片内容,返回文字描述。

ParametersJSON Schema
NameRequiredDescriptionDefault
imageYes本地图片绝对路径或 http(s) 图片 URL
detailNo细节程度,默认 balanced
questionNo可选的针对性问题

TDQS

B3.1/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description must carry the full burden of behavioral disclosure. It states it returns text but does not disclose limitations (e.g., supported formats), whether it is read-only, or potential errors. It is truthful but minimal.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, concise sentence that is front-loaded with the core action and output. No unnecessary words or redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool is simple, and the schema covers parameters. However, with no output schema and many sibling tools, a bit more context about the return format or example use cases would improve completeness. As is, it is minimally adequate.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already documents all three parameters with descriptions (100% coverage). The description adds no additional meaning about how parameters affect behavior, so the baseline score of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool uses a vision model to describe image content and returns text. It is a specific verb+resource, but it does not explicitly differentiate from siblings like analyze_image or ocr_image.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus alternatives such as ocr_image or analyze_image. The description is a single statement with no context on conditions, exclusions, or preferred use cases.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

locate_objectB

在图片中定位目标对象,返回坐标 primitives(让 LLM 输出坐标)。找不到会返回 count=0。

ParametersJSON Schema
NameRequiredDescriptionDefault
imageYes
coordsNo返回坐标单位:pixel(默认,像素)或 norm(0-1000 归一化)
refineNo两阶段精修:粗定位后裁切放大二次定位(更准,代价是每个候选多一次视觉调用),默认 false
targetYes要定位的目标,如:蓝色提交按钮 / 红色圆形 / 报错文字

TDQS

B3/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description adds one key behavioral trait: '找不到会返回 count=0' (returns count=0 if not found), which is useful and goes beyond the schema. It also mentions the coordinates are 'primitives' for LLM output. However, it does not disclose other behaviors such as error handling, performance implications, or whether the operation is read-only/destructive, and with no annotations, the burden is higher.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence that front-loads the core purpose and includes a key behavioral detail (count=0). It is concise with no wasted words, and the structure is clean.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With 4 parameters, no output schema, and no annotations, the description is underspecified. It does not describe the return structure (e.g., shape of primitives, how count is included), doesn't clarify when to use this tool over many siblings, and doesn't explain the coords/refine options sufficiently. The description is too minimal for the tool's complexity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 75% and the description adds little parameter meaning beyond what's in the schema. It mentions '坐标 primitives' which loosely relates to the coords parameter, but does not explain the refine or target parameters or how they affect behavior. The schema descriptions for coords and refine are already present, so the description does not compensate for any gaps.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool locates a target object in an image and returns coordinate primitives, with a specific verb-resource pairing ('locate object in image'). However, it does not differentiate from sibling tools like ui_locate or cv_locate, which also locate objects, so it lacks sibling differentiation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage for object localization but provides no explicit guidance on when to use this tool versus alternatives. There is no mention of exclusions, prerequisites, or when another locator (e.g., ui_locate, som_locate) would be more appropriate.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

ocr_imageB

OCR 提取图片中所有文字块,返回 text + bbox(像素与归一化坐标)。

ParametersJSON Schema
NameRequiredDescriptionDefault
imageYes
languageNo语言提示,默认 auto

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden. It does disclose the output structure (text + bbox with both pixel and normalized coordinates), which is useful. However, it does not mention input constraints, error behavior, or any side effects, leaving gaps in behavioral disclosure.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, succinct sentence that immediately states the core function and return format. No wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool is simple, but the description lacks guidance on prerequisites or image format. It does describe the return output, which compensates for the missing output schema. However, it doesn't cover usage context or when alternatives should be chosen, making it merely adequate.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 50% description coverage, but the tool description adds minimal meaning for the parameters. It identifies the image as the input via context, but does not explain the language parameter or clarify how images are passed (path, URL, base64). The language parameter is only described in the schema, not reinforced.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it performs OCR to extract all text blocks from an image and return text plus bounding boxes. The term 'OCR' and the focus on 'text blocks' distinguishes it from sibling image tools like describe_image or locate_object.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no explicit guidance on when to use this tool versus other vision tools. It neither mentions alternatives nor specifies exclusions. The context of text extraction is implied by the name, but no direct comparison is given.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

reason_graphB

交互式图形推理协议:原语(locate/measure) → 语义(semantic/hypothesis) → 标注(annotate/verify) 多轮循环。session 跨轮传递状态。

ParametersJSON Schema
NameRequiredDescriptionDefault
stepYes本轮动作:{type: locate|measure|annotate|semantic|hypothesis|verify|next, ...}(locate: target/refine;measure: measure=distance|angle|area + refs;verify: id;hypothesis/semantic: text)
imageYes
sessionNo上一轮返回的 session(primitives/annotations/semantics/hypotheses),第一轮可省略
questionNo总体推理目标(各轮可带)

TDQS

B3.3/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden. It discloses the key behavioral trait that session carries state across rounds, which is valuable. However, it does not mention whether the tool is read-only, what it returns, or any side effects, leaving notable gaps in behavioral transparency.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise, using a compact arrow notation to summarize the reasoning flow and a separate sentence for session state. Every word earns its place, making it efficient and front-loaded, though the notation might be slightly cryptic for some readers.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool is complex with a nested step object and multiple action types, yet the description provides only a high-level protocol overview. It lacks operational details such as how to start a session, how to chain steps, expected return values, and error handling. Given no output schema, the description is insufficient for reliable multi-round use.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema covers 75% of parameters with descriptions, including a detailed explanation of the step object. The description adds context about the protocol flow and the role of session in carrying state, which helps clarify parameter usage. However, it does not provide significant per-parameter details beyond what the schema already offers.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool is an interactive graphical reasoning protocol with a multi-round loop from primitives (locate/measure) to semantics (semantic/hypothesis) to annotation (annotate/verify). This distinguishes it from single-step sibling tools like ocr_image or locate_object, though the term 'reasoning protocol' is somewhat abstract.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies use for multi-round iterative reasoning tasks on images, mentioning the loop and session state. However, it does not explicitly state when to use this tool versus simpler alternatives, nor does it provide any exclusions or when-not-to-use guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

scan_anomaliesA

自动扫描图片中的异常/歪斜元件:把区域切成带重叠的块逐块定位候选,再从原图高清裁切逐个验证,输出带置信度与角度/丝印的报告。

ParametersJSON Schema
NameRequiredDescriptionDefault
imageYes
regionNo可选:限定扫描区域 [x1,y1,x2,y2] 像素,默认全图
targetNo要找的异常特征描述,默认:摆放歪斜、方向与周边不一致的元件
verifyNo是否自动高清验证候选,默认 true
overlapNo切块重叠像素,默认 250
max_tilesNo切块数上限(1-12),默认 6
tile_sizeNo切块边长(像素),默认自动

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden. It reveals the multi-step algorithm (overlapping tiles, candidate localization, high-resolution crop verification) and discloses output fields (confidence, angle, silkscreen). This goes beyond a generic 'scan' claim, though it stops short of discussing performance implications or input requirements.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, dense sentence but remains structured and front-loaded with the primary purpose. It packs substantial detail without excessive verbosity, earning its length through workflow explanation. It could be slightly more scannable, but it is far from bloated.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given 7 parameters and no output schema, the description provides a comprehensive overview: purpose, algorithm, and key output fields. It does not explicitly detail every parameter (e.g., target, max_tiles) but the schema covers those. The algorithm context helps explain why certain parameters exist, making the tool's behavior well-rounded.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is high (86%), so baseline is 3. The description adds algorithm context that explains the purpose of region, overlap, tile_size, and verify parameters, linking them to the tiling and verification process. This enriches the semantic meaning beyond the schema's individual descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states the tool scans for abnormal/skewed components in images, using a specific verb ('scan') and resource ('abnormal/skewed components'). It distinguishes itself from sibling tools like analyze_image or locate_object by focusing on anomaly detection and describing a tiled verification workflow.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage for detecting anomalies in images but does not explicitly instruct when to use this tool versus alternatives, nor does it mention exclusions or prerequisites. Sibling tool names provide context, but the description itself lacks comparative guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

screen_captureA

截屏(全屏或指定区域),保存 PNG 并返回路径。配合 locate_object/describe 实现「看屏幕」。

ParametersJSON Schema
NameRequiredDescriptionDefault
regionNo可选 [x1,y1,x2,y2] 屏幕坐标区域,默认全屏
out_pathNo

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries full burden. It discloses the core side effect of saving a PNG and returning a path, but omits details such as file location, overwrite behavior, permission requirements, or error handling, which are relevant for a screen capture tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single concise sentence that front-loads the primary action (截屏) and states the output and usage context. Every part adds value without redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple tool with 2 optional parameters and no output schema, the description covers purpose and regional parameter, but leaves out_path semantics and return-path details vague. It lacks enough detail about the second parameter to be fully complete, though it does provide integration context.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 50%: region has a description in the schema, but out_path does not. The description mentions 'full screen or specified region' for region, but provides no additional meaning for out_path beyond 'saving PNG and returning path', leaving its role ambiguous. It does not fully compensate for the missing schema description.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool captures a screenshot (截屏) of the full screen or a specific region, saves it as PNG, and returns the path. It also distinguishes itself from sibling analysis/action tools by noting it provides the visual input for locate_object/describe.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description includes a direct usage context: '配合 locate_object/describe 实现「看屏幕」' indicating it is used in conjunction with those tools to see the screen. It does not explicitly contrast with alternatives or provide exclusions, but the context is clear enough for selection.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

screen_clickA

鼠标点击(需 VISION_ALLOW_SCREEN_CONTROL=1)。坐标通常来自 locate_object 对截图的定位结果。

ParametersJSON Schema
NameRequiredDescriptionDefault
xYes
yYes
buttonNo默认 left
doubleNo是否双击,默认 false

TDQS

A4.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the burden. It discloses a permission requirement (VISION_ALLOW_SCREEN_CONTROL=1) and that coordinates are derived from locate_object. However, it doesn't disclose what happens if the permission is missing, whether the mouse pointer moves, or any error behavior, which are relevant for a control action.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two short sentences, front-loaded with the action and prerequisite. Every clause adds value, no redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple click tool with 4 parameters and no output schema, the description covers the core purpose, a prerequisite, and the coordinate provenance. It lacks details on optional parameters' behavior beyond schema, but that's not needed. Given siblings, it could mention not to use for dragging, but name already conveys that.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema covers button and double with descriptions, but x and y have no schema descriptions. The description adds meaning to x and y by stating they are typically from locate_object's localization result on screenshots, providing a coordinate source. This compensates for the 50% schema coverage. Button and double are already explained in the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool performs a mouse click (鼠标点击), which is a specific verb and resource. It distinguishes from sibling tools like screen_move and screen_drag by specifying the click action. The additional context about coordinates from locate_object further clarifies its purpose.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It provides a clear usage context: coordinates usually come from locate_object, and it requires the VISION_ALLOW_SCREEN_CONTROL=1 environment variable. This implies when to use (after locating objects) and a prerequisite. It doesn't explicitly state exclusions or alternatives, but for a click tool this is sufficient.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

screen_dragC

鼠标拖拽(需 VISION_ALLOW_SCREEN_CONTROL=1)。

ParametersJSON Schema
NameRequiredDescriptionDefault
x1Yes
x2Yes
y1Yes
y2Yes
buttonNo
durationNo拖拽时长秒,默认 0.2

TDQS

C2.4/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It discloses the VISION_ALLOW_SCREEN_CONTROL requirement, which is useful, but it does not describe the drag behavior, button handling, coordinate system, or potential side effects.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely brief—a single sentence with a necessary prerequisite. There is no waste, but it is under-specified relative to the tool's complexity.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness1/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With 6 parameters, no output schema, and no annotations, the one-line description is far from complete. It lacks coordinate semantics, return values, button details, and behavioral context.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters1/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is only 17% (only duration has a description). The description adds no meaning to x1, y1, x2, y2, or button, leaving these essential parameters undefined.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool performs a mouse drag operation and includes a specific prerequisite. It is distinguishable from siblings like screen_click and screen_move by the action name, though it does not explicitly differentiate.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It mentions the required environment variable VISION_ALLOW_SCREEN_CONTROL=1, which is a precondition, but gives no guidance on when to use drag rather than other input tools, nor any exclusions or typical usage scenarios.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

screen_infoC

屏幕信息:分辨率、DPI、电脑控制开关状态。

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden. It does mention the kind of information returned (resolution, DPI, control switch status), but it is vague about what 'control switch status' means, does not explicitly state it is a read-only query, and omits any side effects or access requirements.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is very short and waste-free, but it is a noun phrase rather than a complete sentence, and it lacks critical details. It is not as well-structured as the highest-conciseness examples, which use full sentences to convey scoping and alternatives.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool is simple (0 parameters, no output schema), but the description still leaves ambiguity around terms like 'control switch status' and does not explain the return format or how this tool fits among screen-related siblings. It is not complete enough for an agent to invoke it confidently without additional context.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has no parameters and the schema is empty, so parameter semantics is trivially clear. The description adds useful context about the data returned, which exceeds the baseline for a zero-parameter tool.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly identifies the resource (screen) and the specific information provided (resolution, DPI, control switch status). It is more informative than just the tool name, but it lacks an explicit verb and does not differentiate it from sibling tools like screen_capture or vision_health.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance on when to use this tool versus alternatives such as screen_capture or vision_health. The description only states what it does, not when it should be preferred, and provides no exclusions or prerequisites.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

screen_keyA

按键或组合键(需 VISION_ALLOW_SCREEN_CONTROL=1)。如 enter / tab / ctrl+c / alt+tab。

ParametersJSON Schema
NameRequiredDescriptionDefault
keyYes单个按键或 ctrl+shift+key 组合

TDQS

A4.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the burden. It discloses the environment variable prerequisite and example key formats, which adds helpful context. However, it does not describe failure behavior, side effects, or what happens if the environment variable is not set.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence with examples, efficiently conveying purpose and prerequisite without unnecessary words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple one-parameter tool, the description covers purpose, prerequisite, and examples. There is no output schema, so return behavior is not explained, but this is a minor gap given the tool's simplicity and the descriptive sibling context.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema already describes the 'key' parameter, and the description adds concrete examples (enter, ctrl+c, alt+tab) that illustrate expected formats and usage, going beyond the schema's basic description.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it presses a key or key combination, with concrete examples (enter, tab, ctrl+c, alt+tab). The verb '按键' (press key) and resource are specific, and it is distinct from sibling tools like screen_click (mouse) and screen_type (typing).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description states a clear requirement (VISION_ALLOW_SCREEN_CONTROL=1) and provides examples, implying usage for keyboard input scenarios. However, it does not explicitly contrast with alternatives like screen_type or screen_click, so no exclusions are stated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

screen_moveC

仅移动鼠标光标(需 VISION_ALLOW_SCREEN_CONTROL=1)。

ParametersJSON Schema
NameRequiredDescriptionDefault
xYes
yYes

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It discloses the environment variable requirement, but it does not explain coordinate semantics (absolute vs. relative), the origin of coordinates, potential side effects, or failure modes. Important behavioral details remain ambiguous.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence that immediately states the action and the key prerequisite. It is appropriately sized for a simple tool, avoids redundancy, and front-loads the essential purpose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Although the tool is simple, the description and schema together lack essential context such as the coordinate system, whether the move is absolute, and expected error behavior. Given the absence of an output schema and annotations, this is insufficient for an agent to reliably invoke the tool with correct parameters.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters1/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema has two numeric parameters with 0% description coverage, and the description does not clarify what x and y represent. It only indicates that the cursor will be moved, offering no guidance on coordinate interpretation, units, or whether the values are absolute or relative.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description explicitly states the operation: 'only move the mouse cursor,' which is a specific verb+resource. The prefix 'only' clearly differentiates it from sibling tools like screen_click and screen_drag, making its purpose unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description mentions a required environment variable (VISION_ALLOW_SCREEN_CONTROL=1) but provides no guidance on when to use this tool versus alternatives such as screen_click or screen_drag. No alternative tools are referenced, and no use cases are described, leaving the agent to infer usage from the name alone.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

screen_scrollA

滚轮滚动(需 VISION_ALLOW_SCREEN_CONTROL=1)。正数向上,负数向下。

ParametersJSON Schema
NameRequiredDescriptionDefault
deltaYes滚动格数(正=上,负=下)

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden. It discloses the direction convention and an environment variable requirement, which are useful behavioral details not present in the schema. However, it does not mention potential error behavior or whether scrolling targets the active element.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two short sentences, front-loaded with the action and prerequisite, and contains no unnecessary words. Every clause adds value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple tool with one parameter and no output schema, the description covers the core behavior, direction, and a key prerequisite. It lacks details on return values or failure modes, but these are likely not critical for a scroll action, and the given context is sufficient for invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema already covers the delta parameter with a description ('滚动格数(正=上,负=下)') that matches the tool description's direction info. Since schema coverage is 100%, the description adds no new parameter-level meaning beyond what is already present.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly identifies the action as wheel scrolling ('滚轮滚动') with specific direction semantics (positive up, negative down). This distinguishes it from sibling tools like screen_click or screen_move, making the purpose unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides a clear prerequisite (VISION_ALLOW_SCREEN_CONTROL=1) that indicates when the tool is permitted to operate. It does not explicitly mention alternatives or exclusions, but the context is clear enough for a simple scroll action.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

screen_typeA

键盘输入文本(需 VISION_ALLOW_SCREEN_CONTROL=1)。ASCII 直接按键;中文等经剪贴板粘贴。

ParametersJSON Schema
NameRequiredDescriptionDefault
textYes

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavior disclosure. It reveals important behavioral traits: the environment variable requirement and the clipboard fallback for non-ASCII characters. This goes beyond a simple 'types text' and gives the agent actionable details. It does not cover error cases or focus management, but for a simple input tool this is substantial.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences, with the core purpose front-loaded in the first three characters. Every word earns its place: the prerequisite and behavioral split are both critical. No fluff or repetition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity (one parameter, no output schema), the description provides sufficient context for correct invocation: what it does, when it works (env var), and how it handles different text types. The only gap is lack of details about focus management or post-input behavior, but these are not essential for basic usage.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema has 0% description coverage, so the description must compensate. It clearly explains the 'text' parameter's role: the text to input, with ASCII characters pressed directly and Chinese (or other non-ASCII) pasted via clipboard. This adds meaningful semantics beyond the bare schema, although it omits details like length limits or format requirements.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's function: '键盘输入文本' (keyboard input text), specifying the verb (input) and resource (text). It also distinguishes from sibling tools like screen_click or screen_scroll by focusing on text entry, and the special ASCII vs clipboard behavior further differentiates it from screen_key.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides an explicit prerequisite: VISION_ALLOW_SCREEN_CONTROL=1, indicating when the tool can be used. It also explains the two input methods (ASCII direct, Chinese via clipboard), which implies when each mode applies. However, it does not explicitly name alternative tools or exclusions, so it lacks a direct 'use this instead of X' comparison.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

som_locateA

Set-of-Mark 编号网格递归定位:叠加编号标记,模型回答目标所在编号,逐轮裁切放大收敛;final=box(默认)时末轮在局部图上直接输出坐标框,精度远高于整图直接定位。对无 grounding 训练的通用 VLM(MiMo 等)比直接输出坐标更准。

ParametersJSON Schema
NameRequiredDescriptionDefault
gridNo网格划分 [列数, 行数],默认 [3,3],范围 1-12
colorNofinal=cv 时的颜色提示(颜色名或 [r,g,b]),如 red/green
finalNo末轮模式:box=局部图直接输出坐标框(默认);number=全部选编号;cv=颜色分割精定位(备选,需 color,像素级)
imageYes
coordsNo返回坐标单位:pixel(默认)或 norm(0-1000 归一化)
expandNo每轮裁切边缘外扩比例,默认 0.15,范围 0-0.5
roundsNo递归轮数(每轮一次视觉调用),默认 2,范围 1-5
targetYes要定位的目标,如:蓝色提交按钮 / 红色圆形 / 报错文字
out_pathNo保存带编号标记的图(必须位于输出目录内)

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden. It discloses the iterative process, the final box mode, and the accuracy advantage. However, it does not mention side effects (e.g., saving annotated images via out_path), the number of vision calls per round, or potential limitations, leaving behavioral transparency incomplete.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two dense sentences with zero filler. The first sentence front-loads the tool's purpose and core algorithm; the second adds a usage guideline. Every clause earns its place, making it highly concise and well-structured for a technical tool.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool has 9 parameters and no output schema, yet the description only explains the high-level algorithm. It does not clarify the return format (pixel vs norm, relative to which image), how the 'box' output is expressed, or how to interpret results after multiple rounds. This is a significant gap for a complex tool, leaving the agent without critical operational details.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 89%, so the baseline is 3. The description's mention of final modes (box/number/cv) and grid recursion aligns with schema descriptions but does not add new parameter-level semantics. Parameters like grid, rounds, expand, and coords are already well documented in the schema, and the description does not elaborate further.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states a specific method: Set-of-Mark numbered grid recursive localization. It details the process (overlay numbers, model answers number, iterative crop/zoom, final box output) and contrasts it with direct coordinate localization, distinguishing it from generic locate tools like locate_object or cv_locate.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides a clear usage context: it is more accurate than direct coordinate output for general VLMs without grounding training (e.g., MiMo). This implies when to prefer this tool. However, it does not explicitly name alternative sibling tools or provide exclusion conditions ('use this when... not when...'), falling short of full explicit guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

ui_locateA

UI 元素定位(文本锚定优先):目标描述 → 关键词 → OCR 文本匹配 → 控件框(像素级)。按钮/输入框/图标等 UI 点击类目标的备选精定位;返回 matched + 候选列表供 VLM 确认。

ParametersJSON Schema
NameRequiredDescriptionDefault
typeNo目标类型过滤,可选
imageYes
coordsNo返回坐标单位:pixel(默认)或 norm(0-1000 归一化)
targetYes目标描述,如:点击「登录」按钮 / 搜索框输入 hello / 右上角图标

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It discloses the OCR-based text-anchoring process, pixel-level bounding box output, and the fact that it returns a candidate list for VLM confirmation. This goes beyond just saying 'locates elements' and provides useful behavioral insight.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise and front-loaded with the core purpose, followed by usage context and return information. It uses two sentences without filler, effectively packing method, use case, and output details.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 4-parameter tool with no output schema, the description covers the method, use case, and high-level return (matched + candidate list). It doesn't detail exact output structure or failure modes, but the given information is sufficient for an agent to select and preliminarily invoke the tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema provides descriptions for 3 of 4 parameters (75% coverage). The description adds meaning to the `target` parameter by explaining the interpretation flow (target description → keywords → OCR text matching), which complements the schema's literal parameter descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool locates UI elements via OCR text matching, with a specific method pipeline (target description → keywords → OCR → bounding box). It also distinguishes itself from siblings like cursor_locate or cv_locate by focusing on clickable UI targets and returning a candidate list for VLM confirmation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description specifies that this is an alternative precise localization method for clickable UI targets (buttons, input boxes, icons), giving clear context for when to use it. However, it does not name alternative tools or explicitly state when not to use it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

ui_parseA

全屏 UI 结构化解析:OCR 文本块 + 矩形控件检测(button/input)+ 图标候选 + 可选 YOLO 检测器(OmniParser icon_detect,models/icon_detect.pt 存在时自动启用),输出带 id 的结构化元素列表。out_path 保存半透明叠加层渲染图(编号框),可直接交 VLM 做编号选择。

ParametersJSON Schema
NameRequiredDescriptionDefault
imageYes
coordsNo返回坐标单位:pixel(默认)或 norm(0-1000 归一化)
out_pathNo保存叠加层渲染图(半透明框+编号,必须位于输出目录内)

TDQS

A3.9/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

无 annotations,描述承担披露责任。它明确说明了 YOLO 检测器在 models/icon_detect.pt 存在时自动启用、输出带 id 的结构化元素列表、out_path 保存半透明叠加层编号图等行为信息,这些都是超出工具名的实际执行细节。但未提及是否只读、失败行为或输出字段细节,因此未到满分。

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

一段长句用分号分隔,信息密度高,主要功能前置,每部分都有必要信息。但语句较长,未采用分点或短句组织,可读性略低于极简两句话的示例,属于高效但不够精致。

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

工具较复杂(OCR+控件检测+图标+可选 YOLO+输出文件),且无 output schema。描述说明了输出为带 id 的结构化元素列表、叠加层图保存的核心用途、YOLO 自动启用条件,足以让 agent 理解如何调用。但未提供元素字段结构或错误场景,完整度略低于满分。

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

schema 已覆盖 coords 和 out_path 并提供说明,image 参数无 schema 描述。描述补充了 image 应为全屏 UI 截图,并解释了 out_path 的叠加层编号图用途,但未说明 image 的具体格式或元素列表字段结构。在 67% schema 覆盖基础上,描述增加的语义有限,符合基线 3。

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

描述以“全屏 UI 结构化解析”开头,明确动词和资源,并列举 OCR 文本块、button/input 矩形检测、图标候选、可选 YOLO 检测器,输出带 id 的结构化元素列表。这与 ocr_image、locate_object 等兄弟工具形成清晰区分,功能定位具体且明确。

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

描述提供了核心适用场景(全屏 UI 解析、输出编号元素供 VLM 选择)以及可选 YOLO 自动启用的条件,但未明确说明何时不使用本工具或与 ui_locate、ui_refine 等替代工具的取舍。使用场景主要是隐含的,缺少显式排除或替代建议。

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

ui_refineA

VLM 审查并语义修正 UI 检测框:删除误检 / 合并重复 / 语义标注 / 文本锚定补漏。不做坐标微调(坐标级修正由 som/cv 负责)。返回修正后元素列表与变更记录。

ParametersJSON Schema
NameRequiredDescriptionDefault
imageYes
coordsNo返回坐标单位:pixel(默认)或 norm(0-1000 归一化)
out_pathNo保存审查用叠加层图(必须位于输出目录内)

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It clearly describes the destructive/transformative operations (deleting, merging, labeling, adding text anchors) and what the tool does NOT do (coordinate fine-tuning). It also discloses the return value (corrected element list and change log). However, it does not discuss side effects on input files, failure modes, or prerequisites beyond the image, so it is not fully transparent.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences with no filler. It front-loads the main purpose, then lists operations, then provides an exclusion, and ends with the return type. Every clause earns its place, making it highly efficient and well-structured.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description explains the tool's purpose, operations, and return value, but fails to clarify a critical aspect: how are the UI detection boxes provided as input? The schema only includes an image parameter, yet the tool is supposed to correct detection boxes. This is a significant gap that leaves the agent uncertain whether boxes are drawn in the image, passed via internal state, or expected to be detected by the VLM itself. Given the moderate complexity and lack of output schema, the description should have addressed this.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 67% (coords and out_path have descriptions, image does not). The tool description does not add any parameter-specific meaning beyond the schema. The baseline for this coverage is 3, and the description does not improve it, so a score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's function with a specific verb: 'VLM 审查并语义修正 UI 检测框' (reviews and semantically corrects UI detection boxes). It lists concrete operations (delete false positives, merge duplicates, semantic labeling, text anchor gap-filling) and explicitly distinguishes itself from coordinate-level correction, which is handled by som/cv. This differentiates it clearly from sibling tools like cv_locate and som_locate.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides an explicit exclusion: '不做坐标微调(坐标级修正由 som/cv 负责)' (does not do coordinate fine-tuning, coordinate-level correction is handled by som/cv), naming the alternative tool for that use case. It implies usage after initial detection to clean up false positives and duplicates, but does not fully enumerate all possible alternatives like ui_parse or ui_locate.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

vision_healthA

检查视觉后端配置与连通性。

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

A3.7/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries full burden. It conveys the core behavior (checking config and connectivity), and the verb 'check' implies a read-only operation. However, it does not explicitly state whether it modifies anything, what it returns, or any side effects, leaving some ambiguity.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single short sentence that directly conveys the purpose without any wasted words. It is perfectly concise for a tool with no parameters.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Although the tool is simple with no parameters, there is no output schema to explain return values. The description does not clarify whether the health check returns a boolean, status details, or an error, making it somewhat incomplete for an agent that needs to interpret results.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters, so the baseline is 4. The description does not need to explain parameter details, and the empty schema covers all parameters trivially.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool checks visual backend configuration and connectivity, using a specific verb ('check') and resource. It distinguishes itself from sibling vision tools (e.g., OCR, image description) by focusing on health/status rather than image processing.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is given on when to use this tool versus alternatives. It does not state that it should be called before other vision tools to verify backend availability, nor does it mention any exclusions or alternative tools for similar health checks.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

zoom_regionB

放大图片指定区域(默认整图 2 倍),保存并返回路径。

ParametersJSON Schema
NameRequiredDescriptionDefault
boxNo[x1,y1,x2,y2],省略则放大整图
imageYes
scaleNo放大倍数 1-8,默认 2
coordsNo坐标单位,默认 pixel
out_pathNo

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden. It discloses that the tool saves the result and returns a path, which is useful. However, it does not mention side effects (e.g., whether the original image is modified), error conditions, or the exact output format beyond 'path.'

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, concise sentence that is front-loaded with the core action and includes the two most important defaults. Every word earns its place with no unnecessary repetition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool has 5 parameters and no output schema, but the description covers the main behavior (zoom, save, return path). Missing details include coordinate system behavior, out_path handling, and edge cases, though some of this is captured in the schema. Overall, it is adequate but not rich.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 60%, and the description only rephrases the defaults already present in the schema (e.g., box omitted = whole image, scale default = 2). It adds no new meaning for parameters like coords or out_path, so the value beyond the schema is minimal.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's function: zoom into a specified region of an image, defaulting to the whole image at 2x, then save and return the path. This is a specific verb+resource combination that distinguishes it from similar tools like crop_image or ocr_image.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. There is no mention of exclusions, prerequisites, or comparison to sibling tools, leaving the agent to infer the appropriate context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

TDQS

B3/5.0
Disambiguation2/5

Multiple tools overlap significantly in purpose: locate_object, som_locate, cursor_locate, and cv_locate all return coordinates; describe_image, analyze_image, and compare_image/compre_infer all analyze visual content. While each has a distinct method, an agent would struggle to choose the right one without carefully parsing dense descriptions.

Naming Consistency2/5

Tool names mix conventions: some use verb_noun (describe_image, crop_image), others use prefix_verb (ui_locate, cv_locate, som_locate), and a few are noun-y (vision_health, reason_graph). Even within the screen_* group, screen_capture vs screen_click are inconsistent in verb placement. This makes the naming unpredictable.

Tool Count2/5

With 27 tools, the server exceeds the range where each tool feels essential. Several tools (e.g., cursor_locate vs som_locate, compare_infer vs compare_images) could be merged, and the count feels heavy for a 'vision primitives' server.

Completeness4/5

The toolkit covers a broad range: OCR, description, structured analysis, localization, annotation, cropping, UI parsing, multi-image comparison, anomaly scanning, and screen control. There are few obvious gaps for a vision-oriented server, though some overlapping implementations suggest room for consolidation.

Maintenance

ActivityMaintained
ResponsivenessSyncing

Resources

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Related MCP Connectors

Related MCP Servers

  • F
    license
    A
    quality
    C
    maintenance
    An MCP server that adds visual understanding to text-only LLMs via image understanding, OCR, and image comparison tools, with multi-provider fallback and context-aware Focus Hint for precise descriptions.
    3
  • F
    license
    A
    quality
    C
    maintenance
    An MCP server for image understanding via OpenAI-compatible vision models, offering tools for OCR, error screenshot diagnosis, technical diagram reading, data visualization analysis, UI-to-code conversion, and UI diff comparison.
    7

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/zouyuanqing/vision-primitives-mcp'

If you have feedback or need assistance with the MCP directory API, please join our Discord server