Skip to main content
Glama

ui_parse

Parse full-screen UI into structured elements: OCR text, buttons, inputs, icons, optional YOLO. Outputs ids and numbered overlay for VLM reference.

Instructions

全屏 UI 结构化解析:OCR 文本块 + 矩形控件检测(button/input)+ 图标候选 + 可选 YOLO 检测器(OmniParser icon_detect,models/icon_detect.pt 存在时自动启用),输出带 id 的结构化元素列表。out_path 保存半透明叠加层渲染图(编号框),可直接交 VLM 做编号选择。

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
imageYes
coordsNo返回坐标单位:pixel(默认)或 norm(0-1000 归一化)
out_pathNo保存叠加层渲染图(半透明框+编号,必须位于输出目录内)
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

无 annotations,描述承担披露责任。它明确说明了 YOLO 检测器在 models/icon_detect.pt 存在时自动启用、输出带 id 的结构化元素列表、out_path 保存半透明叠加层编号图等行为信息,这些都是超出工具名的实际执行细节。但未提及是否只读、失败行为或输出字段细节,因此未到满分。

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

一段长句用分号分隔,信息密度高,主要功能前置,每部分都有必要信息。但语句较长,未采用分点或短句组织,可读性略低于极简两句话的示例,属于高效但不够精致。

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

工具较复杂(OCR+控件检测+图标+可选 YOLO+输出文件),且无 output schema。描述说明了输出为带 id 的结构化元素列表、叠加层图保存的核心用途、YOLO 自动启用条件,足以让 agent 理解如何调用。但未提供元素字段结构或错误场景,完整度略低于满分。

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

schema 已覆盖 coords 和 out_path 并提供说明,image 参数无 schema 描述。描述补充了 image 应为全屏 UI 截图,并解释了 out_path 的叠加层编号图用途,但未说明 image 的具体格式或元素列表字段结构。在 67% schema 覆盖基础上,描述增加的语义有限,符合基线 3。

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

描述以“全屏 UI 结构化解析”开头,明确动词和资源,并列举 OCR 文本块、button/input 矩形检测、图标候选、可选 YOLO 检测器,输出带 id 的结构化元素列表。这与 ocr_image、locate_object 等兄弟工具形成清晰区分,功能定位具体且明确。

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

描述提供了核心适用场景(全屏 UI 解析、输出编号元素供 VLM 选择)以及可选 YOLO 自动启用的条件,但未明确说明何时不使用本工具或与 ui_locate、ui_refine 等替代工具的取舍。使用场景主要是隐含的,缺少显式排除或替代建议。

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Install Server

Other Tools

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/zouyuanqing/vision-primitives-mcp'

If you have feedback or need assistance with the MCP directory API, please join our Discord server