Skip to main content
Glama

vision_detect

Parse screen-parsing detection JSON into AI-friendly elements with bounding boxes and center points for GUI agent control. Accepts inline or file input and returns window-relative coordinates.

Instructions

Parse screen-parsing detections (OmniParser v2 compatible) into AI-friendly elements with bbox/center. Read-only; no model runs on this host.

Tool call template

This tool declares no required parameters. Add optional parameters from the input schema as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {},
    "name": "vision_detect"
  }
}

Input Schema

NameRequiredDescriptionDefault
scoreNo
origin_xNowindow top-left inside the image, X; subtracted to reach window-relative coords
origin_yNowindow top-left inside the image, Y; subtracted to reach window-relative coords
detectionsNoinline detection JSON (used when detections_path is absent)
max_resultsNo
detections_pathNopath to a detection JSON file

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observedv0.2.0

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations supplied, the description carries the full burden, and it does disclose the two most important traits: read-only and no model execution on this host. It does not cover error behavior for malformed/missing detection JSON, whether the score default filters results, or how detections vs detections_path precedence is resolved.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two dense clauses with no filler, and the core purpose is front-loaded before the behavioral qualifier. Nothing is redundant, though the parenthetical version tag slightly interrupts the primary statement.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a six-parameter, zero-required tool with no output schema and no annotations, the description is minimal: it establishes what the tool is and that it is safe, but leaves the agent to infer input source precedence, score filtering, and result-count behavior from the schema alone.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 67%, so the baseline of 3 applies. The description adds some meaning by pinning the expected detection format to OmniParser v2 and by describing the return elements (bbox/center), but it says nothing about score thresholding or max_results truncation, which are the parameters most likely to change results.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

Names a specific verb ('Parse') and resource ('screen-parsing detections, OmniParser v2 compatible') and states the output shape ('AI-friendly elements with bbox/center'). The clause 'no model runs on this host' implicitly separates it from a detection-producing sibling like detect_elements, though no sibling is named explicitly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is only implied: the tool consumes already-produced detection JSON, which conveys that it is a post-processing step rather than a scanning step. There is no explicit when-to-use/when-not guidance, no statement of alternatives, and no prerequisites for obtaining the detection input.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.