Skip to main content
Glama

Read screen text

read_screen_text

Extracts visible on-screen text with on-host OCR, including images, canvases, WebViews, and native dialogs, and returns each text run with tap coordinates.

Instructions

Read the visible text off the screen with on-host OCR (Apple Vision). Works without a Metro connection and sees text the layout tree cannot: images, canvases, WebViews, native dialogs. Returns one line per recognized text run as <text> @x,y — x,y is the centre in tap units and can be passed straight to tap {x,y}. Lines that also exist in the app's JS layout are marked [tree]: prefer tap {text} for those, since an OCR box is just text, not a control. Optional: search (substring filters, case-insensitive), languages (e.g. ["de","en"]; auto-detected by default), compare_tree (default true). macOS host only; the first call compiles a small Vision helper (~10–30s, cached).

Input Schema

NameRequiredDescriptionDefault
deviceNo
searchNo
languagesNo
compare_treeNo

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observedv0.1.0

TDQS

A4.9/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden and does: it discloses the OCR engine (Apple Vision), that it runs on-host, the macOS-only platform constraint, and a first-call latency cost (~10-30s helper compile, then cached). It also explains the return line format and the [tree] marker semantics — none of which any structured field provides.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Dense but front-loaded: purpose first, then differentiators, then output format, then optional params, then platform/latency caveats. Every sentence carries information an agent needs — no filler and no restatement of the name.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema, and the description compensates by specifying the exact line format (`<text> @x,y` in tap units) and the [tree] marker. Combined with platform limits, cold-start latency and parameter defaults, an agent has everything needed to call and interpret this tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate, and it does for three of four params: search is substring/case-insensitive, languages takes codes like ["de","en"] and defaults to auto-detect, compare_tree defaults true. The `device` parameter is never mentioned, leaving one of four params undocumented in both schema and prose.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb+resource ("Read the visible text off the screen with on-host OCR") and immediately scopes it against siblings: it sees text the layout tree cannot (images, canvases, WebViews, native dialogs). An agent can distinguish this from get_component_tree, get_layout and screenshot without opening any schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives explicit when-to-use (works without a Metro connection; text the layout tree can't see), and explicit routing rules for interpreting results: lines marked [tree] should be handled with tap {text}, OCR boxes with tap {x,y}. Alternatives are named, not implied.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.