Skip to main content
Glama

ui_ocr

Read-only

Reads text from windows with almost no UI tree via Windows OCR, returning each line with a reference and screen rectangle so agents can target text-based click actions.

Instructions

Only when ui_tree shows almost nothing (games, Qt, Sciter, canvas UIs): the window's text read by Windows OCR, one line per text with a ref (t3) and screen rect. ui_act takes a t-ref for click, double-click, right-click, hover, wheel and drag, posted at the text's center as mouse messages (no real cursor, no focus change). About 0.5 to 2 s; works while covered. t-refs expire after 60 s.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "title": "<title>"
    },
    "name": "ui_ocr"
  }
}

Input Schema

NameRequiredDescriptionDefault
titleYespart of the window title (case-insensitive; an exact title wins) or a handle from list_windows (0x...)

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observedv0.1.0

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already mark this as read-only and non-open-world, and the description adds substantial behavior beyond that: output shape, t-ref interaction through ui_act, mouse messages without cursor movement or focus changes, latency, visibility while covered, and 60-second ref expiry. These details materially help an agent use the result correctly.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded with the fallback condition and core OCR behavior, then efficiently adds the downstream t-ref use and operational limits. Every sentence supplies useful context, and there is no redundant or filler wording.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a read-only OCR tool with no output schema, the description explains the return format, ref expiry, approximate runtime, and how refs are consumed by ui_act. Combined with the fully described title parameter, it is complete enough for an agent to invoke and interpret the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has full description coverage for the single title parameter, including partial-title and handle behavior, so the schema already carries parameter semantics. The description does not add meaning about the title argument beyond what the schema provides, making the baseline 3 appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific action and resource: reading a window's text via Windows OCR and returning text lines with t-refs and screen rects. It explicitly frames this as a fallback when ui_tree shows almost nothing, which distinguishes it from the sibling UI-tree approach.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives an explicit usage condition: only when ui_tree shows almost nothing, naming affected UI types such as games, Qt, Sciter, and canvas UIs. That effectively states when to use it and when another approach is preferable.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.