Skip to main content
Glama
bigduu

Nova

by bigduu
OverviewSchemaSourceRelated ServersDiscussionsAdmin
Rust
Local

ocr

Read on-screen text via platform OCR, returning recognized lines with clickable coordinates. Use it on canvas, games, or custom-drawn views where accessibility trees are empty or partial.

Instructions

Read on-screen TEXT via the platform OCR engine (Apple Vision on macOS, Windows.Media.Ocr on Windows). Captures the display (or window="") and returns the recognized text lines, each with a clickable center in the same pixel space as a screenshot — so you can both READ the text and click a line with left_click(x, y). Returns text only (no image), so it is a cheap, fast way to pull text off the screen. Best when use it after ax_read reports absent/partial coverage and you need rendered TEXT from a surface where semantic nodes are empty or sparse — canvas, games, image-rendered or custom-drawn views — or to grab a lot of text at once without parsing a screenshot. For native/web UI with an Accessibility/UIA tree, ax_read + ax_activate is still more precise. mode=auto (default) tries fast recognition first and only falls back to accurate for empty/low-confidence output; mode=fast minimizes latency and mode=accurate forces maximum quality. roi={x,y,width,height} scopes OCR in the CURRENT image's pixels: the rectangle is strictly validated, mapped to the desktop, and re-captured through the native region path (never cropped from a downscaled JPEG). roi and window are mutually exclusive. Languages default to Simplified Chinese + English; pass languages=[...] (BCP-47) to override.

Input Schema

NameRequiredDescriptionDefault
roiNoOptional strict region of interest in the CURRENT image's pixel space. Nova maps it back to global logical coordinates and re-captures through the platform's native region path before JPEG encoding/OCR. It is never cropped from an already-downscaled JPEG. Every value must be finite, the size positive, and the entire rectangle in bounds. Mutually exclusive with `window`.
modeNoRecognition latency/quality policy. `auto` (default) starts with the fast Apple Vision recognizer and falls back to accurate only for empty or low-confidence output. On engines without a native mode knob the setting preserves that engine's normal behavior.
windowNoCapture only a single on-screen window (case-insensitive substring of its title or app name) instead of the whole display. Smaller, sharper image → better recognition of small text.
languagesNoBCP-47 language hints in priority order (e.g. ["zh-Hans", "en-US"]). Omitted, defaults to Simplified Chinese + English.

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed4 schema fields changedv0.3.0
    • addedInput schema / $defs
      Added value: +{
+  "OcrModeParam": {
+    "oneOf": [
+      {
+        "const": "auto",
+        "description": "Fast first; retry accurately only for empty/low-confidence Vision output.",
+        "type": "string"
+      },
+      {
+        "const": "fast",
+        "description": "Lowest latency, potentially lower recognition quality.",
+        "type": "string"
+      },
+      {
+        "const": "accurate",
+        "description": "Highest quality, potentially slower.",
+        "type": "string"
+      }
+    ]
+  },
+  "OcrRoiParams": {
+    "properties": {
+      "height": {
+        "description": "ROI height in current-image pixels; must be finite and > 0.",
+        "format": "double",
+        "type": "number"
+      },
+      "width": {
+        "description": "ROI width in current-image pixels; must be finite and > 0.",
+        "format": "double",
+        "type": "number"
+      },
+      "x": {
+        "description": "Left edge in the CURRENT image's pixel space.",
+        "format": "double",
+        "type": "number"
+      },
+      "y": {
+        "description": "Top edge in the CURRENT image's pixel space.",
+        "format": "double",
+        "type": "number"
+      }
+    },
+    "required": [
+      "x",
+      "y",
+      "width",
+      "height"
+    ],
+    "type": "object"
+  }
+}
    • addedInput schema / properties / mode
      Added value: +{
+  "anyOf": [
+    {
+      "$ref": "#/$defs/OcrModeParam"
+    },
+    {
+      "type": "null"
+    }
+  ],
+  "description": "Recognition latency/quality policy. `auto` (default) starts with the\nfast Apple Vision recognizer and falls back to accurate only for empty\nor low-confidence output. On engines without a native mode knob the\nsetting preserves that engine's normal behavior."
+}
    • addedInput schema / properties / roi
      Added value: +{
+  "anyOf": [
+    {
+      "$ref": "#/$defs/OcrRoiParams"
+    },
+    {
+      "type": "null"
+    }
+  ],
+  "description": "Optional strict region of interest in the CURRENT image's pixel space.\nNova maps it back to global logical coordinates and re-captures through\nthe platform's native region path before JPEG encoding/OCR. It is never\ncropped from an already-downscaled JPEG. Every value must be finite, the\nsize positive, and the entire rectangle in bounds. Mutually exclusive\nwith `window`."
+}
    • removedInput schema / title
      Removed value: -"OcrParams"
  2. First observedv0.1.0

TDQS

A4.6/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and does well: it discloses the return shape (text lines with clickable centers in screenshot pixel space), that no image is returned, the cheap/fast tradeoff, and the ROI re-capture path. It omits operational caveats like required screen-capture permissions and failure/error behavior, so it stops short of a 5.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Purpose and the cost/return profile are front-loaded, and the sentences are dense with operational detail. It runs long as a single block and repeats some schema content, but nearly every clause adds actionable information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Despite no output schema, the description explains exactly what comes back (recognized lines with clickable centers usable via left_click) plus engine selection, modality, and param interactions. An agent has everything needed to call it and use the result correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description adds real meaning beyond it: the roi/window mutual exclusion, the strict validation and native re-capture of roi (never cropped from a downscaled JPEG), the auto/fast/accurate fallback policy, and the default language pair.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb+resource ('Read on-screen TEXT via the platform OCR engine') and immediately names the concrete engines. It also distinguishes itself from siblings by declaring it returns text only and routes UI-tree cases to ax_read + ax_activate.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives explicit when-to-use ('after ax_read reports absent/partial coverage', canvas/games/image-rendered views, bulk text grabbing) and explicit when-not-to ('For native/web UI with an Accessibility/UIA tree, ax_read + ax_activate is still more precise'). Alternatives and selection conditions are both present.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Deploy Server

Other Tools