AInotate
Attaches over CDP to an existing Arc browser session (one already logged in), captures the tab, applies viewport/device presets and annotates the result, then restores the tab.
Attaches over CDP to an existing Brave browser session to capture and annotate web pages, applying the preset to that single tab and restoring it afterwards.
Drives a Playwright-managed Firefox instance along with Chromium and WebKit to open pages, run actions (click, fill, wait, scroll) and capture screenshots for annotation.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@AInotateScreenshot https://example.com and number the Sign up button as step 1"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
Annotated screenshots for AI agents: the agent shows you the button instead of describing where it is.
Website · Get started · Privacy

Ask an agent how to post a link on Hacker News and you usually get directions: the orange bar at the top, the last link after jobs. With AInotate it sends the picture above instead and says "click submit, where the blue arrow points".
It captures a web page, a window or a screenshot you already have, finds the elements you mean, draws numbered steps, arrows, boxes and labels where they hide the least, blacks out emails and tokens, and looks at the result before sending it. It works in Claude Code, Cowork, Claude Desktop, Codex, Gemini CLI, Cursor and any other MCP client, and from the command line or Python.
We built it at DontPayFull because our own agents kept answering "where is it?" with a paragraph. A picture with a number on the right button settles the question in a second, in a chat, a ticket or a guide.
What it is for
"Where is it?" Settings, menus, buttons that hide behind icons. The agent answers with the screen itself, the control boxed and numbered.
"Walk me through it." One image per step, in the order to follow;
ainotate guideturns them into a Markdown, HTML or PDF guide."Your turn." A permission switch, a 2FA code, a payment or consent screen: the agent does not click for you. It shows exactly what to press and says what the click does.
Bug reports and tickets. What is wrong, outlined in red, what is right in green, emails and tokens blacked out before anyone sees them.
Docs and changelogs. Screenshots that point at the thing the text talks about, re-shot from the same spec when the UI changes; before and after plates for a change.
Related MCP server: Screenshot API
Set it up in your agent
1. Install, one of:
With | Command |
Homebrew (macOS) |
|
pipx or uv (Python 3.10+) |
|
Claude plugin | only uv: step 2 brings AInotate along |
Claude Desktop extension | nothing to install first |
Then ainotate doctor checks the machine and prints the exact fix for
anything missing, including the one command that downloads Chromium for
web capture. To update later: brew upgrade ainotate, pipx upgrade ainotate or uv tool upgrade ainotate; the plugin with /plugin marketplace update ainotate; the Desktop extension by opening the newer
.mcpb from Releases.
2. Connect it to the agent you use:
Agent | How |
Claude Code |
|
Cowork | Customize > Plugins > Add marketplace > |
Claude Desktop | download |
Codex |
|
Gemini CLI |
|
Cursor |
|
VS Code, Cline, other MCP clients | command |
{"mcpServers": {"ainotate": {"command": "ainotate", "args": ["mcp"]}}}AInotate runs on your own computer: Cowork reaches it through the Claude
desktop app, so keep the app open while a task uses it. The plugin runs
AInotate with uvx, pinned to its release, so it needs
uv.
The skill teaches an agent when and how to annotate: pick a source, find exact positions, annotate, read the image to verify, deliver. The plugin brings it along; without the plugin it ships with the package:
ainotate install-skill # Claude Code and Codex (~/.claude/skills, ~/.agents/skills)
ainotate install-skill --zip ~/Desktop # a ZIP for Claude Desktop, Cowork, claude.ai:
# Customize > Skills > Upload a skill
npx skills add dontpayfull/AInotate # any agent the skills CLI knows (Cursor, Windsurf, ...)Config paths for every OS, permissions and the tool list: skills/ainotate/references/mcp.md.
3. Ask. "Show me where to switch Wikipedia to dark mode." The agent captures the page, annotates it, checks it and sends the image. With the skill it also does this on its own whenever an answer depends on where something is on the screen: a "how do I" question, a visual bug, a before/after of a change.
Gallery
Every image below was made in one shoot call against a public site:
open the page, find the elements, place the labels, draw, frame. The specs
are in docs/images/specs.
|
|
Numbered steps on the busy header of DontPayFull, coupons for 20,000+ stores | Box, arrow and highlight on GitHub |
|
|
Step-by-step guide with browser chrome and a gradient frame | Bug report, emails redacted automatically by the privacy scan |
|
|
Magnifier and keycaps for tiny controls and shortcuts | Before / after plate from two shots |

Labels that stay off the content

A label dropped at a fixed spot next to its target usually lands on the text the reader needs. AInotate places every label itself:
It maps the page content first (text, icons, lines, images) and tries hundreds of spots per label. Empty space near the target wins; a spot over text is used only when the frame has no free room.
Arrows are priced too: a long arrow, or one that crosses text, another label or another mark, loses to a shorter, cleaner one. When the straight path crosses text, the arrow curves around it.
All labels are planned together, so the first one cannot take the only clean spot a later one needs.
Step badges sit on the corner of their box that covers the least.
The image is never enlarged to make room. If a label still has to cover
something, AInotate says so in a warning, and label_at pins a label
exactly where you want it.
From the command line
Save this as hn.json:
{"url": "https://news.ycombinator.com",
"frame": {"chrome": "browser"},
"marks": [
{"type": "step", "n": 1, "target": {"text": "new", "exact": true},
"label": "Newest"},
{"type": "step", "n": 2, "target": {"text": "login"},
"label": "Sign in"}]}ainotate shoot hn.json --draft # temp file; drop --draft to saveThe output path is printed on stdout. Images are saved to
~/Pictures/AInotate unless you configure another folder. ainotate --help
lists every command (capture, annotate, locate by OCR, grid, zoom, window
capture with UI element targets, elements, guide, compare, animate, copy,
install-skill); the full spec is in
skills/ainotate/references/spec.md. Exit codes: 0 ok,
1 cannot save, 2 invalid spec, 3 cannot draw, 4 capture failed or an
optional dependency is missing, 5 ambiguous text target, 6 target not
found.
Python. The same pipeline is importable:
from ainotate.shoot import shoot
res = shoot({"url": "https://en.wikipedia.org/wiki/Screenshot",
"marks": [{"type": "box",
"target": {"text": "View history"},
"label": "Past edits"}]}, draft=True)
print(res.paths[0], res.redactions)Features
Marks: numbered
step,box,arrow,clickripple,keys(keycaps),magnify(loupe),highlight,spotlight,text, solidredact, and decorativeblur/pixelate.Placement that reads well: labels go to free space near their target and are planned together; arrows are tapered Skitch-style shapes with a soft shadow that curve around text (see Labels that stay off the content). Warnings for overlapping labels, crossing arrows, more than 6 marks or labels over 4 words.
Web capture in one call:
ainotate shootopens a page (laptop, wide or phone preset), runs actions (click, fill, wait, scroll), measures each target, redacts, annotates and saves.Any browser: its own Playwright Chromium, Firefox or WebKit, or attach over CDP to a Chromium you already use and are logged into (Chrome, Edge, Brave, Arc, BrowserOS). The preset is applied to that one tab and restored afterwards.
Any image:
locatefinds text by OCR (Apple Vision, Windows OCR or Tesseract);gridandzoomlocate icons at full resolution. Screen, window and clipboard capture are built in.Desktop apps on macOS:
ainotate window ID --targetmeasures buttons, switches and fields through the Accessibility API, so marks sit exactly on the control, icons included;ainotate elements --app NAMElists what it can find.Frames: gradient backgrounds (auto from the screenshot, or presets), rounded corners, shadow, browser or window chrome, social aspect ratios.
Sharing: guides in Markdown, HTML and PDF; before/after plates; APNG and GIF animations; copy to the clipboard.
Strict specs: a typo stops the run with a list of every problem instead of saving a wrong or unredacted image. Clear exit codes.
Diacritics: bundled Inter font, so Romanian, German, French and other Latin-script labels render correctly everywhere.
Privacy
Redaction is solid.
redactpaints an opaque block. AInotate never uses blur to hide data: blur and pixelation can be reversed.blurandpixelateexist only to de-emphasize clutter, and a mark flagged"sensitive": trueis refused for them.Automatic redaction is on for web shots.
shootscans the live page (text and input values) for emails, phone numbers, card numbers, IBANs, API keys, tokens, JWTs, password and personal-data fields, and blacks them out. On images it runs on OCR when you pass--privacy auto. You can allow-list your own addresses or add regexes.It is best-effort. Detection misses things: text drawn in images or canvas, unusual formats, data split across elements, embedded frames from other sites (reported, not redacted). Always look at the image before you share it. The skill makes this check mandatory for agents.
Local. Capture, OCR and rendering run on your machine. AInotate makes no network requests of its own beyond loading the pages you ask it to capture.
Platform support
macOS | Windows | Linux | |
Annotate, frames, export | verified | experimental | experimental |
Web capture (Playwright, CDP) | verified | experimental | experimental |
Screen and window capture | verified | experimental | experimental (X11) |
UI elements ( | verified (Accessibility) | not yet | not yet |
OCR ( | verified (Vision) | experimental (Windows OCR) | experimental (Tesseract) |
Clipboard | verified | experimental | experimental |
MCP server | verified | experimental | experimental |
Experimental means implemented and unit-tested with mocks, not yet verified on real Windows or Linux machines. Reports are welcome.
License
Copyright © 2026 DontPayFull. AInotate is free software under the GNU Affero General Public License v3.0 or later. If you run a modified AInotate as a network service, the AGPL requires you to offer its source to the users of that service.
Bundled and derived third-party work (the Inter font, arrow proportions from Arrowshot, a label-placement idea from github/awesome-copilot) and dependency licenses are listed in NOTICE.
Available Tools
16 toolsanimateA
Animation from image paths, one frame per image: APNG (full color, default) or GIF (plays everywhere: Slack, mail, Jira). Use it to show a short sequence of states; for steps the user should read, make_guide is clearer. Draft by default (temp folder): check the preview, then call again with draft=false to save to the output folder. The preview shows the first frame. Writes one file; the inputs are not changed. Returns the path, frame count, format, size and a preview.
| Name | Required | Description | Default |
|---|---|---|---|
| loop | No | Repeat count; 0 = forever. | |
| name | No | Base file name in the output folder, no extension. Empty = tool default. | |
| draft | No | true (default): write to a temp folder so you can check the preview first. false: save to the output folder (~/Pictures/AInotate unless configured) and the backup folder. | |
| format | No | "apng" (full color, default) or "gif". | apng |
| frames | Yes | Image paths, one per frame, in order. Frames are scaled to the first one. | |
| duration_ms | No | Time per frame in ms (default 1500). | |
| crossfade_ms | No | Length of a blended transition between frames in ms; 0 = cut. | |
| durations_ms | No | Old name of duration_ms; used only when duration_ms is not given. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare non-read-only, non-destructive, non-idempotent, closed-world behavior, and the description adds substantial context beyond that: default draft writes to a temp folder, draft=false writes to the output folder plus a backup, only one file is written, inputs are not mutated, and the preview shows the first frame. This is exactly the kind of behavioral detail an agent needs before invoking a file-writing tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Dense but front-loaded: purpose and format first, then sibling routing, then the draft workflow, then return values. Every sentence carries distinct information with no filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
There is no output schema, and the description compensates by enumerating the return payload (path, frame count, format, size, preview). Combined with the workflow and side-effect disclosure, nothing an agent needs to call this correctly is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3, but the description adds real meaning beyond the schema for format ('APNG full color default' vs 'GIF plays everywhere: Slack, mail, Jira') and the draft flag's side effects, including the backup folder. Loop, name, and duration fields are left to the schema, which already documents them.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource (animation from image paths, one frame per image), names the two output formats and their defaults, and explicitly distinguishes itself from the sibling make_guide. An agent can identify this tool and its output without opening the schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives an explicit when-to-use ('show a short sequence of states') and when-not ('for steps the user should read, make_guide is clearer'), plus the recommended two-step workflow: draft first, inspect the preview, then re-call with draft=false. Alternatives and conditions are fully spelled out.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
annotateA
Draw marks on an existing image: numbered steps, boxes, arrows, labels, highlights, spotlight, text and solid redaction. Use it once you have an image and exact rects (from capture_web, capture_window targets, locate, or grid + zoom); for a web page in one call use shoot instead. Draft by default (temp folder): check the preview, then call again with draft=false to save to the output folder. spec: {"input": path, "scale": 1|2.., "crop": "auto"|[x1,y1,x2,y2], "name": "...", "marks": [{"type": "step"|"box"|"arrow"|"highlight"|"spotlight"|"redact"|"text"|..., "rect": {x,y,w,h} or [x1,y1,x2,y2], "label": "...", "n": 1, "color": "look|bad|good|info"}]}. Every coordinate in one unit; scale converts it to image px. Full reference: resource ainotate://spec. Writes PNG files only; never changes the input image. Returns paths, image size, warnings, legend (step numbers + labels), redactions and a preview.
| Name | Required | Description | Default |
|---|---|---|---|
| look | No | Mark style: "default" (bold, built in) or the name of an installed look package (e.g. "neat"). Omit it to use the spec's look, else the user's default (AINOTATE_LOOK, config.toml). A look that is not installed or fails falls back to default with a warning listing the installed looks. | |
| spec | Yes | Render spec as a JSON object (or a JSON string). Full schema: resource ainotate://spec. | |
| draft | No | true (default): write to a temp folder so you can check the preview first. false: save to the output folder (~/Pictures/AInotate unless configured) and the backup folder. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already disclose readOnlyHint=false, destructiveHint=false, idempotentHint=false and openWorldHint=false, so the safety profile is carried by structured data. The description adds genuinely useful context the annotations don't: draft writes go to a temp folder, draft=false persists to the output folder plus a backup folder, output is PNG only, and the input image is never modified. It stops short of disclosing failure behavior or how partially-valid specs are handled, so it is solid but not exhaustive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The critical scoping and workflow guidance is front-loaded in the first two sentences, and the reference pointer (ainotate://spec) prevents the spec dump from needing to be exhaustive. The middle section is a dense run-on packed with pipes and braces that takes effort to parse, which costs a point, but essentially every clause carries operational information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a complex, spec-driven rendering tool, the definition covers the input contract, the draft/save lifecycle, coordinate units, output location, artifact type, and return contents (paths, size, warnings, legend, redactions, preview), and links to the full spec resource for depth. Nothing an agent needs to call it correctly is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3, but the description goes further by inlining the actual spec grammar (input, scale, crop with 'auto' or coords, name, marks with type/rect/label/n/color) and by stating the key unit rule that every coordinate uses one unit and scale converts to image px. That meaning is not derivable from the schema, which only points at a resource.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Opens with a specific verb+resource ('Draw marks on an existing image') and enumerates the exact mark types (steps, boxes, arrows, labels, highlights, spotlight, text, redaction). It explicitly distinguishes itself from the sibling 'shoot' (one-call web page capture) and from simple preview tools, so an agent can select it without opening a schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
States the prerequisite condition plainly ('once you have an image and exact rects') and routes to the sources that produce them (capture_web, capture_window targets, locate, grid + zoom). It also names the alternative for the web-page case ('use shoot instead') and prescribes the draft-then-confirm workflow, covering when-not and how.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
capture_screenARead-onlyIdempotent
Screenshot the whole screen or one region, as the user sees it right now. Use it for something that spans several windows or is not in any one window; prefer capture_window for one app (no overlap from other windows) and capture_web for web pages (exact element rects). macOS needs the Screen Recording permission for the app that runs this server (Claude Desktop, the terminal...); without it the capture comes back blank and the error says how to fix it. Linux needs mss (X11) or grim (Wayland). Writes a PNG to the temp folder. Returns its path and size, and a preview; then locate elements with locate or grid + zoom and annotate with scale 1.
| Name | Required | Description | Default |
|---|---|---|---|
| region | No | Part of the screen: [x, y, w, h] in screen points, or the same four numbers as a string "x,y,w,h". Omit for the whole main display (all displays on Windows). |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations only cover the safety profile (readOnly/idempotent/non-destructive); the description adds substantial context beyond them: required OS permissions, that absence yields a blank capture with a self-explaining error, Linux backend dependencies, that a PNG is written to the temp folder, and what is returned (path, size, preview). This is unusually rich behavioral disclosure.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with purpose and alternatives, then prerequisites and returns; most sentences earn their place. The trailing chaining advice ('locate elements with locate or grid + zoom and annotate with scale 1') is useful but slightly dense and less crisp than the rest.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema exists, yet the description still states the return surface (path, size, preview) as well as prerequisites and failure behavior, so an agent knows how to call it, what to expect back, and what to do next. Nothing material is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% and the single 'region' parameter is fully documented in the schema (formats, defaults, whole-display omission). The description only restates the whole-screen/one-region scope without adding format or coordinate semantics, so the baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('Screenshot the whole screen or one region') with scope ('as the user sees it right now'), and names the siblings it differs from (capture_window, capture_web) with the distinguishing condition (overlap from other windows / exact element rects). An agent can select it without opening any other schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives an explicit positive trigger ('spans several windows or is not in any one window') and two explicit alternatives with the condition selecting each ('prefer capture_window for one app', 'capture_web for web pages'). Platform prerequisites (macOS Screen Recording, Linux mss/grim) are also stated with the failure mode.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
capture_webA
Screenshot a web page in a headless browser (Playwright) and measure named elements, so you can annotate with exact rects. Use it for web UIs; for desktop apps use capture_window, for what is on screen right now capture_screen. Pass url to load a page, or cdp_url + page_url_contains to attach to a browser that is already running and logged in (the tab is captured as is). Loads the page over the network; actions (click, fill) run in that page and can change state on the site, so only pass actions the user asked for. Presets: laptop 1280x800@2, wide 1600x1000@2, phone 390x844@3. Needs ainotate[web] and a Playwright browser (doctor checks). Returns the PNG path (temp folder), rects in CSS px, scale (use it as the spec scale), missing targets with the reason, a spec stub and a preview with the rects outlined.
| Name | Required | Description | Default |
|---|---|---|---|
| url | No | Page to load (http/https). Omit when attaching with cdp_url. | |
| preset | No | Viewport: "laptop" 1280x800@2, "wide" 1600x1000@2 or "phone" 390x844@3. | laptop |
| actions | No | Steps run in the page before the capture, in order: {"click": T}, {"fill": T, "value": "..."}, {"wait": T}, {"scroll_to": T}, {"wait_ms": 500}; T is a target as in targets. | |
| browser | No | Engine to launch: "chromium" (default), "firefox" or "webkit". | chromium |
| cdp_url | No | DevTools address of a running browser to attach to, e.g. http://127.0.0.1:9222; captures its logged-in tab instead of launching a new browser. | |
| targets | No | Named elements to measure: {"name": {"text": "Checkout"} | {"selector": "css"} | {"role": "button", "name": "Save"}}, each with optional "within", "nth" (0-based), "exact", "fit". | |
| full_page | No | Capture the whole scrollable page instead of the viewport. | |
| timeout_ms | No | Time limit in ms for loading and for each action or target. | |
| hide_overlays | No | Hide cookie banners and similar overlays before the capture and after each click. | |
| user_data_dir | No | Browser profile folder to reuse (cookies, logins) when launching. | |
| page_url_contains | No | With cdp_url: substring of the URL of the tab to capture. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond the annotations it discloses that the page loads over the network, that actions run in-page and can change state on the site (with a caution to only run user-requested actions), and that it requires ainotate[web] plus a Playwright browser verified by doctor. This is meaningful extra context, and it is consistent with readOnlyHint=false and openWorldHint=true.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loads purpose, then alternatives, then usage paths, then requirements, then return values. It is dense and slightly packed, but nearly every clause carries actionable information and nothing is redundant with structured fields.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Although there is no output schema, the description spells out what is returned (PNG path in temp folder, rects in CSS px, scale to use as spec scale, missing targets with reasons, spec stub, preview with outlined rects). Nothing an agent needs to invoke and interpret the call is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3, but the description adds integration-level semantics: the url-or-(cdp_url + page_url_contains) attach flow, the preset values, and the fact that actions execute before capture. This meaningfully complements the schema beyond field-level docs.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb+resource with scope: 'Screenshot a web page in a headless browser (Playwright) and measure named elements... so you can annotate with exact rects.' It explicitly distinguishes itself from siblings capture_window (desktop) and capture_screen (screen).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives explicit when-to-use ('for web UIs') and when-not with named alternatives ('for desktop apps use capture_window, for what is on screen right now capture_screen'). It also disambiguates the url vs cdp_url attach paths and warns to only pass user-requested actions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
capture_windowARead-onlyIdempotent
Screenshot one window by id (from list_windows); never moves, resizes or focuses it. On macOS other windows covering it do not show; on Windows and Linux they may, since the fallback grabs the window's screen area. Use it for desktop apps; capture_screen for several windows, capture_web for web pages. macOS only: targets {name: {"element": label, "role"?, "nth"?, "exact"?}} measures buttons and fields through Accessibility (needs that permission) and returns their rects plus the scale for the annotate spec. Needs Screen Recording on macOS. Writes a PNG to the temp folder. Returns its path, size, window_id, rects and scale when targets were given, and a preview.
| Name | Required | Description | Default |
|---|---|---|---|
| id | Yes | Window id from list_windows. | |
| targets | No | macOS only. Named controls to measure: {"name": {"element": "label", "role"?: "button", "nth"?: 0, "exact"?: true}}. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare this read-only, idempotent and non-destructive, and the description adds substantial context beyond them: platform-specific occlusion behavior (macOS hides covered windows, Windows/Linux fallback grabs the screen area), the Screen Recording permission requirement, the Accessibility permission needed for targets, and that a PNG is written to temp. The PNG artifact write is disclosed rather than hidden and is an output artifact, not a mutation of the target, so it does not contradict readOnlyHint.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Dense but front-loaded: the one-line core behavior plus the sibling routing comes first, and platform caveats and return details follow. Every sentence carries information, though the targets/permission material is packed into a long clause that could be marginally tightened.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema exists, and the description compensates by enumerating the return payload (path, size, window_id, rects, scale, preview). Combined with platform requirements and sibling routing, an agent has everything needed to invoke it correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3, but the description adds real semantic value: it explains that targets is macOS-only, that it measures buttons and fields through Accessibility, that the named spec is {name: {element, role, nth, exact}}, and that the call returns rects plus the scale needed for the annotate spec.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource: screenshot one window by id, sourced from list_windows. It immediately distinguishes itself from siblings by name, contrasting capture_window with capture_screen (several windows) and capture_web (web pages).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicit when-to-use with named alternatives: 'Use it for desktop apps; capture_screen for several windows, capture_web for web pages.' It also constrains scope by noting it never moves, resizes or focuses the window, so the agent knows what side effects to expect from the call.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
clipboard_imageARead-onlyIdempotent
Save the image that is on the clipboard (e.g. a screenshot the user just copied) as a PNG in the temp folder, so you can annotate it. Use it when the user says they copied a screenshot; for a file they sent, pass its path to annotate or locate directly. Reads the clipboard, does not change it. Linux needs wl-paste or xclip. Returns the PNG path, size and a preview.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnly/idempotent/non-destructive, and the description adds what they cannot: it reads but does not modify the clipboard, has an external dependency (wl-paste or xclip on Linux), and writes to the temp folder. These are real behavioral disclosures, though the failure behavior when the clipboard is empty is not stated.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with the action and destination, then the trigger, then exclusions, then caveats. Every sentence adds something an agent needs, and nothing is redundant with the empty schema.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description fills the gap by naming the return values (PNG path, size, preview), which is what the agent needs to chain into annotate. Only the empty-clipboard/no-tool-installed failure mode is left implicit.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Zero parameters, so the schema carries no semantics to compensate for; the baseline of 4 applies. The description usefully notes that the result is a PNG path despite there being no inputs.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Specific verb and resource: save the clipboard image as a PNG in the temp folder. It also notes the downstream goal (annotate) and distinguishes itself from file-path handling in siblings annotate/locate.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicit trigger: 'when the user says they copied a screenshot'. It also names the alternative route ('for a file they sent, pass its path to annotate or locate directly'), which is exactly the branch an agent must decide against.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
compareA
Before/after plate: two images side by side or stacked, each with a label (default "Before", "After"; pass labels in the audience's language). Use it to show a UI change or a fix. Draft by default (temp folder): check the preview, then call again with draft=false to save to the output folder. Writes one PNG; the inputs are not changed. Returns the path, size and a preview.
| Name | Required | Description | Default |
|---|---|---|---|
| name | No | Base file name in the output folder, no extension. Empty = tool default. | |
| after | Yes | Path of the after image. | |
| draft | No | true (default): write to a temp folder so you can check the preview first. false: save to the output folder (~/Pictures/AInotate unless configured) and the backup folder. | |
| before | Yes | Path of the before image. | |
| labels | No | Two labels, before then after. Default ["Before", "After"]. | |
| layout | No | "auto" (picks by aspect ratio), "side" (side by side) or "stack" (one above the other). | auto |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations declare a non-read-only, non-destructive, non-idempotent operation, and the description adds real substance beyond that: it writes one PNG, leaves inputs unchanged, defaults to a temp folder, and moves to the output folder only with draft=false. It also discloses the return contents (path, size, preview). It omits any note about overwriting behavior in the output/backup folder, which is the one remaining gap.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Four tight sentences with no filler, covering what it produces, when to use it, the draft workflow, and the return. The framing leans slightly toward leading with the visual format rather than the action, but nothing is wasted.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema and modest annotations, the description carries the full burden and does so: it explains the output artifact, side effects on disk, the draft/save lifecycle, and the return values. An agent can invoke this correctly without further inference.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all six parameters including defaults for labels, layout, draft, and name. The description restates the label defaults and adds a useful localization hint ('pass labels in the audience's language'), but adds little otherwise, matching the baseline 3 for high-coverage schemas.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a concrete verb and artifact: a before/after plate with two labeled images, side by side or stacked. It explicitly names the use case (showing a UI change or a fix), which cleanly separates it from siblings like annotate, grid, or preview.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives clear context for when to use it ('to show a UI change or a fix') and a concrete two-step workflow: draft first, check the preview, then call again with draft=false. It does not name an explicit alternative tool for comparison, so it falls short of a 5, but the usage path is unambiguous.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
copy_to_clipboardADestructiveIdempotent
Put an image on the system clipboard so the user can paste it into Slack, Jira or mail. It REPLACES whatever the clipboard holds, so tell the user, and use it only when they want to paste; otherwise give them the file path. Works on macOS, Windows and Linux (wl-copy or xclip). Returns the path that was copied.
| Name | Required | Description | Default |
|---|---|---|---|
| path | Yes | Path of a PNG, JPEG or WebP image on this machine (~ allowed). |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already flag destructiveHint=true and idempotentHint=true, but the description goes further by naming exactly what is destroyed ('REPLACES whatever the clipboard holds'), requiring the agent to notify the user, and documenting platform/back-end support (macOS, Windows, Linux wl-copy/xclip). This is precisely the extra context annotations cannot carry.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three short sentences, front-loaded with the action and the destructive consequence, followed by the usage constraint and platform info. Every sentence carries information the agent needs; no filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a single-parameter tool with no output schema, the description covers the side effect, the user-facing obligation, the return value, and cross-platform behavior. An agent has everything required to call it correctly and warn the user.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% and the single 'path' parameter is fully documented in the schema (formats, tilde expansion). The description adds no syntax or format detail beyond it, so the baseline 3 applies; the only added element ('Returns the path that was copied') concerns the result, not the input.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb+resource ('Put an image on the system clipboard') plus the downstream intent ('paste it into Slack, Jira or mail'), which is far more than a restatement of the name. It is also distinguishable from the sibling clipboard_image, which reads the clipboard rather than writing to it.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives an explicit use condition ('use it only when they want to paste') and an explicit alternative ('otherwise give them the file path'), plus a procedural duty ('tell the user'). Nothing about when to reach for this tool versus an alternative is left to inference.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
doctorARead-onlyIdempotent
Check what AInotate can do on this machine: Pillow, font, output folder, Playwright and its browser, a CDP browser, screen capture permission, window listing, OCR engines, clipboard and mcp. Use it first when a tool fails with a missing dependency or permission, or before a long task on a new machine. Read-only: changes nothing. Returns ok, os, every check with its status and the exact fix for each failed one.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint/idempotentHint/destructiveHint, so the safety profile is covered; the description redundantly confirms 'changes nothing' but adds real value by describing the return payload ('ok, os, every check with its status and the exact fix for each failed one'). It stops short of runtime traits such as latency, cost, or whether browser-launching checks have side effects, so it is strong but not exhaustive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, both front-loaded: what is checked first, then when to call it and what comes back. No filler or repetition beyond the minor restatement of read-only behavior.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema and no parameters, the description carries the full burden and discharges it: it names the inspected dependencies, the call conditions, and the shape of the result including per-check fixes.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool takes zero parameters and the schema is empty, so there is nothing for the description to disambiguate; baseline is 4 for a parameterless tool.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description gives a specific verb ('Check') plus an explicit enumeration of what is inspected: Pillow, font, output folder, Playwright/browser, CDP browser, screen capture permission, window listing, OCR engines, clipboard, and mcp. No sibling tool overlaps with this diagnostic role, so an agent can distinguish it immediately.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It states two concrete triggering conditions: 'Use it first when a tool fails with a missing dependency or permission' and 'before a long task on a new machine.' This is explicit when-to-use guidance rather than implied context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
gridARead-onlyIdempotent
Overlay a labelled coordinate grid every step px on an image, to find roughly where an element is when locate cannot read it (icons, images, no text). Then call zoom on that area for the exact rect. Writes the grid image to the temp folder; the input is not changed. The preview IS the grid, labels in source image px. Returns the grid path and the source image size.
| Name | Required | Description | Default |
|---|---|---|---|
| step | No | Grid spacing in image px (> 0). | |
| image_path | Yes | Path of a PNG, JPEG or WebP image on this machine (~ allowed). |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With readOnlyHint/destructiveHint already declaring safety, the description adds genuinely useful context: output goes to the temp folder, the input image is not modified, the preview IS the grid, and labels are in source image px. It also discloses the return payload (grid path and source image size), which matters because no output schema exists. Only minor gap is no mention of temp-file lifetime or cleanup.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Four short sentences, front-loaded with the action and the purpose, then workflow routing, then side effects and returns. Nothing is padded, though it is dense enough that a reader must parse several clauses in one sentence.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a two-parameter tool with no output schema, the description covers purpose, workflow placement, side effects, and return values. Nothing an agent needs in order to invoke it correctly is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so both parameters are already documented, and the schema already states 'Grid spacing in image px (> 0)'. The description's 'every `step` px' and 'labels in source image px' restate the same unit semantics rather than adding new meaning. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('overlay a labelled coordinate grid ... on an image') with the exact granularity ('every `step` px'). It also distinguishes itself from siblings by naming the workflow: use grid when `locate` cannot read the element, then use `zoom` for the exact rect.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives an explicit when-to-use trigger ('when locate cannot read it — icons, images, no text') and names both the precursor alternative (locate) and the follow-up tool (zoom) with the condition that selects it. An agent can route correctly without inferring anything.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_windowsARead-onlyIdempotent
List the visible windows: [{id, app, title, bounds}], to get the id for capture_window. filter = case-insensitive substring of the app name or title. Reads window metadata only (on macOS the titles need the Screen Recording permission). Returns count, windows and a suggestion.
| Name | Required | Description | Default |
|---|---|---|---|
| filter | No | Case-insensitive substring of the app name or window title. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, idempotentHint and destructiveHint=false, so the safety profile is covered. The description adds genuinely new context beyond that: it reads metadata only, and on macOS window titles require Screen Recording permission — a real precondition an agent would not otherwise know.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three dense sentences, front-loaded with the action and return shape, followed by parameter meaning and a permission caveat. Slightly run-on in the first sentence but no wasted content.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description carries the burden of describing returns and does so: item fields, plus 'count, windows and a suggestion'. Combined with the permission note and the sibling reference, an agent has everything needed to call and consume this tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the single parameter is already documented. The description restates the same case-insensitive substring semantics rather than adding format or edge-case detail, so it meets the baseline but no more.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('List the visible windows') and immediately gives the return shape [{id, app, title, bounds}]. It also distinguishes itself from the sibling capture_window by naming it as the downstream consumer of the returned ids.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly frames the tool's role: 'to get the id for capture_window', which tells the agent exactly why and when to call it. It stops short of stating when not to use it or what alternative to prefer for other listing needs, so it falls just below the top score.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
locateARead-onlyIdempotent
Find visible text on an image by OCR and return its rect, for images where you have no DOM or accessibility data (desktop captures, the user's screenshots). Rects are [x1,y1,x2,y2] in image px (spec scale 1), best match first; several equally good matches = ambiguous: ask the user, or pass nth (0-based, reading order) or within [x1,y1,x2,y2]. all=true also lists weaker matches. Runs on this machine (macOS Vision, Windows OCR or Tesseract); reads the image, writes nothing. If the text is not found, use grid + zoom. Returns matches with score and a preview outlining them.
| Name | Required | Description | Default |
|---|---|---|---|
| all | No | Also list weaker matches, not only the best ones. | |
| nth | No | Pick one of several equal matches, 0-based in reading order. | |
| lang | No | OCR language code, e.g. "en", "ro", "de". Omit for the default. | |
| text | Yes | Visible text to find: a word or short phrase as it appears on the image. | |
| exact | No | Require the exact text (case and spacing), not a close match. | |
| within | No | Search only inside [x1, y1, x2, y2], image px. | |
| backend | No | OCR engine: "vision" (macOS), "winrt" (Windows), "tesseract", or omit for auto. | |
| image_path | Yes | Path of a PNG, JPEG or WebP image on this machine (~ allowed). |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnly/idempotent/non-destructive, but the description adds real context beyond them: it runs locally on this machine (macOS Vision, Windows OCR, Tesseract), touches only the image, and defines result ordering (best match first) and ambiguity behavior. It does not discuss latency or engine-specific caveats, so not a full 5.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single front-loaded paragraph that leads with the purpose and then packs scope, coordinate format, ambiguity handling, backends, and failure fallback. Dense but each clause carries information; slightly overstuffed for one sentence-chunk.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With 8 parameters and no output schema, the description still tells the agent the result shape (matches with score and a preview outlining them) and the failure path. Missing only minor operational detail such as whether multiple matches come back by default when all=false.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3, but the description adds meaning the schema lacks: rects are [x1,y1,x2,y2] in image px at spec scale 1, nth is 0-based in reading order, and all=true surfaces weaker matches. This meaningfully enriches interpretation of nth/within/all beyond their short schema blurbs.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb+resource ('Find visible text on an image by OCR and return its rect') and immediately scopes it to the cases where it applies (no DOM or accessibility data). It is clearly separable from siblings like grid, zoom, or capture_* tools, which are named as fallbacks.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Names when to use it (desktop captures, user screenshots), the alternative path when it fails ('If the text is not found, use grid + zoom'), and the ambiguity policy (ask the user, or pass nth/within). Both the selection condition and the failure fallback are explicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
make_guideA
Build a step-by-step guide from annotated images: Markdown, HTML and/or PDF, one image and text per step, every image at its own size. Use it after annotate when the user wants a how-to or documentation; for a single image just deliver the annotate output. Draft by default (temp folder): check the preview, then call again with draft=false to save to the output folder. PDF uses Playwright when available, else a Pillow page per step. Writes files only. Returns the file path per format, the PDF backend, warnings and a preview of the first step.
| Name | Required | Description | Default |
|---|---|---|---|
| name | No | Base file name in the output folder, no extension. Empty = tool default. | |
| draft | No | true (default): write to a temp folder so you can check the preview first. false: save to the output folder (~/Pictures/AInotate unless configured) and the backup folder. | |
| steps | Yes | Guide steps in order: [{"image": path, "text": "...", "title"?: "...", "alt"?: "..."}]. | |
| title | No | Guide title. | |
| formats | No | Any of "md", "html", "pdf". Default md + html. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations declare non-readOnly/non-destructive, but the description adds substantial context beyond them: draft writes to a temp folder, draft=false writes to the output folder and a backup folder, PDF rendering falls back from Playwright to a Pillow page per step, and it 'writes files only'. It also enumerates return contents (file path per format, PDF backend, warnings, first-step preview).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with purpose, then usage, workflow, backend behavior, and return shape — every sentence carries information. It is dense and clause-heavy (long run-on sentences), which costs a little readability, but there is no filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description compensates by describing the return payload (paths per format, PDF backend, warnings, first-step preview). Combined with the draft/save workflow and the format defaults, an agent has everything needed to call it correctly in one or two turns.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the schema already documents name, draft, steps, title and formats; baseline would be 3. The description still adds semantics not in the schema, notably 'one image and text per step' and 'every image at its own size', and frames draft as a review-then-commit workflow.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb+resource ('Build a step-by-step guide from annotated images') plus the output formats (Markdown, HTML, PDF) and the step structure. It also explicitly differentiates from the sibling 'annotate' tool, so an agent can route correctly without opening schemas.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives explicit when-to-use ('after annotate when the user wants a how-to or documentation') and when-not ('for a single image just deliver the annotate output'), naming the alternative. The draft workflow ('check the preview, then call again with draft=false') is spelled out as an operational sequence.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
previewARead-onlyIdempotent
Debug render of an annotate spec: draws the marks plus cyan target rects and magenta label boxes so you can see why a label moved or a mark misses its element. Use it when the annotate preview looks wrong; it is not a deliverable. Writes only a temp file, saves nothing to the output folder. Returns the temp path, warnings and the preview.
| Name | Required | Description | Default |
|---|---|---|---|
| look | No | Mark style: "default" (bold, built in) or the name of an installed look package (e.g. "neat"). Omit it to use the spec's look, else the user's default (AINOTATE_LOOK, config.toml). A look that is not installed or fails falls back to default with a warning listing the installed looks. | |
| spec | Yes | Render spec as a JSON object (or a JSON string). Full schema: resource ainotate://spec. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Goes beyond the annotations by disclosing the side effect ('Writes only a temp file, saves nothing to the output folder') and the return payload (temp path, warnings, preview). This usefully nuances readOnlyHint=true, which would otherwise suggest no filesystem activity at all. It stops short of describing failure modes or how warnings should be interpreted.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, front-loaded with the core purpose, then the usage rule, then the side-effect/return disclosure. The middle clause ('so you can see why a label moved or a mark misses its element') is mildly verbose but earns its place by explaining the diagnostic value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description compensates by naming the return values (temp path, warnings, preview), and it points at the spec schema resource via the parameter definition. An agent has enough to invoke it and interpret the result correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so both 'look' and 'spec' are already documented in the schema (including the fallback-with-warning behavior for an uninstalled look). The description adds no additional parameter meaning, so the baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb+resource ('Debug render of an annotate spec') and enumerates exactly what it draws (marks, cyan target rects, magenta label boxes). It is immediately distinguishable from the sibling 'annotate', which produces the deliverable rather than a diagnostic render.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly says when to reach for it ('Use it when the annotate preview looks wrong') and when not to ('it is not a deliverable'), implicitly routing normal output generation to 'annotate'. Both the trigger condition and the exclusion are stated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
shootA
One call from URL to annotated image: capture_web + annotate, with marks that point at elements by "target" instead of "rect". Use it for web pages when you know the visible text, role or selector of every element; use capture_web + annotate when you need to inspect rects first. spec = render spec without input/scale, plus url (or cdp_url + page_url_contains), preset, actions and marks with "target": {...} (same forms as capture_web). Emails, tokens, card numbers and password fields are redacted with solid blocks automatically. Loads the page over the network; actions run in the page. Draft by default (temp folder): check the preview, then call again with draft=false to save to the output folder. Returns paths, warnings, legend, redactions, the capture's resolved rects and a preview.
| Name | Required | Description | Default |
|---|---|---|---|
| look | No | Mark style: "default" (bold, built in) or the name of an installed look package (e.g. "neat"). Omit it to use the spec's look, else the user's default (AINOTATE_LOOK, config.toml). A look that is not installed or fails falls back to default with a warning listing the installed looks. | |
| spec | Yes | Render spec as a JSON object (or a JSON string). Full schema: resource ainotate://spec. | |
| draft | No | true (default): write to a temp folder so you can check the preview first. false: save to the output folder (~/Pictures/AInotate unless configured) and the backup folder. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations only declare the safety envelope (readOnlyHint=false, openWorldHint=true, destructiveHint=false). The description adds behavior beyond that: automatic redaction of emails/tokens/card numbers/password fields with solid blocks, network page loading, in-page action execution, and where artifacts land in draft vs non-draft mode. This is exactly the extra context annotations cannot carry.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Dense but front-loaded: purpose and sibling routing come first, then spec contents, redaction, and the draft workflow. The spec sentence is a long parenthetical list, but every clause maps to a real caller decision, so little is wasted.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite no output schema, the description enumerates the return payload ('paths, warnings, legend, redactions, the capture's resolved rects and a preview'), and it covers prerequisites (network access, known selectors/text) plus the fallback path to sibling tools. Nothing an agent needs to invoke it correctly is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so baseline is 3. The description does add real value for 'spec', enumerating its contents (url or cdp_url + page_url_contains, preset, actions, marks with target) which the schema merely delegates to the ainotate://spec resource, but it says nothing extra about 'look', leaving the balance of meaning to the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Opens with a precise verb+resource framing — 'One call from URL to annotated image: capture_web + annotate' — and immediately states the differentiator ('marks that point at elements by "target" instead of "rect"'). An agent can distinguish it from capture_web, annotate and preview without opening any schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly gives the when-to-use and when-not-to-use rule: use shoot 'when you know the visible text, role or selector of every element; use capture_web + annotate when you need to inspect rects first.' It also prescribes the two-step draft workflow ('check the preview, then call again with draft=false').
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
zoomARead-onlyIdempotent
Enlarged crop of [x1,y1,x2,y2] with a fine grid labelled in SOURCE image px, so you can read an exact rect straight off the preview (spec scale 1). Use it after grid, or to check a rect from locate. Keep regions under ~500 px wide or the labels get too dense. Writes the crop to the temp folder; the input is not changed. Returns the zoom path, region, size, notes and the preview.
| Name | Required | Description | Default |
|---|---|---|---|
| x1 | Yes | Left edge of the region, source image px. | |
| x2 | Yes | Right edge of the region, source image px (greater than x1). | |
| y1 | Yes | Top edge of the region, source image px. | |
| y2 | Yes | Bottom edge of the region, source image px (greater than y1). | |
| step | No | Fine grid spacing in source px (> 0). | |
| image_path | Yes | Path of a PNG, JPEG or WebP image on this machine (~ allowed). |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnly/idempotent/non-destructive, and the description adds genuinely new behavior: the crop is written to the temp folder, the input image is unmodified, and the return payload is enumerated (zoom path, region, size, notes, preview). The only gap is the lifecycle of the temp file (overwritten? unique per call? cleanup?), which matters for an idempotent-hinted writer.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The core purpose is front-loaded in the first clause, followed by usage routing, a size caveat, and side effects — a logical, information-dense order with no filler. Slightly run-on as a single packed first sentence, but every clause earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description compensates by listing the returned fields, and it covers side effects, prerequisites, and size limits — enough for correct invocation. It falls short of 5 only because the temp-file naming/lifetime and whether repeated calls overwrite a prior crop are unstated for a tool that writes output.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so all six parameters — including the step default and the x1<x2 / y1<y2 constraints — are already documented in the schema; baseline 3 applies. The description reinforces the coordinate semantics ('labelled in SOURCE image px', 'spec scale 1') which is useful disambiguation against scaled previews, but adds no new per-parameter detail.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Opens with a concrete noun phrase — an enlarged crop of a given rect with a fine grid labelled in source-image pixels — and states the concrete outcome (reading an exact rect off the preview). It implicitly separates itself from 'grid' (referenced as a prior step) and 'locate' (whose rect it verifies), so an agent can distinguish it without inspecting schemas.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
'Use it after grid, or to check a rect from locate' explicitly names the two sibling workflows and the ordering condition that selects this tool. It also gives a practical usage constraint ('keep regions under ~500 px wide or the labels get too dense') that prevents a bad call — this is exactly the when/when-not guidance the dimension rewards.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
14 tool updates
v0.1.1- Changed
animate8 fields changed- added
Input schema / properties / crossfade_ms / descriptionAdded value: +"Length of a blended transition between frames in ms; 0 = cut." - added
Input schema / properties / draft / descriptionAdded value: +"true (default): write to a temp folder so you can check the preview first. false: save to the output folder (~/Pictures/AInotate unless configured) and the backup folder." - added
Input schema / properties / duration_ms / descriptionAdded value: +"Time per frame in ms (default 1500)." - added
Input schema / properties / durations_ms / descriptionAdded value: +"Old name of duration_ms; used only when duration_ms is not given." - added
Input schema / properties / format / descriptionAdded value: +"\"apng\" (full color, default) or \"gif\"." - added
Input schema / properties / frames / descriptionAdded value: +"Image paths, one per frame, in order. Frames are scaled to the first one." - added
Input schema / properties / loop / descriptionAdded value: +"Repeat count; 0 = forever." - added
Input schema / properties / name / descriptionAdded value: +"Base file name in the output folder, no extension. Empty = tool default."
- Changed
annotate2 fields changed- added
Input schema / properties / draft / descriptionAdded value: +"true (default): write to a temp folder so you can check the preview first. false: save to the output folder (~/Pictures/AInotate unless configured) and the backup folder." - added
Input schema / properties / spec / descriptionAdded value: +"Render spec as a JSON object (or a JSON string). Full schema: resource ainotate://spec."
- Changed
capture_screen1 field changed- added
Input schema / properties / region / descriptionAdded value: +"Part of the screen: [x, y, w, h] in screen points, or the same four numbers as a string \"x,y,w,h\". Omit for the whole main display (all displays on Windows)."
- Changed
capture_web11 fields changed- added
Input schema / properties / actions / descriptionAdded value: +"Steps run in the page before the capture, in order: {\"click\": T}, {\"fill\": T, \"value\": \"...\"}, {\"wait\": T}, {\"scroll_to\": T}, {\"wait_ms\": 500}; T is a target as in targets." - added
Input schema / properties / browser / descriptionAdded value: +"Engine to launch: \"chromium\" (default), \"firefox\" or \"webkit\"." - added
Input schema / properties / cdp_url / descriptionAdded value: +"DevTools address of a running browser to attach to, e.g. http://127.0.0.1:9222; captures its logged-in tab instead of launching a new browser." - added
Input schema / properties / full_page / descriptionAdded value: +"Capture the whole scrollable page instead of the viewport." - added
Input schema / properties / hide_overlays / descriptionAdded value: +"Hide cookie banners and similar overlays before the capture and after each click." - added
Input schema / properties / page_url_contains / descriptionAdded value: +"With cdp_url: substring of the URL of the tab to capture." - added
Input schema / properties / preset / descriptionAdded value: +"Viewport: \"laptop\" 1280x800@2, \"wide\" 1600x1000@2 or \"phone\" 390x844@3." - added
Input schema / properties / targets / descriptionAdded value: +"Named elements to measure: {\"name\": {\"text\": \"Checkout\"} | {\"selector\": \"css\"} | {\"role\": \"button\", \"name\": \"Save\"}}, each with optional \"within\", \"nth\" (0-based), \"exact\", \"fit\"." - added
Input schema / properties / timeout_ms / descriptionAdded value: +"Time limit in ms for loading and for each action or target." - added
Input schema / properties / url / descriptionAdded value: +"Page to load (http/https). Omit when attaching with cdp_url." - added
Input schema / properties / user_data_dir / descriptionAdded value: +"Browser profile folder to reuse (cookies, logins) when launching."
- Changed
capture_window2 fields changed- added
Input schema / properties / id / descriptionAdded value: +"Window id from list_windows." - added
Input schema / properties / targets / descriptionAdded value: +"macOS only. Named controls to measure: {\"name\": {\"element\": \"label\", \"role\"?: \"button\", \"nth\"?: 0, \"exact\"?: true}}."
- Changed
compare6 fields changed- added
Input schema / properties / after / descriptionAdded value: +"Path of the after image." - added
Input schema / properties / before / descriptionAdded value: +"Path of the before image." - added
Input schema / properties / draft / descriptionAdded value: +"true (default): write to a temp folder so you can check the preview first. false: save to the output folder (~/Pictures/AInotate unless configured) and the backup folder." - added
Input schema / properties / labels / descriptionAdded value: +"Two labels, before then after. Default [\"Before\", \"After\"]." - added
Input schema / properties / layout / descriptionAdded value: +"\"auto\" (picks by aspect ratio), \"side\" (side by side) or \"stack\" (one above the other)." - added
Input schema / properties / name / descriptionAdded value: +"Base file name in the output folder, no extension. Empty = tool default."
- Changed
copy_to_clipboard1 field changed- added
Input schema / properties / path / descriptionAdded value: +"Path of a PNG, JPEG or WebP image on this machine (~ allowed)."
- Changed
grid2 fields changed- added
Input schema / properties / image_path / descriptionAdded value: +"Path of a PNG, JPEG or WebP image on this machine (~ allowed)." - added
Input schema / properties / step / descriptionAdded value: +"Grid spacing in image px (> 0)."
- Changed
list_windows1 field changed- added
Input schema / properties / filter / descriptionAdded value: +"Case-insensitive substring of the app name or window title."
- Changed
locate8 fields changed- added
Input schema / properties / all / descriptionAdded value: +"Also list weaker matches, not only the best ones." - added
Input schema / properties / backend / descriptionAdded value: +"OCR engine: \"vision\" (macOS), \"winrt\" (Windows), \"tesseract\", or omit for auto." - added
Input schema / properties / exact / descriptionAdded value: +"Require the exact text (case and spacing), not a close match." - added
Input schema / properties / image_path / descriptionAdded value: +"Path of a PNG, JPEG or WebP image on this machine (~ allowed)." - added
Input schema / properties / lang / descriptionAdded value: +"OCR language code, e.g. \"en\", \"ro\", \"de\". Omit for the default." - added
Input schema / properties / nth / descriptionAdded value: +"Pick one of several equal matches, 0-based in reading order." - added
Input schema / properties / text / descriptionAdded value: +"Visible text to find: a word or short phrase as it appears on the image." - added
Input schema / properties / within / descriptionAdded value: +"Search only inside [x1, y1, x2, y2], image px."
- Changed
make_guide5 fields changed- added
Input schema / properties / draft / descriptionAdded value: +"true (default): write to a temp folder so you can check the preview first. false: save to the output folder (~/Pictures/AInotate unless configured) and the backup folder." - added
Input schema / properties / formats / descriptionAdded value: +"Any of \"md\", \"html\", \"pdf\". Default md + html." - added
Input schema / properties / name / descriptionAdded value: +"Base file name in the output folder, no extension. Empty = tool default." - added
Input schema / properties / steps / descriptionAdded value: +"Guide steps in order: [{\"image\": path, \"text\": \"...\", \"title\"?: \"...\", \"alt\"?: \"...\"}]." - added
Input schema / properties / title / descriptionAdded value: +"Guide title."
- Changed
preview1 field changed- added
Input schema / properties / spec / descriptionAdded value: +"Render spec as a JSON object (or a JSON string). Full schema: resource ainotate://spec."
- Changed
shoot2 fields changed- added
Input schema / properties / draft / descriptionAdded value: +"true (default): write to a temp folder so you can check the preview first. false: save to the output folder (~/Pictures/AInotate unless configured) and the backup folder." - added
Input schema / properties / spec / descriptionAdded value: +"Render spec as a JSON object (or a JSON string). Full schema: resource ainotate://spec."
- Changed
zoom6 fields changed- added
Input schema / properties / image_path / descriptionAdded value: +"Path of a PNG, JPEG or WebP image on this machine (~ allowed)." - added
Input schema / properties / step / descriptionAdded value: +"Fine grid spacing in source px (> 0)." - added
Input schema / properties / x1 / descriptionAdded value: +"Left edge of the region, source image px." - added
Input schema / properties / x2 / descriptionAdded value: +"Right edge of the region, source image px (greater than x1)." - added
Input schema / properties / y1 / descriptionAdded value: +"Top edge of the region, source image px." - added
Input schema / properties / y2 / descriptionAdded value: +"Bottom edge of the region, source image px (greater than y1)."
16 tool updates
v0.1.0- First observed
animate - First observed
annotate - First observed
capture_screen - First observed
capture_web - First observed
capture_window - First observed
clipboard_image - First observed
compare - First observed
copy_to_clipboard - First observed
doctor - First observed
grid - First observed
list_windows - First observed
locate - First observed
make_guide - First observed
preview - First observed
shoot - First observed
zoom
TDQS
Scored across 16 tools
Each tool targets a distinct resource or stage: capture_web/capture_window/capture_screen differ by source, shoot is explicitly a shortcut combining capture_web+annotate, and preview is clearly framed as a non-deliverable debug render. The locate vs grid+zoom split (text vs non-text) and the annotate/make_guide/compare/animate outputs are all spelled out, so misselection is unlikely.
Names mix bare verbs (annotate, preview, shoot, locate, compare, animate) with verb_noun compounds (capture_web, list_windows, make_guide, copy_to_clipboard) and bare nouns (grid, zoom, doctor). Every name is readable and snake_case is used throughout, but there is no single predictable pattern.
16 tools is slightly on the heavy side but each earns its place, covering capture, coordinate-finding, annotation, composition, delivery and diagnostics without obvious redundancy. It fits the multi-stage screenshot-annotation workflow well.
The surface covers the full lifecycle: acquire (web/window/screen/clipboard), locate coordinates (OCR/grid/zoom), annotate, compose (guide/compare/animate), deliver (clipboard/file) and self-diagnose (doctor). Minor gaps like a general crop/convert or image-info utility exist, but agents can work around them via annotate's crop.
Maintenance
Related MCP Connectors
Clean PNG/JPEG screenshots via REST or MCP, with goal-driven multi-step navigation.
Screenshot, PDF, OG-image, and page extraction (markdown/JSON) over MCP. Bearer key or x402.
Capturewren: screenshot any URL (banners removed) plus status, title and clean page text for agents.
Capturewren: screenshot any URL (banners removed) plus status, title and clean page text for agents.
Related MCP Servers
- FlicenseAqualityDmaintenanceEnables LLMs to capture screenshots and screen recordings through MCP with chunked session-based transfers for reliable image consumption. Supports multi-monitor selection, timeline capture, and compatibility with both vision and non-vision language models.111-
- AlicenseNot gradedqualityCmaintenanceEnables AI agents to capture any public URL as PNG, JPEG, or PDF via REST API or MCP tools, including screenshot capture, page description, and PDF rendering.11 npmMIT
- AlicenseAqualityCmaintenanceEnables MCP clients to capture web pages or render HTML as images or PDFs, with screenshots returned inline to the assistant.327 npmMIT
- AlicenseNot gradedqualityBmaintenanceEnables users to capture any screen or URL, box the region they mean, attach notes, and deliver that annotated feedback—including cropped images, numbered boxes, and the underlying DOM elements or component names—directly to an AI coding agent over MCP. It also lets agents open the annotator, capture a URL, and wait for or retrieve feedback bundles from earlier sessions.MIT





