Skip to main content
Glama

⚡ Flash computer use — Ultra-Fast Hybrid Browser & Desktop MCP Server

License: MIT Python 3.11+ MCP Compatible OS: Linux | Windows | macOS Tests: 74 Passed

Flash computer use is a high-speed, precision-engineered Model Context Protocol (MCP) server in Python for controlling both web browsers (Playwright) and the physical OS desktop (Linux X11, Windows, macOS).

Unlike traditional screenshot-only computer-use agents that take 20–50 seconds per action and generate hallucinated [x, y] coordinates, Flash computer use combines instant deterministic DOM & native OS accessibility targeting (2–7 ms) with calibrated Set-of-Mark / hierarchical grid visual decision fallback (LiquidAI D1)—while always moving the real physical OS mouse pointer along a smooth, human-like minimum-jerk trajectory.


✨ Key Highlights

  • 🚀 Up to 200× Faster Hybrid Execution & Deep DOM ↔ D1 Synergy (hybrid mode):

    • Context & Spatial DOM Disambiguation: Instantly resolves unique accessible labels, child SVG/image icon labels, <select> dropdown options, and even duplicate buttons across product cards or page zones (e.g. "Select Plan for Enterprise Suite" or "Sign in button in the header") in 2–7 ms (0 VLM calls), verifying unoccluded click coordinates via 9-point elementFromPoint hit-testing.

    • Smart Candidate Pre-Filtering & Cropping for D1: When visual disambiguation is needed on dense pages (60+ elements), select_d1_candidates narrows the Set-of-Mark annotation to only the relevant candidate subset, crops tightly around them, places non-occluding exterior badges, and enriches each option with [role | zone | card context].

    • Hybrid Offscreen Scrolling (scroll_until_visible): Detects offscreen DOM targets below the fold and scrolls directly until they enter the viewport without wasting intermediate VLM calls.

    • Desktop Native Accessibility: Queries native OS accessibility trees (Linux AT-SPI, Windows UI Automation, macOS AXUIElement) with automatic cache invalidation.

  • 🖱️ Human-Like Physical OS Mouse Movement (Web + Desktop):

    • Always moves the real physical OS cursor along a minimum-jerk trajectory ($10t^3 - 15t^4 + 6t^5$) with a subtle natural wrist arc and exact pixel landing.

    • In visible browser mode, automatically maps Playwright viewport coordinates to global screen coordinates before every click, typing action, scroll, drag, or popup action.

  • 🎨 Paint & <canvas> Drawing + Dragging:

    • Automatically detects the largest white/light drawing sheet (canvas_bounds) in desktop paint apps (MS Paint, KolourPaint, Pinta, etc.) or HTML5 <canvas> elements in the browser.

    • Supports smooth direct dragging (mouse_drag), continuous freehand & multi-stroke paths with anti-gap interpolation (draw_path), and 12 ready-to-use geometric shapes (draw_shape).

  • 🛡️ Zero Coordinate Hallucination:

    • Uses LiquidAI D1 via native /v1/systemone decision probabilities (noul, choice, score with zero generated tokens) over Set-of-Mark badges and coarse-to-fine grids (Ollama, local Hugging Face Transformers, or Cloud API).

  • 💻 Cross-Platform Precision:

    • Built-in Windows Per-Monitor DPI Awareness, macOS Retina multi-monitor scaling, and Linux X11 native clipboard & pointer integration.


Related MCP server: Desktop Controller

📖 Table of Contents

  1. Speed & Accuracy Benchmarks

  2. System Requirements

  3. Step-by-Step Installation

  4. Ollama & D1 Model Setup

  5. MCP Client Configuration (Cursor, Claude, Codex, Antigravity)

  6. Environment Variables Reference

  7. Complete Reference of All 15 MCP Tools

  8. Drawing in Paint, KolourPaint & Web Canvas

  9. Cross-Platform Support (Linux, Windows, macOS)


1. Speed & Accuracy Benchmarks

Measured on a dense synthetic e-commerce page (50+ interactive elements) using LiquidAI D1 3B (Q8_0) on Ollama 0.40.4 (NVIDIA GTX 1050 4 GB VRAM). Full details and raw JSON outputs are in benchmarks/REPORT.md.

Benchmark Step

Baseline (Unoptimized Vision)

Optimized Vision (D1_BROWSER_STRATEGY=vision)

Hybrid Mode (D1_BROWSER_STRATEGY=hybrid, Default)

Readiness Check (wait_until_ready)

24.247 s (1280×800 D1)

4.068 s (768×480 D1)

0.002 – 0.007 s (2–7 ms, DOM)

Search Fill + Verify + Submit (smart_type)

26.795 s (Failed, low confidence)

2.556 s (646×115 cropped D1, Verified)

0.177 – 0.251 s (177–251 ms, Verified)

Total End-to-End Tool Execution

51.042 s (Failed)

6.624 s (100% Pass)

~0.18 – 0.26 s (100% Pass)

VLM Inference Calls

2

2

0

Exact Input Value & Submit Verified

No (0/1)

Yes (1/1)

Yes (3/3)

Why It Is So Fast

  1. Deterministic DOM & Native Accessibility First (hybrid): Unique accessible labels and roles bypass VLM calls entirely (2–7 ms).

  2. Candidate Region Cropping & Downscaling (vision): When D1 vision fallback is triggered, images are proportionally downscaled (D1_IMAGE_MAX_SIDE=768) and input candidates are tightly cropped (646×115), cutting GPU latency by 10× (26.8 s → 2.55 s) and increasing model confidence from 0.40 to 0.91.

  3. Zero-Lag OS Pointer: Removes PyAutoGUI's default 100 ms per-step sleep, executing smooth minimum-jerk human curves in ~40 ms.

Reproduce the benchmarks locally:

# Hybrid mode benchmark (3 runs)
D1_BENCH_RUNS=3 .venv/bin/python benchmarks/browser_speed.py benchmarks/after.json

# Forced D1 Vision fallback benchmark
D1_BROWSER_STRATEGY=vision D1_BENCH_RUNS=1 .venv/bin/python benchmarks/browser_speed.py benchmarks/after-vision.json

# Desktop hierarchical grid precision benchmark
.venv/bin/python benchmarks/desktop_precision.py

2. System Requirements

  • Python: >= 3.11

  • Package Manager: uv (recommended) or pip

  • Decision Engine Provider (choose one):

    • Ollama (recommended, local): version supporting /v1/systemone (e.g. >= 0.40) with d1:3b

    • Local Transformers: CUDA / Apple MPS / CPU with LiquidAI/d1-3B

    • Cloud API: any endpoint implementing /v1/systemone

  • OS Desktop Prerequisites:

    • Linux: active X11 session (DISPLAY) and xclip or xsel for fast Unicode clipboard pasting.

    • Windows: interactive desktop session (Per-Monitor DPI Awareness is enabled automatically).

    • macOS: grant Accessibility and Screen Recording permissions to your terminal / MCP client.


3. Quick Start & Installation

Option A: Zero-Clone Install via npx (Recommended)

You can run or register Flash computer use directly from GitHub using npx—it automatically bootstraps an isolated Python virtual environment (~/.cache/flash-computer-use/venv or %LOCALAPPDATA%\flash-computer-use\venv) and installs Playwright Chromium on first launch:

# Run environment diagnostics via npx
npx -y github:shortsspectrumai-del/flash-computer-use doctor

# Generate portable MCP JSON config via npx
npx -y github:shortsspectrumai-del/flash-computer-use config --npx

# Start the stdio MCP server directly via npx
npx -y github:shortsspectrumai-del/flash-computer-use

Option B: Manual Clone & Local Virtual Environment

Linux / macOS

git clone https://github.com/shortsspectrumai-del/flash-computer-use.git
cd flash-computer-use

# 1. Create virtual environment and install dependencies
uv venv
uv pip install -e '.[dev]'

# 2. Install Playwright Chromium
PLAYWRIGHT_BROWSERS_PATH=.runtime/playwright .venv/bin/python -m playwright install chromium

Windows (PowerShell)

git clone https://github.com/shortsspectrumai-del/flash-computer-use.git
cd flash-computer-use

uv venv
uv pip install -e ".[dev,accessibility]"
$env:PLAYWRIGHT_BROWSERS_PATH=".runtime\playwright"
.\.venv\Scripts\python.exe -m playwright install chromium

4. Ollama & D1 Model Setup

Pull the d1:3b decision model and verify your environment:

# Pull LiquidAI D1 3B model
ollama pull d1:3b

# Check OS desktop capabilities, accessibility backend, and Python dependencies
npx -y github:shortsspectrumai-del/flash-computer-use doctor

# Verify D1 decision inference (binary noul, choice, score) on a synthetic image
D1_PROVIDER=ollama D1_MODEL_NAME=d1:3b npx -y github:shortsspectrumai-del/flash-computer-use check --inference

5. MCP Client Configuration (Cursor, Claude, Codex, Antigravity, Pi)

A. Cursor / Claude Desktop / Google Antigravity (mcp.json / claude_desktop_config.json)

Using npx (Works on Linux, macOS, and Windows without manual cloning)

Add the following block to your client's MCP configuration file (see examples/mcp.ollama.json):

{
  "mcpServers": {
    "Flash computer use": {
      "command": "npx",
      "args": [
        "-y",
        "github:shortsspectrumai-del/flash-computer-use"
      ],
      "env": {
        "D1_PROVIDER": "ollama",
        "D1_MODEL_NAME": "d1:3b",
        "D1_OLLAMA_BASE_URL": "http://localhost:11434",
        "D1_TIMEOUT_SECONDS": "120",
        "D1_BROWSER_STRATEGY": "hybrid",
        "D1_DESKTOP_STRATEGY": "hybrid",
        "D1_IMAGE_MAX_SIDE": "768",
        "D1_REAL_MOUSE": "true",
        "D1_MOUSE_DURATION": "0.04"
      }
    }
  }
}

(Or if using a local clone, set "command": "/absolute/path/to/flash-computer-use/run-mcp.sh" on Linux/macOS or "C:\\absolute\\path\\to\\flash-computer-use\\run-mcp.cmd" on Windows).


B. OpenAI Codex CLI (~/.codex/config.toml)

Register via npx with a single command:

codex mcp add "Flash computer use" \
  --env D1_PROVIDER=ollama \
  --env D1_MODEL_NAME=d1:3b \
  --env D1_OLLAMA_BASE_URL=http://localhost:11434 \
  --env D1_TIMEOUT_SECONDS=120 \
  --env D1_BROWSER_STRATEGY=hybrid \
  --env D1_DESKTOP_STRATEGY=hybrid \
  -- npx -y github:shortsspectrumai-del/flash-computer-use

Or copy examples/codex.ollama.toml into ~/.codex/config.toml:

[mcp_servers."Flash computer use"]
command = "npx"
args = ["-y", "github:shortsspectrumai-del/flash-computer-use"]
startup_timeout_sec = 60
tool_timeout_sec = 120

[mcp_servers."Flash computer use".env]
D1_PROVIDER = "ollama"
D1_MODEL_NAME = "d1:3b"
D1_OLLAMA_BASE_URL = "http://localhost:11434"
D1_TIMEOUT_SECONDS = "120"
D1_BROWSER_STRATEGY = "hybrid"
D1_DESKTOP_STRATEGY = "hybrid"
D1_IMAGE_MAX_SIDE = "768"
D1_REAL_MOUSE = "true"
D1_MOUSE_DURATION = "0.04"

C. Claude Code CLI

claude mcp add --transport stdio \
  --env D1_PROVIDER=ollama \
  --env D1_MODEL_NAME=d1:3b \
  --env D1_OLLAMA_BASE_URL=http://localhost:11434 \
  "Flash computer use" -- npx -y github:shortsspectrumai-del/flash-computer-use

D. Pi Coding Agent (~/.pi/agent/mcp.json)

pi mcp add flash-computer-use \
  --exposure direct \
  --env D1_PROVIDER=ollama \
  --env D1_MODEL_NAME=d1:3b \
  --env D1_OLLAMA_BASE_URL=http://localhost:11434 \
  --env D1_TIMEOUT_SECONDS=120 \
  --env D1_BROWSER_STRATEGY=hybrid \
  --env D1_DESKTOP_STRATEGY=hybrid \
  -- npx -y github:shortsspectrumai-del/flash-computer-use

6. Environment Variables Reference

See .env.example for a complete template.

Variable

Default

Description

D1_PROVIDER

local

Decision provider: ollama, local (Transformers), or api

D1_MODEL_NAME

d1:3b / LiquidAI/d1-3B

Model identifier for Ollama (d1:3b) or Hugging Face / API

D1_OLLAMA_BASE_URL

http://localhost:11434

Ollama server base URL

D1_OLLAMA_KEEP_ALIVE

10m

Keeps the model warm in memory between tool calls

D1_API_BASE_URL

—

Base URL for /v1/systemone when D1_PROVIDER=api

D1_API_KEY

—

Bearer API key when D1_PROVIDER=api

D1_TIMEOUT_SECONDS

60

Provider HTTP timeout in seconds

D1_BROWSER_STRATEGY

hybrid

hybrid (fast DOM + D1 fallback) or vision (always D1)

D1_DESKTOP_STRATEGY

hybrid

hybrid (native OS accessibility + D1 fallback) or vision

D1_IMAGE_MAX_SIDE

768

Max image dimension sent to D1 (0 disables downscaling)

D1_CONFIDENCE_THRESHOLD

0.65

Minimum confidence required to authorize a click

D1_BROWSER_HEADLESS

false

false for visible browser window, true for headless

D1_BROWSER_CDP_URL

—

Optional CDP endpoint (e.g. http://localhost:9222) to attach to an existing browser

D1_BROWSER_PAGE_URL

—

Exact tab URL to select when CDP exposes multiple tabs

D1_MONITOR

1

MSS monitor index for desktop screen capture

D1_REAL_MOUSE

true

Always move the real physical OS mouse cursor during visible browser actions (false to disable, always to force even in headless)

D1_MOUSE_DURATION

0.04

Default duration (in seconds) of the human-like minimum-jerk pointer trajectory


7. Complete Reference of All 15 MCP Tools

Every tool call is serialized via an async lock and returns compact structured JSON including execution time (elapsed_ms).

Browser & Popup Navigation

  1. browser_open(url): Opens a URL and returns page title, ready status, accessible input field labels, short text preview, and any detected popup/cookie banner details.

  2. handle_popup(action="inspect" | "reject" | "accept" | "dismiss", max_scrolls=8): Inspects or resolves cookie banners, modal dialogs, and Google/YouTube consent pages (automatically scrolling terms if the action button is disabled until the bottom).

  3. browser_scroll(direction="down" | "up", pixels=600, container="auto" | "popup" | "page"): Scrolls the active popup container or the main page.

Smart Hybrid Actions (mode="browser" or mode="desktop")

  1. smart_click(target_description, mode="browser", double_click=False): Locates the target via DOM/accessibility or Set-of-Mark / hierarchical D1 grid, moves the real physical pointer along a human curve, and clicks an unoccluded point.

  2. smart_move(target_description, mode="desktop"): Locates the target and moves the mouse pointer onto it without clicking (hover).

  3. smart_type(target_description, text_to_type, press_enter=True, mode="browser"): Filters editable fields, replaces existing text, verifies the exact typed value (typed_verified=true), and optionally presses Enter.

  4. scroll_until_visible(target_description, direction="down", max_scrolls=8, mode="browser"): Scrolls step-by-step until $P(\text{visible}) \ge 0.80$.

  5. visual_assert(questions, mode="browser"): Evaluates 1–64 binary visual questions in a single D1 call. Probabilities between 0.20 and 0.80 return answer=null, certain=false.

  6. wait_until_ready(timeout_seconds=10, mode="browser"): Fast DOM readiness check in hybrid mode or visual polling $P(\text{ready}) \ge 0.80$ in vision/desktop mode.

  7. desktop_inspect(): Returns OS portability status, visible native accessibility elements, and auto-detected drawing sheet bounds (canvas_bounds).

Direct Mouse Control & Paint / Canvas Drawing

  1. mouse_move(x, y, duration=0.04, mode="desktop"): Moves the pointer to (x, y) along a minimum-jerk human curve without clicking.

  2. mouse_click(x=None, y=None, button="left", double_click=False, mode="desktop"): Clicks (left, right, middle) at the current cursor position or after moving to (x, y).

  3. mouse_drag(start_x, start_y, end_x, end_y, button="left", duration=0.12, mode="desktop"): Moves to (start_x, start_y), presses the mouse button, drags smoothly along an interpolated path to (end_x, end_y), and safely releases.

  4. draw_path(points, canvas_bounds=None, duration=0.4, mode="desktop"): Draws by dragging the mouse across the canvas sheet. Supports both a single polyline [[x, y], ...] and multi-stroke drawings [[[x, y], ...], ...] (lifting the mouse button between strokes). When canvas_bounds is omitted, automatically detects the active Paint sheet or <canvas>.

  5. draw_shape(shape, x=None, y=None, width=None, height=None, canvas_bounds=None, duration=0.4, mode="desktop"): Draws a geometric shape (line, rectangle, ellipse, circle, triangle, diamond, star, arrow, heart, house, spiral, wave). Automatically detects the canvas and centers the shape when bounds or coordinates are omitted.


8. Drawing in Paint, KolourPaint & Web Canvas

Flash computer use can control raster and vector drawing applications just like a human user:

  • Automatic Drawing Sheet Detection (canvas_bounds): You do not need to manually measure the canvas coordinates. The server inspects accessibility roles (canvas, drawing area) and falls back to a fast largest-rectangle histogram detector (detect_canvas_from_image) that locates the white/light drawing sheet inside Paint, KolourPaint, Pinta, or browser <canvas> apps.

  • Anti-Gap Stroke Interpolation: Fast mouse drags in Paint often leave dotted gaps if points are far apart. interpolate_stroke subdivides long segments into dense ~6–8 px steps while preserving exact sharp corners.

  • 12 Built-In Shapes (draw_shape): line, rectangle, ellipse, circle, triangle, diamond, star, arrow, heart, house, spiral, wave.

  • Guaranteed mouseUp Safety: Every drag and drawing operation wraps pointer state in a try ... finally block so the mouse button is never left held down on error.


9. Cross-Platform Support (Linux, Windows, macOS)

Operating System

Native Accessibility Backend

Coordinate & DPI Scaling

Unicode Text Input

Linux (X11)

AT-SPI (gi.repository.Atspi)

Direct X11 screen coordinates

xclip / xsel clipboard + Ctrl+V or pyautogui.write fallback

Windows

UI Automation (pywinauto / PowerShell UIA)

Automatic Per-Monitor DPI Awareness (shcore / user32)

Set-Clipboard + Ctrl+V or pyautogui.write fallback

macOS

Accessibility API (ApplicationServices / osascript)

Automatic Retina physical-to-logical scaling (mss ↔ pyautogui)

pbcopy + Cmd+V or pyautogui.write fallback


🧪 Running the Test Suite

Run all 68 automated unit and integration tests (covering MCP stdio protocol, D1 HTTP contract, DOM & accessibility targeting, popup handling, human mouse trajectories, and real HTML5 <canvas> drawing):

.venv/bin/python -m pytest -q

📄 License

Released under the MIT License.

Available Tools

15 tools
browser_openA

Open a URL; returns title, readiness, input labels and a short page preview. Returns popup details when one is present: inspect/resolve it with handle_popup using the user's requested choice. If ready is true, skip wait_until_ready. Use the returned field label in smart_type; a preview may reveal an interstitial instead of the requested page.

NameRequiredDescriptionDefault
urlYes

Output Schema

NameRequiredDescription

No output parameters

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the burden and does well: it discloses return contents, the popup branch, and the caveat that a preview may show an interstitial instead of the requested page. It omits failure modes such as unreachable URLs, timeouts, or auth-walled pages, which matter for a navigation call.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three tight sentences, front-loaded with the action and result, with no filler. Slightly telegraphic in the popup clause, but every sentence carries actionable information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so the return-value prose is a bonus rather than a necessity, and the description adds the non-obvious parts: readiness flag semantics, input labels for smart_type, and the interstitial risk. Only the error/edge-case behavior is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0% and the single 'url' parameter is undocumented in both schema and description — no guidance on absolute vs relative URLs or scheme requirements. The parameter is largely self-evident from its name, so this is a real but modest gap rather than a blocking one.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb+resource ('Open a URL') and immediately characterizes the payload (title, readiness, input labels, short page preview), so an agent knows both the action and the observable result. It is clearly the navigation entry point among click/type/scroll siblings.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly routes downstream: use handle_popup to inspect/resolve a popup, skip wait_until_ready when ready is true, and pass the returned field label into smart_type. Each condition names the sibling alternative and the trigger for using it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

browser_scrollB

Scroll without visual inference. auto scrolls an active popup or its terms panel; otherwise the page. Returns moved and at_end. Use container=popup to prevent scrolling the underlying page.

NameRequiredDescriptionDefault
pixelsNo
containerNoauto
directionNodown

Output Schema

NameRequiredDescription

No output parameters

TDQS

B3.3/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the burden and does disclose key behavior: auto popup/page targeting, popup prevention via container=popup, and return fields moved/at_end. It still omits safety/read-only status, side effects, permissions, and whether scrolling is animated or instant.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three tight sentences with no wasted text. The core behavior is front-loaded, followed by return values and the popup-specific caveat.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Adequate for a simple scroll tool with an output schema, since return fields need not be explained in depth. However, it leaves gaps around pixels and direction semantics and does not provide alternative-tool guidance, which matters given 0% schema description coverage and no annotations.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0% for three parameters. The description explains container and auto behavior, but says nothing about the meaning or effect of pixels or direction, leaving two of three parameters without descriptive semantics beyond their enum/default values.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Scroll') and adds scope detail: it scrolls without visual inference, auto-selects an active popup/terms panel, otherwise the page. It clearly contrasts with visual-based siblings implicitly, but does not name scroll_until_visible or another sibling explicitly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides clear usage context for container=popup to prevent scrolling the underlying page, and explains the auto behavior. However, it does not state when to use this tool instead of scroll_until_visible, nor does it give explicit exclusions or prerequisites.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

desktop_inspectA

Read-only platform requirements, accessible controls, and detected drawing canvas of the foreground app. Use exact labels in smart_click/type for the fast path; unavailable accessibility falls back to D1 vision.

NameRequiredDescriptionDefault

No parameters

Output Schema

NameRequiredDescription

No output parameters

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden: it declares the tool read-only and discloses the fallback path to D1 vision when accessibility data is unavailable. It does not discuss permissions or rate limits, but the read-only and fallback behavior are the key traits for this inspection tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the core purpose and followed by usage guidance. The return list is dense but earns its place, and there is no filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The output schema already describes the return shape, so the description need not repeat it. Given no parameters and a read-only inspection role, the description covers purpose, usage, and fallback behavior sufficiently, though it could mention when inspection is unnecessary.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters, so there is no parameter semantics to explain; the description correctly does not invent any. Baseline 4 applies for a parameterless tool, and nothing in the description contradicts the empty schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb (inspect), a specific resource (foreground app), and enumerates what it reads: platform requirements, accessible controls, and detected drawing canvas. It is clear enough to distinguish from the mouse/click/draw siblings, though it does not explicitly differentiate from visual_assert or wait_until_ready.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives actionable downstream guidance: use exact labels from this tool in smart_click/smart_type for the fast path, and fall back to D1 vision when accessibility is unavailable. It does not state explicit exclusions or when not to call desktop_inspect, but it provides clear usage context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

draw_pathA

Draw on a canvas by dragging the left mouse button. Accepts a single polyline [[x,y],...] or multiple strokes [[[x,y],...],...] (lifts the brush between strokes). Auto-detects the Paint/browser canvas sheet if canvas_bounds is omitted.

NameRequiredDescriptionDefault
modeNodesktop
pointsYes
durationNo
canvas_boundsNo

Output Schema

NameRequiredDescription

No output parameters

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description must carry the full burden. It usefully discloses that strokes lift the brush between polylines and that canvas_bounds defaults to auto-detecting the Paint/browser canvas, but it omits whether browser mode requires a prior browser_open, what happens if coordinates fall outside bounds, and any preconditions for input focus.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three tight sentences, front-loaded with the action, followed by input format and then the default-detection note. No filler, though the stroke-format sentence is dense and could be split for scanning.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Output schema exists, so return values need no explanation, and the core mechanics and point format are covered. The gap is the undocumented `mode` and `duration` parameters, which matter for choosing desktop versus browser targeting.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, so the description must compensate. It does explain the two accepted forms of the required `points` argument and the default behavior of `canvas_bounds`, but leaves `mode` (desktop vs browser) and `duration` (drawing speed) completely undocumented in both schema and prose.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Draw on a canvas by dragging the left mouse button') and clarifies the input shape. It implicitly contrasts with draw_shape by describing freeform polylines, but never names that sibling, so differentiation stays inferential.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is only implied by the mechanism description; there is no explicit 'use this instead of draw_shape/mouse_drag' guidance or prerequisites. An agent must infer when freeform paths beat shape primitives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

draw_shapeB

Draw a shape (line, rectangle, ellipse, circle, triangle, diamond, star, arrow, heart, house, spiral, wave) by dragging the mouse on the sheet. Auto-detects canvas_bounds and centers the shape on the canvas if coordinates are omitted.

NameRequiredDescriptionDefault
xNo
yNo
modeNodesktop
shapeYes
widthNo
heightNo
durationNo
canvas_boundsNo

Output Schema

NameRequiredDescription

No output parameters

TDQS

B3.1/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations the description carries the full burden, and it does disclose a genuinely useful behavioral trait: it auto-detects canvas_bounds and centers the shape when coordinates are omitted. However it is silent on prerequisites (focused browser/desktop window), what mode does, and what happens with invalid coordinates.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences with the shape enumeration up front and the auto-detection fallback following. The long shape list is functional rather than wasteful; no filler text.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For an 8-parameter tool with zero annotation and zero schema-description coverage, the description omits too much: whether x/y is the shape's top-left or center, what width/height measure, and what duration controls (draw speed?) are all unaddressed. The output schema relieves it of explaining return values, but the input semantics gap is significant.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, so the description must compensate and only partially does: it explains coordinates being optional, canvas_bounds auto-detection, and centering. It says nothing about width, height, duration, or mode, leaving four of eight parameters meaning-free for the agent.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (draw) and resource (shape), and enumerates the exact shapes allowed, which is strong evidence of scope. It never mentions the sibling draw_path or mouse_drag, so an agent must infer which drawing primitive to use from the shape enumeration alone.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies use ('by dragging the mouse on the sheet') and notes a fallback behavior when coordinates are omitted, but gives no explicit when-to-use / when-not guidance against draw_path, mouse_drag, or smart_move. Usage is inferable but not stated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

handle_popupA

Inspect a popup or explicitly accept/reject/dismiss it. Never assumes consent. Handles consent iframes and scrolls terms to unlock disabled buttons. Use the user's requested choice; reject never falls back to accept. popup_closed confirms closure; action_sent means do not repeat blindly.

NameRequiredDescriptionDefault
actionNoinspect
max_scrollsNo

Output Schema

NameRequiredDescription

No output parameters

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden; it discloses meaningful behavior: never assumes consent, handles consent iframes, scrolls terms to unlock buttons, and reject does not fall back to accept. It also explains output statuses, though it omits explicit no-popup/error behavior and detailed side effects of accepting.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The purpose is front-loaded in the first sentence, and the remaining sentences each add safety, behavior, or output semantics. It is dense but efficient, with no filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given no annotations and 0% schema coverage, the description covers purpose, behavioral caveats, and output statuses well. It still leaves max_scrolls and explicit no-popup/error handling unspecified, so it is not fully complete for a low-schema-coverage tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It names the four action values and explains the safety implication of reject, but never mentions the max_scrolls parameter, leaving one of two parameters semantically uncovered.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States specific verbs (inspect/accept/reject/dismiss) and resource (popup), and explicitly covers consent iframes and disabled buttons. It is clearly scoped to popup handling, distinguishing it from generic siblings like smart_click and mouse_click.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives clear usage guidance: use the user's requested choice, inspect is the default framing, and reject never falls back to accept. It does not explicitly say when not to use it or when a generic sibling should be used instead.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

mouse_clickA

Click the real desktop pointer, optionally moving to global screen coordinates first. Browser mode requires x/y in the controlled tab viewport and moves the real OS pointer first. Use smart_click for semantic targets and handle_popup for an explicit consent choice.

NameRequiredDescriptionDefault
xNo
yNo
modeNodesktop
buttonNoleft
double_clickNo

Output Schema

NameRequiredDescription

No output parameters

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the burden well on the most non-obvious trait: this drives the real OS pointer, not a synthetic DOM event, and in browser mode it moves the real pointer first before clicking in the viewport. That is material side-effect disclosure an agent could not infer from the schema. It stops short of stating behavior when x/y are null (presumably click at the current position) or any failure/timeout behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three short sentences, zero filler, with the core action and the pointer type front-loaded and the sibling routing pushed to the end. Each sentence adds distinct information: what it does, how modes differ, where to go instead.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The output schema covers return values and there are no annotations, so the description's job is mode/coordinate semantics and sibling routing, both of which it handles for a five-parameter mutation tool. The remaining hole is that button and double_click are undocumented in both description and schema, which for a click tool is a notable but not fatal gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It adds real meaning for x/y by assigning them coordinate spaces (global screen coordinates in desktop mode, controlled-tab viewport in browser mode) and implies the mode enum. However, 'button' and 'double_click' are never mentioned, leaving two of five parameters documented nowhere.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb+resource ('Click the real desktop pointer') and immediately delimits the two operating modes, including the coordinate space each uses. It explicitly names two sibling tools (smart_click, handle_popup) with the conditions that select them, so an agent can route without opening any schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It names the alternatives and their selection criteria: smart_click for semantic targets, handle_popup for an explicit consent choice, and a hard requirement that browser mode needs x/y in the tab viewport. What is missing is guidance on when NOT to use this at all (e.g. preferring smart_click for anything addressable) and what happens when coordinates are omitted.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

mouse_dragB

Move the pointer to (start_x, start_y), hold the mouse button, drag smoothly to (end_x, end_y), and release. Ideal for Paint tools, sliders, selections, and drag-and-drop.

NameRequiredDescriptionDefault
modeNodesktop
end_xYes
end_yYes
buttonNoleft
start_xYes
start_yYes
durationNo

Output Schema

NameRequiredDescription

No output parameters

TDQS

B3.3/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden, and it does disclose the core behavioral model: an intermediate mouse-down state, smooth interpolation, and release. It omits anything about failure modes, focus requirements, whether coordinates are screen- or viewport-relative, and how mode=desktop vs mode=browser changes behavior — significant gaps for an OS-level input tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two tight sentences. The action sequence is front-loaded and the use cases follow as secondary context. No filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return values need not be described. However, for a 7-parameter tool with 0% schema coverage and no annotations, the description leaves three parameters and the desktop/browser distinction unexplained, which is a real completeness gap for an agent trying to invoke it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It only explains the four start/end coordinates; button (which button is held), mode (desktop vs browser), and duration (which governs the smoothness implied by "drag smoothly") are completely undocumented, and mode in particular is a behavior-changing enum an agent cannot infer.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb sequence and resource: move to start point, press, drag, release at end point. This differentiates it from mouse_click and mouse_move by the press-hold-release semantics. It does not explicitly distinguish itself from draw_path or draw_shape, which overlap heavily in the drag/gesture space.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

"Ideal for Paint tools, sliders, selections, and drag-and-drop" gives positive usage contexts, which is more than nothing. But there is no when-not guidance and no mention of siblings like draw_path or smart_move, leaving the agent to infer which drag-like tool to pick when several qualify.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

mouse_moveA

Move without clicking. desktop moves the real OS pointer using global screen pixels; browser moves the tab's mouse using viewport pixels and syncs the real OS pointer on visible browsers. Prefer smart_move when only a target description is known.

NameRequiredDescriptionDefault
xYes
yYes
modeNodesktop
durationNo

Output Schema

NameRequiredDescription

No output parameters

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the behavioral burden and does well by explaining that the tool only moves without clicking, that desktop mode uses global screen pixels, and that browser mode uses viewport pixels while syncing the real OS pointer on visible browsers. It omits details such as hover-event side effects, permissions, and duration behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact and front-loaded: purpose first, then mode-specific coordinate behavior, then the alternative tool recommendation. Every sentence adds useful information without repetition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return values do not need description coverage. The main gaps are the undocumented duration parameter and the absence of side-effect or error-condition details, but the core behavior and usage routing are otherwise complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It clarifies the coordinate semantics for x and y in both modes and explains the mode enum, but it does not explain the duration parameter at all, leaving part of the four-parameter surface undocumented.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific action and resource: moving the mouse pointer without clicking. It distinguishes the two modes and names the sibling tool smart_move as the alternative for target-description-based movement.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It explicitly says to prefer smart_move when only a target description is known, and it gives clear guidance for choosing desktop versus browser mode based on the coordinate system and visible browser syncing. This is enough to route an agent between moving by known coordinates and moving by description.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

scroll_until_visibleA

Scroll in steps until visual P(visible) reaches 0.80. Browser scroll is scoped to an active popup if present; prefer browser_scroll for deterministic scrolling without inference.

NameRequiredDescriptionDefault
modeNobrowser
directionNodown
max_scrollsNo
target_descriptionYes

Output Schema

NameRequiredDescription

No output parameters

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and does add real context: the 0.80 visibility threshold, stepwise scrolling, and popup scoping. It is silent on what happens when the threshold is never reached, how max_scrolls interacts with the loop, and whether scrolling is reversible.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two short sentences, zero filler, with the core behavior and threshold front-loaded ahead of the sibling routing advice.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists so return values needn't be explained, but for a four-parameter inference-driven UI action with no annotations, the missing failure semantics and undocumented parameter meanings leave gaps an agent would need to guess at.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate, but it never explains mode (browser vs desktop), direction, max_scrolls, or the format expected for target_description. Only the popup/browser scoping hints at any parameter's effect.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb+resource ('scroll in steps') with the exact success criterion (visual P(visible) >= 0.80), which immediately distinguishes it from deterministic scrolling siblings. An agent can tell what this does and how success is measured without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly points to browser_scroll as the alternative and gives the selection criterion ('deterministic scrolling without inference'), plus notes scroll scoping to an active popup. It stops short of an explicit 'use this tool when...' statement, leaving the inverse condition to be inferred.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

smart_clickB

Click a uniquely named visible DOM target immediately; otherwise use D1 vision. Duplicate names are ambiguous. Reports method and UI change.

NameRequiredDescriptionDefault
modeNobrowser
double_clickNo
target_descriptionYes

Output Schema

NameRequiredDescription

No output parameters

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full behavioral burden. It usefully discloses immediate clicking, ambiguity handling, fallback behavior, and that the result reports method and UI change, but it omits side effects, permissions, and browser-vs-desktop mode differences.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is short, front-loaded with the primary click behavior, and every sentence adds actionable information about fallback, ambiguity, or reporting. There is no filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 3-parameter tool with no annotations, the description is under-specified. It covers the core click behavior and ambiguity fallback, but does not explain the mode or double_click parameters, and provides no safety or permission context.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It clarifies target_description via 'uniquely named visible DOM target' but does not explain the mode enum (browser/desktop) or double_click, leaving two of three parameters undocumented.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: click a uniquely named visible DOM target, with a fallback to D1 vision. It implies a smart-targeting behavior distinct from raw mouse_click, but does not explicitly name sibling tools, so it falls short of full sibling differentiation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives clear usage conditions: use when the target is uniquely named and visible, avoid when duplicate names make it ambiguous, and fall back to D1 vision otherwise. It does not explicitly compare with coordinate-based mouse_click, leaving one alternative implicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

smart_moveA

Locate a described target and move/hover without clicking. Desktop mode uses a unique accessible control or D1 vision and the real OS pointer; browser mode uses DOM or D1 and moves both the tab mouse and the real OS pointer. Low confidence or a stale target prevents movement.

NameRequiredDescriptionDefault
modeNodesktop
target_descriptionYes

Output Schema

NameRequiredDescription

No output parameters

TDQS

A3.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the burden well: it discloses that desktop mode uses accessible controls or D1 vision with the real OS pointer, browser mode moves both the tab mouse and real OS pointer, and that movement is blocked on low confidence or stale targets. It does not cover permission requirements or side effects of pointer movement, but the core behavior is well explained.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is three tight sentences with the core action front-loaded and mode-specific behavior following. Every sentence adds relevant information without redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return values needn't be explained. For a pointer-movement tool with no annotations, the description covers mode behaviors and the low-confidence/stale-target guard sufficiently; only permission or pointer-state side effects are missing, which is minor.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate for both parameters. It gives useful context for the mode enum by describing desktop vs browser behavior, but leaves target_description's expected format and matching semantics vague ('described target'). This partially compensates but leaves a clear gap.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific action ('move/hover without clicking') and resource ('described target'), clearly distinguishing it from click-oriented siblings like smart_click and mouse_click. Sibling differentiation is implicit rather than explicit, but the no-click constraint is stated.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It explains the difference between desktop and browser modes, which guides mode selection, and notes that low confidence or a stale target prevents movement. However, it never explicitly says when to use smart_move instead of siblings like mouse_move or smart_click, leaving the alternative-selection reasoning implied.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

smart_typeA

Fill a visible editable input, replacing old text; verify its exact value before optional Enter. Quote its accessible label from browser_open for a fast precise match. Check status and typed_verified before claiming success.

NameRequiredDescriptionDefault
modeNobrowser
press_enterNo
text_to_typeYes
target_descriptionYes

Output Schema

NameRequiredDescription

No output parameters

TDQS

A3.9/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and does well: it discloses that old text is replaced, that the exact value is verified before optional Enter, and that status and typed_verified should be checked before claiming success. It does not explain failure behavior or differences between browser and desktop modes.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three dense sentences with the core action front-loaded, followed by targeting advice and verification guidance. There is little waste and each sentence adds operational value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description is sufficient for the default browser typing path, especially since an output schema exists and return values need not be explained. However, for a four-parameter tool with no schema descriptions and no annotations, the unexplained mode parameter and desktop behavior leave a meaningful gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It maps well to target_description (accessible label from browser_open), text_to_type (text being filled), and press_enter (optional Enter), but it never mentions the mode parameter or the browser/desktop distinction.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource: fill a visible editable input, replacing old text. This clearly separates it from navigation and clicking tools, but it does not explicitly contrast itself with smart_click or other text-entry alternatives.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives actionable usage context: use a visible editable input, quote its accessible label from browser_open, and verify the exact value before optional Enter. It lacks explicit when-not-to-use guidance or a direct comparison to sibling tools such as smart_click.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

visual_assertB

Evaluate up to 64 visual questions in one model call. answer=null means uncertain, not confirmation. Prefer DOM field labels and typed_verified for typing; avoid repeated visual assertions for an already verified action.

NameRequiredDescriptionDefault
modeNobrowser
questionsYes

Output Schema

NameRequiredDescription

No output parameters

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and does add real value: the 64-question batching limit and the crucial 'answer=null means uncertain, not confirmation' semantics. However it omits whether the call captures a screenshot, which target is asserted against in browser vs desktop mode, and any cost/latency or side-effect profile.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three dense sentences, front-loaded with the batching capability and the null-semantics warning. Minor deduction: 'typed_verified' is referenced without definition and is not one of this tool's parameters, which may momentarily confuse an agent.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return-value explanation is not owed, and the null semantics cover the most important part of the response. Still, an agent does not learn the difference between browser and desktop mode or what visual source is being asserted against, which matters for a vision tool in a 14-sibling browser/desktop toolbox.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate and largely does not. It clarifies the `questions` array's cap (64) and element type, but the `mode` enum (browser/desktop) is never mentioned, leaving half the parameters uninterpretable from either source.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The verb (evaluate) and resource (visual questions) are explicit, and the batching cap of 64 in one model call is a concrete distinguishing trait. It does not state that the questions are answered against a captured screen, so an agent must infer the screenshot/vision mechanism from the name, but the intent is unambiguous and separable from siblings.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It names preferred alternatives ('Prefer DOM field labels and typed_verified for typing') and an explicit when-not ('avoid repeated visual assertions for an already verified action'). What is missing is a positive trigger — the description never says 'use this when DOM inspection is unavailable/inconclusive'.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

wait_until_readyA

Wait for DOM readiness in hybrid mode, or visually in vision mode. Skip this call when browser_open returned ready=true. Readiness does not confirm task success or rule out a captcha.

NameRequiredDescriptionDefault
modeNobrowser
timeout_secondsNo

Output Schema

NameRequiredDescription

No output parameters

TDQS

A3.6/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and does useful work: it warns that readiness does not confirm task success and does not rule out a captcha, which sets correct agent expectations about false positives. It omits what happens on timeout (error vs. partial result), which is the main remaining behavioral unknown.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three short sentences, the skip condition is front-loaded before the caveats, and nothing is padding. The only waste is the mode-name mismatch, which costs clarity rather than length.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists so return shape needn't be explained, and the caveats cover the main conceptual trap. But for a two-parameter timing tool, the absence of any timeout semantics (unit behavior on expiry, whether it throws) and the mode/enum mismatch leave real gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, so the description must explain both parameters. It gestures at 'mode' but with terminology (hybrid/vision) that does not match the enum values browser/desktop, and it says nothing at all about timeout_seconds (default 10, presumably a maximum wait). The parameter guidance is actively confusing rather than merely absent.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

Names a specific action (wait for readiness) and distinguishes two operating contexts, and explicitly separates itself from browser_open by referencing that tool's ready=true return. However, it uses 'hybrid mode' and 'vision mode' while the schema enum is browser/desktop, so the agent must guess the mapping.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives an explicit skip condition ('Skip this call when browser_open returned ready=true') which is exactly the kind of when-not guidance that prevents redundant calls. It does not, however, say when vision/desktop mode should be preferred over browser mode.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 15 tool updatesv0.1.0
    • First observedbrowser_open
    • First observedbrowser_scroll
    • First observeddesktop_inspect
    • First observeddraw_path
    • First observeddraw_shape
    • First observedhandle_popup
    • First observedmouse_click
    • First observedmouse_drag
    • First observedmouse_move
    • First observedscroll_until_visible
    • First observedsmart_click
    • First observedsmart_move
    • First observedsmart_type
    • First observedvisual_assert
    • First observedwait_until_ready

TDQS

A3.5/5.0

Scored across 15 tools

Disambiguation3/5

Several tool families intentionally overlap: smart_click vs mouse_click, smart_move vs mouse_move, browser_scroll vs scroll_until_visible, and mouse_drag vs draw_path vs draw_shape. The descriptions do provide explicit tie-breakers (semantic target vs raw coordinates, deterministic vs visual inference), but the agent must reason about more than one or two confusable pairs before choosing.

Naming Consistency4/5

All names are snake_case and mostly verb_noun or prefix_noun (browser_open, mouse_drag, handle_popup, draw_shape). Minor deviation is the 'smart_' adjective prefix family (smart_click/smart_type/smart_move) and the outlier scroll_until_visible, but the pattern remains readable and predictable.

Tool Count4/5

15 tools is within a healthy range for a browser+desktop automation agent, and each tool maps to a real primitive (open, click, type, scroll, wait, popup, draw). A few could be consolidated (dual smart_/mouse_ pointer pairs, three drag/draw tools), making it slightly heavy but not bloated.

Completeness3/5

Core browse-and-manipulate loop is covered, but notable gaps exist: no general key-press/shortcut tool, no back/forward/tab navigation, no page text/content extraction beyond a short preview, and no desktop-app typing (smart_type is scoped to visible editable browser inputs). These omissions will force agents into workarounds on non-trivial tasks.

Maintenance

ActivityMaintained
ResponsivenessNo recent community issues

Related MCP Connectors

Related MCP Servers

  • A
    license
    Not graded
    quality
    D
    maintenance
    Enables full Linux desktop control including windows, mouse, keyboard, clipboard, audio, screenshots, OCR, accessibility, and system management through MCP-compatible AI agents.
    1
    MIT
  • A
    license
    Not graded
    quality
    C
    maintenance
    Enables AI agents to capture multi-monitor screenshots with coordinate grids and perform mouse, keyboard, scrolling, dragging, and window-management actions on Windows via MCP tools or CLI. It provides pixel-accurate desktop automation for computer-use agents.
    MIT
  • A
    license
    Not graded
    quality
    C
    maintenance
    Enables AI agents to access and act on web, desktop, and mobile interfaces via semantic accessibility trees, using MCP tools for querying, navigating, and executing deterministic UI actions without relying on screenshots or pixel coordinates.
    13 npm
    Apache 2.0
  • A
    license
    A
    quality
    B
    maintenance
    Enables MCP-speaking AI agents to operate a desktop and browser through structured UI inspection, mouse/keyboard gestures, tab-scoped browsing, and approved local screen recording with editing.
    3
    1
    MIT