Skip to main content
Glama
kumard3

Ghosthand

by kumard3
OverviewSchemaSourceRelated ServersDiscussionsAdmin
TypeScript
Local

AgentCursor

Local, free, human-like cursor for AI agents: any Mac app or browser tab, over MCP.

AgentCursor gives you (and any coding agent or automation script) a real browser driven with visible, convincingly human cursor movement and timing.

Use it as:

  • A powerful MCP tool for Claude, Cursor, Grok, custom agents, etc.

  • A realistic E2E / acceptance testing tool that works on actual production sites (human paths + timing are more resilient than robotic Playwright clicks).

  • A workflow automation engine for complex multi-step processes (logins, form flows, data entry, admin tasks) with natural hover, move, type, and scroll behavior.

  • A debugging / demo automation tool (the cursor is visible so you can watch exactly what the automation did).

All local. All free. MIT licensed. No cloud, no paywalled stealth.

The major browser automation MCPs often make realistic movement a cloud-only feature. AgentCursor brings the realistic cursor to your local machine for agents and traditional testing/automation use cases.

Status: phase 1 (Chrome extension) and phase 2 (macOS OS-cursor for genuinely trusted events) are both implemented. See docs/DESIGN.md.

Quick start

git clone https://github.com/kumard3/agentcursor.git
cd agentcursor && pnpm install && pnpm build
node dist/index.js setup

setup starts the local service, lists the AI apps it finds on your machine (Claude Code, Cursor, VS Code, Codex, Windsurf, Claude Desktop, Gemini CLI) and opens a setup page at http://127.0.0.1:8931. From there you connect each app with one click, grant the two macOS permissions, load the optional Chrome extension, and press Test the cursor. Prefer the terminal? node dist/index.js setup --all connects every detected app (or --client=cursor,codex).

Then ask your AI app something like: "Use agentcursor: open Notes, create a new note and write a 3 item shopping list."

How it fits together: every AI app launches agentcursor as an ordinary stdio MCP server. The first launch starts one shared background service (MCP over HTTP at 127.0.0.1:8931/mcp, the extension bridge on 8930) and every later launch reuses it, so several apps can use AgentCursor at once without fighting over a port. After a rebuild, the next launch replaces an older running service; an idle service exits after 10 minutes. The service only accepts requests from this machine and rejects other websites (Host and Origin checks).

Related MCP server: mcp-browser-use

Changelog (key updates)

  • Unreleased: E2E testing. AgentCursor.launch() starts a private Chrome (throwaway profile, its own copy of the extension on a free port) with its own visible cursor, so tests never touch your mouse, your browser or the MCP server on 8930, and several can run side by side. Works headed or headless: true. Computer use reaches the SDK too: Desktop.open("Notes") drives any Mac app with the real cursor and text-first reads. New agent CLI: every tool is a shell command (agentcursor read_page, agentcursor desktop_click --text Save) sharing one background service, so no tool schemas ever enter the model's context. Refs now stay with an element across reads, so read_page --changes and desktop_read --changes return only what changed: a repeat read of an unchanged window costs ~8 estimated tokens instead of ~397. Page reads got ~43% smaller, and the stdio tool list dropped from about 3,178 to 2,735 estimated tokens. New expect() with auto-retrying matchers (toBeVisible, toBeHidden, toHaveText, toContainText, toHaveCount, toHaveURL, .not). navigate now waits for the page to load, commands wait briefly for the content script after a navigation, isVisible()/count() answer immediately instead of waiting 5s, and headless screenshots no longer return the previous frame.

  • 0.4.0: Desktop control for any Mac app: desktop_read returns the window as compact text with [dN] refs from the macOS accessibility tree (a small Swift helper), then desktop_click / desktop_type / desktop_key / desktop_scroll drive the real cursor and keyboard with the same persona-based human motion. desktop_screenshot is a cropped, downscaled fallback. Chromium and Electron apps get their accessibility tree switched on automatically. New onboarding: agentcursor setup plus a local setup page that connects seven AI apps, checks permissions and tests the cursor. All AI apps now share one background service (stdio launches proxy to it), which removes the port 8930 already in use failure. The extension WebSocket now rejects connections from web pages.

  • 0.3.0: Programmatic SDK — import { AgentCursor } from "agentcursor" with a Playwright-shaped locator API (getByRole/getByText/getByLabel/getByPlaceholder/getByTestId/css + chaining + filter/nth + click/type/fill/hover/dragTo/press/scrollIntoView + boundingBox/isVisible/count/waitFor), every action driven by the human cursor. connect() and os() lifecycles. Library entry split from the MCP bin so importing the package no longer boots a server; built with tsup (ships .d.ts). Locator resolution uses @testing-library/dom in the content script.

  • 0.2.9: Active-tab resolution no longer requires Chrome to be the OS-focused window — it falls back to the active tab in any window, then any open http(s) tab. Fixes No active tab found when an agent drives the browser while you're in your editor/terminal (the normal case).

  • 0.2.8: Interaction: press_key — press Enter / Escape / Tab / arrows / Home / End / etc. on the focused element, content or stealth (trusted CDP key event). Rounds out the Comet-style action set: Navigation, Identification, Interaction.

  • 0.2.7: Identification tools — find (locate elements by visible text / accessible name, shadow-DOM aware) and click_text (find the best text match, then human-move + click, content or stealth, with a re-read retry). Target by what the element says, not by ref or pixel coords.

  • 0.2.6: Stealth typing inserts the whole string in one Input.insertText call. Per-character insertion landed at a reset caret on controlled editors (X's Draft.js) and typed text backward.

  • 0.2.5: Content-script drag reports the held button (buttons mask) during the move, matching the stealth and OS drivers, so JS drag handlers see a real drag. Added a Known Limitations section.

  • 0.2.4: read_page resolves multi-ID aria-labelledby names (it was passing the whole space-separated list to getElementById as one ID, so those labels came back empty); now shadow-DOM-tree-scope aware.

  • 0.2.3: drag now performs a real drag (press at the start, move with the button held, release at the end) in the stealth (chrome.debugger) and OS-cursor drivers — previously a move-then-click. pnpm smoke asserts the screenshot image content.

  • 0.2.2: drag tool. screenshot now returns a viewport-scaled image (1 image pixel = 1 click coordinate) for a vision loop — see the page, then click/move_to by x/y. Stealth (chrome.debugger) moves also animate the visible overlay, so the cursor stays on screen. Ships as a Claude Code plugin. Internal snapshot refresh resolves refs past the 60th element. pnpm build no longer mutates version files (use pnpm reload for the extension dev loop).

  • 0.2.0: Added screenshot, hover, status MCP tools. Deep shadow DOM traversal in read_page / snapshot (critical for X.com, Reddit, modern SPAs). Library re-exports for programmatic use. Repositioned as general local automation/testing/workflow tool over MCP. Version bumps and packaging polish.

  • 0.1.0: Initial MCP server, human path engine, extension bridge, OS cursor driver, basic tools (read_page, click, type, etc.).

How it works

Three layers, with one shared wire contract (src/protocol):

coding agent ──MCP/stdio──▶ MCP server ──localhost WebSocket──▶ Chrome extension ──▶ your real tab
                            (src/server)                        (extension/)
                                 │
                                 ▼
                          human-path engine (src/path-engine)
            from + to → timed cursor samples with overshoot, log-normal
            velocity, jitter, off-center landing, dwell — fresh every call

The MCP server generates the cursor sample stream; the extension is a thin replayer. The same stream works for the content-script driver, the chrome.debugger stealth driver, and (phase 2) the OS cursor — they all implement one BrowserDriver interface.

Why human-like movement is hard

Modern detectors (DataDome, Castle, reCAPTCHA v3, PerimeterX) flag overly smooth Bézier paths, constant velocity, dead-center clicks, zero dwell, teleporting jumps, and replayed identical paths. The engine addresses each:

  • Fitts's law sets per-move duration from distance and target size.

  • Asymmetric, eased velocity — not a symmetric min-jerk bell.

  • Overshoot-and-correct on long moves.

  • Sub-pixel Gaussian jitter, zero at the endpoints.

  • Off-center landing inside the target.

  • Right-skewed dwell before the press.

  • Per-call entropy — paths are never cached or replayed.

Realism is necessary but not sufficient: content-script events are isTrusted=false, and chrome.debugger still leaks CDP tells. The real evasion endgame is the phase-2 OS cursor (genuine, trusted OS events).

Install

git clone https://github.com/kumard3/agentcursor.git
cd agentcursor
pnpm install
pnpm build      # builds dist/index.js + extension/dist/*

Install as a Claude Code plugin (one step)

AgentCursor ships as a Claude Code plugin that registers the MCP server for you:

claude plugin marketplace add kumard3/agentcursor
claude plugin install agentcursor

That registers the agentcursor MCP server automatically (no manual claude mcp add). You still load the extension once (step 1 below) if you want browser tabs. If you previously registered it by hand, remove that to avoid duplicate tools: claude mcp remove agentcursor.

1. Load the extension

  1. Open chrome://extensions, enable Developer mode.

  2. Load unpacked → select the extension/ folder.

  3. Keep a normal http(s) tab open and focused (not chrome:// or the Web Store — content scripts can't run there).

2. Connect via MCP (agents, Cursor, Claude, custom tools, etc.)

The easy way is node dist/index.js setup (see Quick start). To do it by hand:

Claude Code:

claude mcp add --scope user agentcursor -- node /absolute/path/to/agentcursor/dist/index.js

Cursor, Windsurf, or any MCP-capable coding environment:

Add to your MCP servers config (exact format depends on the host):

{
  "mcpServers": {
    "agentcursor": {
      "command": "node",
      "args": ["/absolute/path/to/agentcursor/dist/index.js"]
    }
  }
}

Any other MCP client (including future Grok harnesses, custom agents, test runners that speak MCP) — just point it at the stdio server the same way.

HTTP-capable clients can skip the stdio launcher and point straight at http://127.0.0.1:8931/mcp while the service runs (node dist/index.js serve keeps it in the foreground).

The service exposes the WebSocket bridge on ws://127.0.0.1:8930 (override with AGENTCURSOR_WS_PORT; the HTTP port defaults to that plus one, or set AGENTCURSOR_HTTP_PORT). The extension auto-reconnects. AGENTCURSOR_TOOLS=desktop or =browser registers only one tool family, which keeps unused tool definitions out of your context.

3. Programmatic SDK (import { AgentCursor })

Drive the human cursor from your own Node/TS code with a Playwright-shaped locator API — no MCP client needed. Same engine, same stealth; every action moves a real cursor.

import { AgentCursor } from "agentcursor";

// Attach to a running Chrome that has the extension loaded.
// Pass { stealth: true } for trusted CDP events; use AgentCursor.os() for the nut-js OS cursor.
const ac = await AgentCursor.connect();

await ac.navigate("https://example.com");
await ac.getByRole("button", { name: "Buy now" }).click();
await ac.getByLabel("Email").fill("a@b.com");
await ac.getByText("Submit").click();
await ac.getByLabel("Email").press("Enter");

// chaining + filtering, just like Playwright
await ac.locator(".row").filter({ hasText: "Pro" }).nth(0).getByText("Edit").click();

if (await ac.getByTestId("checkout").isVisible()) {
  await ac.getByTestId("checkout").click();
}

await ac.screenshot({ path: "out.png" });
await ac.close();

Lifecycles

  • AgentCursor.launch({ headless?, userDataDir?, args?, executablePath?, seed?, stealth? }): start a browser with its own cursor. executablePath picks the binary (Chrome for Testing, Brave, Edge, BrowserOS); userDataDir drives a real, logged-in profile with no copy and leaves it intact; headless runs it in the background. See Attach any browser and E2E tests.

  • AgentCursor.connect({ port?, stealth?, timeoutMs? }): attach to a running Chrome with the extension loaded (works with your real, logged-in profile).

  • AgentCursor.os({ stealth?, ... }): same locator API, but the real OS cursor is moved via nut-js (genuinely trusted events); page sensing still goes through the extension.

Run JS in the page: ac.evaluate(fn, ...args), Playwright-shaped, runs in the page realm via CDP (page cookies/session, awaits promises, ignores the page CSP):

const title = await ac.evaluate(() => document.title);
const res = await ac.evaluate(async (id) => {
  const r = await fetch(`/api/item/${id}`, { method: "POST", credentials: "include" });
  return { status: r.status, body: await r.text() };
}, 42);

Locators (lazy, chainable, Playwright-shaped)

  • Find: locator(css), getByRole(role, { name }), getByText, getByLabel, getByPlaceholder, getByTestId.

  • Refine: .filter({ hasText }), .nth(i), .first(), .last(), and chaining (a.locator(b)).

  • Act: .click(), .dblclick(), .hover(), .type(), .fill(), .press(key), .dragTo(other), .scrollIntoView().

  • Query: .boundingBox(), .textContent(), .isVisible(), .count(), .waitFor({ state }).

Each action resolves the locator in the page (role/label/etc. via @testing-library/dom, css/text via the DOM), then drives the human-path engine to the element. ac.actions exposes the lower-level ActionService (move by coords, find, clickText, scroll) as an escape hatch.

A full runnable example is in examples/sdk-quickstart.mjs; pnpm smoke:sdk runs the end-to-end pipeline against a simulated browser.

Note: in-page (stealth: false) events are isTrusted=false. Use stealth: true (CDP) or AgentCursor.os() when you need trusted events.

4. CLI for coding agents (agentcursor <command>)

Every MCP tool is also a shell command. One background service holds the page, the [refs] and the frontmost app, so commands chain like a session, and an agent that uses the CLI loads no tool schemas at all.

agentcursor help                        # one line per command (agentcursor tools for full descriptions)

# browser
agentcursor navigate https://example.com
agentcursor read_page --includeText false
agentcursor click_text "Buy now"
agentcursor read_page --changes         # only what changed since your last read
agentcursor find Submit                 # a handful of tokens instead of a whole page
agentcursor screenshot                  # writes a file, prints the path

# any Mac app (computer use)
agentcursor desktop_open Notes
agentcursor desktop_read --max 40       # the window as text, not pixels
agentcursor desktop_click --text "New Note"
agentcursor desktop_type "hello" --submit

Positionals fill a command's parameters in the order agentcursor help lists them, and anything can be given as --flag value. Dashes work too (click-text). AGENTCURSOR_OUT sets where screenshots are written.

Reading only the changes. A [ref] stays with the same element for the life of the page (or until you switch apps), so a ref you learned earlier keeps working and reads can be compared. Pass --changes to read_page or desktop_read and you get the added and removed lines plus a count of the ones that merely moved:

- [e5] text "Email" <input> @293,300
+ [e5] text "Email" value="a@b.com" <input> @293,300
(5 moved, 2 unchanged, 10 total)

Measured on a Finder window: a repeat read costs 29 chars (~8 estimated tokens) against 1390 (~397) for the full read. The first read, or a diff that would be bigger than the full read, returns the full read instead.

5. Computer use from the SDK (Desktop)

import { Desktop } from "agentcursor";

const d = await Desktop.open("TextEdit");     // seed: 7 for reproducible motion
console.log(await d.read());                  // window as compact text with [dN] refs
await d.type("Dear team,", { clear: true });
await d.click("Save");                        // a [dN] ref or any visible label
await d.key("cmd+s");
if (await d.waitForText("Saved")) console.log("done");

find() returns only matching elements (tens of tokens), view() gives the structured elements, and screenshot({ path }) is the fallback when the text is not enough. macOS only, and it needs Accessibility permission for whichever app starts it.

Background runs, and a cursor per session. By default computer use moves the one real pointer, which takes your machine over. background: true posts input straight to the target process instead: your pointer never moves, the app is never raised, and every Desktop keeps a cursor of its own, so runs happen while you work and several can run at once.

const alice = await Desktop.open("Notes", { background: true, showCursor: { color: "#4ade80", label: "alice" } });
const bob = await Desktop.open("Reminders", { background: true, showCursor: { color: "#60a5fa", label: "bob" } });
await Promise.all([alice.type("from alice"), bob.type("from bob")]);

showCursor is off unless you ask for it: it draws a click-through cursor above every window, coloured and named, so a background run is watchable. For MCP and the CLI, set AGENTCURSOR_BACKGROUND=1 and AGENTCURSOR_SHOW_CURSOR=1.

Measured on a background TextEdit while the terminal stayed frontmost: the text landed, the frontmost app did not change, and the pointer sat at 471,628 before and after.

The limit worth knowing: this works for native (AppKit) apps. Chromium and Electron apps ignore process-posted events, measured on Chrome, which ignored them even while frontmost, so it covers Notes, Mail, Finder, TextEdit and friends but not Chrome, Arc, VS Code, Slack or Discord. For browsers use AgentCursor.launch({ headless: true }), which is fully background anyway and draws its cursor in the page.

Driving a browser with computer use instead of the DOM. A page can also be read and clicked as an app, with no extension and no DOM: the clicks are real OS clicks, so the page sees isTrusted=true. Chrome only builds its accessibility tree for a screen reader, so ask for it at launch:

const ac = await AgentCursor.launch({ accessibility: true });   // --force-renderer-accessibility
await ac.goto("http://localhost:3000");

const d = await Desktop.open();
console.log(await d.find("Menu"));     // [d23] button "Menu" @87,481
await d.click("Menu");                  // real cursor, trusted click

Without that option a desktop read of Chrome sees the toolbar and no page content (16 elements against 61 on the same window, measured). For your own Chrome, start it with --force-renderer-accessibility.

Which path to pick:

Extension (DOM)

Computer use (accessibility tree)

Events

synthetic, or trusted with stealth: true

always real OS input

Reads

~112 est tokens for a page

~609 for the same window, browser chrome included

Runs headless / in parallel

yes

no: needs a visible, frontmost window and the real mouse

Works outside the browser

no

any Mac app, including Electron

find costs about 8 tokens on either path, and --changes works for desktop reads too.

Attach any browser (real profile, in the background)

AgentCursor drives any Chromium browser (Chrome, Chrome for Testing, Brave, Edge, BrowserOS, and mostly Arc) using your real, logged-in profile, in the background (it dispatches synthetic or CDP events, so the window need not be focused and your physical mouse is never touched). This is "computer use, but token-cheap and headless": read_page / find / evaluate cost tens to a few hundred tokens each versus a screenshot vision loop. os() is the only mode that runs in the foreground (it moves the real OS cursor).

Three ways to attach:

Mode

Browser

Profile

Background

Setup

Extension (connect() / MCP, default)

any Chromium

your real one

yes

Load unpacked once

agentcursor launch / AgentCursor.launch({ userDataDir })

any Chromium binary

real (no copy) or throwaway

yes (--headless)

one command

os()

frontmost

real

no (moves your mouse)

Accessibility grant

A. Extension mode (recommended for agents). Load extension/ unpacked into the browser you want (its real profile), point your MCP client at the stdio server, and drive it. See Load the extension and Connect via MCP. Nothing is copied; the browser you already use is the one being driven.

B. agentcursor launch (one command, real profile, no manual extension load). Starts a browser wired to the running service and holds it open until Ctrl-C:

# real, logged-in profile, in the background:
agentcursor launch \
  --user-data-dir "$HOME/Library/Application Support/Google/Chrome" \
  --chrome "/Applications/Google Chrome for Testing.app/Contents/MacOS/Google Chrome for Testing" \
  --headless

# throwaway profile, visible window (the default):
agentcursor launch

Flags: --user-data-dir DIR (real profile, left intact; omit for a throwaway copy), --chrome PATH (any Chromium binary; also AGENTCURSOR_CHROME), --headless, --port (WS port, default 8930). Then drive it with the normal tools (agentcursor read_page, click_text, evaluate, ...).

Caveats:

  • Profile lock: with --user-data-dir, the browser must be fully quit first. Chromium will not share a running profile's data dir.

  • Arc: Chromium underneath, but it wraps its own launch, so --user-data-dir + the CDP pipe is flaky. For Arc, prefer extension mode (A).

  • Same-origin fetch inside evaluate carries that profile's cookies, so authenticated requests work against the logged-in session.

Tools (MCP)

Tool

What it does

read_page

Interactive elements with stable [ref] handles, roles, rects, visible text. Call this first in almost every test or workflow.

find

Identification: locate elements by their visible text / accessible name (shadow-DOM aware). Returns ranked [ref] matches — target by what the element says, no ref needed.

click_text

Identification + interaction in one step: find the best text match and human-move + click it (re-reads if needed). Supports nth, double, stealth.

move_to

Human-like path to a [ref] or x/y (no click).

click

Full human move + click (supports button, double, stealth mode for trusted events).

hover

Human approach + hover events (mouseover/mouseenter). Critical for dropdowns, tooltips, nav, and realistic workflows.

drag

Human path drag from ref/coords to target while holding button (sliders, reorder, canvas).

type

Human-timed keystrokes (auto human-clicks ref to focus if provided).

press_key

Press a single key (Enter, Escape, Tab, arrows, Home/End, PageUp/Down, Space, or a character) on the focused element. content or stealth.

scroll

Eased, human-stepped scrolling.

screenshot

Capture the visible tab as an image, scaled so 1 image pixel = 1 click coordinate — see the page, then click/move_to by x/y (the vision loop). Also for visual assertions and agent grounding.

navigate

Load a URL in the active tab.

get_url

Current tab URL.

evaluate

Run a JS function in the page (CDP Runtime.evaluate): page realm, uses the page's own cookies/session, awaits promises, ignores the page CSP. For reads and requests with no button (() => document.title, async () => (await fetch('/api/x', { method: 'POST', credentials: 'include' })).status). Returns the JSON result. Shows the debugger banner while it runs.

wait_for

Wait for element ref or visible text (up to timeout). Use for resilient testing flows.

status

Health / connection status, driver, active URL, port. Great for CI, long-running workflows, and monitoring.

Any driving action accepts stealth: true to deliver trusted events through the chrome.debugger driver (this shows Chrome's "debugging this browser" banner).

Desktop tools (any Mac app)

No extension needed. Grant Accessibility (and Screen Recording, only for screenshots) to the app your AI runs in; the setup page has buttons for both.

Tool

What it does

desktop_apps

Running apps, frontmost marked.

desktop_open

Open or switch to an app by name and bring it to the front.

desktop_read

The window as compact text: buttons, fields, links, menus, list items and visible text, each with a [dN] ref and center point. find returns only the best matches for a label.

desktop_click

Human path + click on a [dN] ref, a visible label (text), or x/y. Double and right click supported.

desktop_move

Move there without clicking (hover menus, tooltips).

desktop_type

Persona-timed typing into the focused field, or click a field first (ref, into label, x/y). clear replaces, submit presses Enter.

desktop_key

Keys and shortcuts: enter, esc, tab, cmd+s, cmd+shift+t.

desktop_scroll

Scroll by pixels over a target or where the cursor is.

desktop_screenshot

One window (or the area around a ref), downscaled JPEG, with the formula to turn image pixels into screen x/y. For canvases and custom-drawn UI.

Why text first. Measured on a MacBook (text tokens estimated at 3.5 characters per token; image tokens from Anthropic's width x height / 750):

App

desktop_read (whole window)

desktop_read find / click by label

Screenshot at 1024px wide

Finder

~454

~9

662

System Settings

~543

~8

1181

Arc

~705

~9

852

The bigger saving is the loop: clicking by ref or label needs no screenshot to find coordinates and none to check where the click landed.

E2E tests: a Playwright alternative with a visible cursor

AgentCursor.launch() opens its own Chrome with its own cursor. Your mouse stays yours, your normal browser and the MCP agent's cursor keep working, and every launch is independent, so two launches give you two cursors (two users in a chat app, a buyer and a seller). Use any runner; this is plain node:test:

import { test } from "node:test";
import { AgentCursor, expect } from "agentcursor";

test("checkout", async () => {
  const ac = await AgentCursor.launch({ seed: 7 }); // headless: true for CI
  try {
    await ac.goto("http://localhost:3000");
    await ac.getByRole("button", { name: "Buy now" }).click();
    await ac.getByLabel("Email").fill("agent@cursor.dev");
    await ac.getByRole("button", { name: "Submit" }).click();
    await expect(ac.getByText("Order placed")).toBeVisible();
    await expect(ac).toHaveURL(/\/thanks$/);
  } finally {
    await ac.close();
  }
});

test("two cursors at once", async () => {
  const [alice, bob] = await Promise.all([
    AgentCursor.launch({ args: ["--window-position=0,0", "--window-size=760,900"] }),
    AgentCursor.launch({ args: ["--window-position=780,0", "--window-size=760,900"] }),
  ]);
  // alice and bob act in parallel, each with a visible cursor in its own window
  await Promise.all([alice.close(), bob.close()]);
});

  • expect(locator): toBeVisible, toBeHidden, toHaveText (string or RegExp), toContainText, toHaveCount, and expect(ac).toHaveURL. Each retries until it passes or { timeout } (default 5s) runs out; .not inverts.

  • Chrome is found automatically (or executablePath / AGENTCURSOR_CHROME). The extension is loaded over --remote-debugging-pipe, so stable Chrome works.

  • Same seed means the same motion and typing, so runs (and recorded demos) are reproducible.

  • Not there yet: multiple tabs per launch, network mocking, traces, and non-Chromium browsers. Default events are isTrusted=false; pass stealth: true for trusted CDP input.

pnpm e2e runs test/e2e/showcase.e2e.mjs (HEADLESS=1 pnpm e2e for CI).

Using as a Testing & Workflow Automation Tool

AgentCursor is not only for agents — it's a practical local browser automation primitive you can use directly in tests and scripts via MCP or by importing the core.

Why it shines for testing/automation on real sites:

  • Human cursor paths + dwell + jitter + off-center clicks make interactions look like a real person (useful when sites have light behavioral signals).

  • The visible cursor + overlay makes it excellent for demo videos, manual review of automation, and debugging failing flows.

  • screenshot + read_page + wait_for + hover give you the primitives for visual + functional checks.

  • Works against your real Chrome profile (cookies, extensions, logins) — perfect for realistic E2E that headless tools struggle with.

Example flow an agent or a test script might do:

read_page
hover "nav-menu"
click "Products"
wait_for text:"Featured"
screenshot
type {ref: "search", text: "laptop"}
click "search button"
...

Direct / programmatic use (API style): The core ActionService, path engine, and drivers are designed to be importable. See "Programmatic Use" below.

Example: Using with Claude Code to post on X.com / Reddit

With the MCP integration, you can tell Claude Code (or Cursor) to use agentcursor for realistic posting/automation on real sites:

  1. Have a logged-in tab open on x.com (or reddit.com).

  2. Start the server (ideally with OS driver on mac for best results).

  3. In Claude: "Add agentcursor MCP if not present, then use the tools to navigate to x.com if needed, read the page, hover and click the compose area, type a test post, screenshot for verification, and click the post button. Use human-like actions and wait_for as needed. Report status often."

The shadow DOM support (added in 0.2.0) helps surface elements inside X's web components. Combine with screenshot + status + loops of read_page / wait_for for resilience on SPAs.

See the testing section above for general flow patterns. Always start with status and read_page, use screenshot to ground the agent.

Trusted OS cursor (phase 2, macOS)

Content-script events are isTrusted=false, and chrome.debugger still leaks CDP tells. For genuinely trusted, indistinguishable input, switch to the OS-cursor driver, which moves the real macOS system cursor along the same human path:

pnpm add @nut-tree-fork/nut-js        # optional native dependency
AGENTCURSOR_DRIVER=os node dist/index.js

It still reads the page through the extension (keep a normal tab focused), but every move/click/scroll becomes a real OS event. Requires the Chrome window visible and foregrounded at 100% zoom, and Accessibility permission for your terminal/Node in System Settings → Privacy & Security. Coordinate mapping for multi-monitor / fractional-scaling setups is still rough.

Known limitations

  • Desktop control is macOS only for now. It reads the accessibility tree, so apps that draw their own UI without accessibility (games, some canvases) need desktop_screenshot plus x/y clicks.

  • macOS grants permissions to the app that launched AgentCursor (Terminal, Cursor, Claude...), not to AgentCursor itself. If the shared service was first started from a different app, grant that one, or stop the service (curl -X POST http://127.0.0.1:8931/shutdown) and let your main app start it.

  • Desktop typing does not render typo corrections. The persona's timing applies, but only the final characters are typed.

  • Content-script events are isTrusted=false. For detection-sensitive sites pass stealth: true (the chrome.debugger driver, trusted events) or use the OS-cursor driver. The visible overlay cursor shows in every mode.

  • React-controlled inputs (X's composer, some design systems) can ignore content-script typing, which sets value directly. Use stealth: true (CDP Input.insertText) or the OS driver there.

  • wait_for by text and the snapshot text field use innerText, which does not pierce shadow DOM. read_page's element list does traverse shadow roots, so prefer waiting on a [ref] over page text on web-component-heavy sites (X, Reddit).

  • The OS-cursor driver assumes 100% browser zoom and a single display; multi-monitor and fractional scaling can be off.

  • hover dispatches its hover events through the content script (the approach move is trusted under stealth, the explicit mouseover/mouseenter are not).

  • Content-mode press_key carries key/code but not legacy keyCode — a constructed KeyboardEvent always reports keyCode: 0. Modern handlers read key; for sites that still check keyCode/which, use stealth: true (the CDP key event sets the real virtual key code).

Measuring realism

Serve the detector over http (the extension's content script only runs on http(s), not file://):

python3 -m http.server 8080 --directory test-detector

Open http://localhost:8080, click the targets by hand, then drive them with the agent. Each click is scored on straightness, velocity variance, dwell, off-center landing, overshoot, and isTrusted — the same features detectors use. Use it to tune the engine.

Development

pnpm dev         # run the server with tsx (no build)
pnpm typecheck    # tsc --noEmit
pnpm test             # vitest (path-engine + coord-map unit tests)
pnpm build:ext    # rebuild just the extension (no version change)
pnpm reload       # rebuild the extension AND patch-bump the version, so a chrome://extensions reload is visibly new
pnpm smoke        # end-to-end run: real MCP client + server, simulated browser (now covers screenshot/hover/status too)
pnpm build:native # rebuild the macOS accessibility helper (dist/native/agentcursor-ax)
AGENTCURSOR_HTTP_PORT=8931 node scripts/desktop-live.mjs   # live desktop run: reads Finder, types in TextEdit, closes it without saving

The smoke script is also a good template for writing your own automation or test runners that drive AgentCursor over MCP.

Credits

The path engine builds on the ghost-cursor lineage (Bézier + Fitts) and the mouse-dynamics literature — WindMouse, SapiAgent, BeCAPTCHA-Mouse, and the vendor write-ups from DataDome and Castle on what makes synthetic movement detectable. See docs/DESIGN.md.

License

MIT

Available Tools

15 tools
clickB

Human-like move + click on an element ([ref]) or x/y. Supports button, double-click, and stealth (trusted-event) mode.

ParametersJSON Schema
NameRequiredDescriptionDefault
xNo
yNo
refNo
buttonNo
doubleNo
stealthNo

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, description adds value by mentioning 'human-like move' and 'stealth (trusted-event) mode', but lacks details on default behavior, error handling, or interaction between x/y and ref.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Single sentence packs key information (action, target options, variations) without fluff; slightly dense but effective.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Moderate complexity (6 params, no output schema) not fully addressed; missing prerequisites, order of operations, and result description.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 0% schema description coverage, the description explains most parameters (button, double, stealth, ref, x/y) but is vague on how x/y and ref relate and lacks detail on all parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool performs a human-like move and click on an element or coordinates, and distinguishes from sibling tools like hover, move_to, and click_text by mentioning button options, double-click, and stealth mode.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance on when to use this tool vs alternatives like click_text or hover; no context on prerequisites or exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

click_textA

Identification + interaction in one step: find the element that best matches the given text/label, then human-move the cursor to it and click. Re-reads the page if the element isn't there yet. nth picks a later match, stealth:true delivers trusted events, double double-clicks.

ParametersJSON Schema
NameRequiredDescriptionDefault
nthNo
textYes
doubleNo
stealthNo

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Discloses key behaviors: re-reads page if element missing, nth picks later match, stealth delivers trusted events, double double-clicks. Lacks info on failure handling but overall transparent given no annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences with front-loaded purpose, each sentence adding value with no waste.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a complex tool with 4 parameters and no output schema, description covers re-reading, event trust, and matching variants. Lacks details on edge cases but sufficient for typical usage.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Adds meaning for nth, stealth, and double parameters, but text parameter is obvious and parameter types not described. With 0% schema coverage, description partially compensates but could be more comprehensive.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it combines identification and interaction in one step by finding an element matching text/label and clicking it, distinguishing it from siblings like click or find.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Implicitly clear that it is for text-based identification and clicking, but lacks explicit when-to-use vs. alternatives like click or hover.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

dragA

Perform a human-like drag from one element/ref or coords to another (e.g. for sliders, reordering, canvas drawing). Uses the realistic path engine while holding the mouse button.

ParametersJSON Schema
NameRequiredDescriptionDefault
toXNo
toYNo
fromXNo
fromYNo
toRefNo
buttonNo
fromRefNo
stealthNo

TDQS

A3.9/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations, so description carries full burden. Discloses 'human-like drag', 'realistic path engine', and 'holding mouse button', giving insight into behavior. Could mention whether it waits for completion or supports cancellation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with main purpose, then behavioral detail. No redundant or unnecessary words. Efficient.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With 8 parameters, 0% schema coverage, no output schema, and no annotations, the description is too brief. Does not cover parameter roles or output format. Needs more detail for a complex interaction tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, and description does not explain any of the 8 parameters. Parameter names (toX, fromY, button, stealth, etc.) are somewhat self-explanatory, but lack of clarity on 'stealth' and 'toRef' vs coordinates reduces utility.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states verb 'drag' with specific resources (elements/refs/coords) and provides examples (sliders, reordering, canvas drawing). Distinguishes from sibling tools like click, move_to, hover.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides context with examples (e.g. sliders, reordering, canvas drawing) implicitly guiding when to use. Lacks explicit when-not-to-use or alternatives, but context is sufficient for typical choices.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

findA

Identification: locate on-screen elements by their visible text or accessible name (shadow-DOM aware), the way a human scans a page. Returns ranked matches with [ref], role, and on-screen rect. Use when you don't already have a ref, then click/move_to/hover by [ref] — or use click_text to do it in one step.

ParametersJSON Schema
NameRequiredDescriptionDefault
textYes
maxResultsNo

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It discloses shadow-DOM awareness, return data (ranked matches with ref, role, rect). It implies a read-only operation by its nature. However, it does not explicitly state it has no side effects, which would be beneficial.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences: first describes functionality and output, second provides usage guidance and alternatives. It is front-loaded, concise, and every sentence adds value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given no output schema, the description covers return value well. Parameters are partially explained. Sibling tool context is provided via click_text. The description could be more complete by mentioning maxResults, but overall it is adequate.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It explains that the 'text' parameter matches visible text or accessible name, adding meaning. However, it does not explain 'maxResults' at all, leaving a gap for that parameter.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool locates on-screen elements by visible text or accessible name, is shadow-DOM aware, and returns ranked matches with ref, role, and rect. It distinguishes from siblings like click_text by noting that click_text does the find and click in one step.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It explicitly tells when to use: 'when you don't already have a ref.' It provides alternatives: 'use click_text to do it in one step.' It also suggests subsequent actions: 'click/move_to/hover by [ref].' This gives clear context and alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_urlA

Return the active tab's current URL.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description must carry the full burden. It simply states the return value does not disclose side effects, permissions needed, or any edge cases. However, as a read-only operation with no parameters, the transparency is adequate.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, precise sentence that front-loads the key purpose. Every word earns its place with no redundancy or extraneous information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given zero parameters and no output schema, the description is nearly complete. It could optionally specify whether the URL includes protocol, query strings, or fragment, but this is minor. Overall, it sufficiently informs the agent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

No parameters exist, so schema coverage is 100% by default. The description does not add anything beyond the schema, but the baseline score of 3 is elevated to 4 because there are no parameters to document.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool returns the active tab's current URL, using a specific verb and resource. It distinguishes itself from sibling tools like click, navigate, and read_page, which perform different actions.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance on when to use this tool versus alternatives like read_page or navigate. The description does not mention prerequisites, limitations, or context for use.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

hoverC

Human-like move the cursor to an element or coordinates and fire hover events (mouseover, mouseenter). Essential for dropdowns, tooltips, navigation menus, and realistic workflow/testing automation.

ParametersJSON Schema
NameRequiredDescriptionDefault
xNo
yNo
refNo
stealthNo

TDQS

C2.8/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description must fully disclose behavior. It only mentions 'Human-like' movement and events, but omits details on stealth behavior, coordinate vs element precedence, or any side effects.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences, front-loaded with the core purpose. It is efficient but could be slightly more compact. The examples add value without being verbose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the lack of annotations and output schema, the description is incomplete. It does not detail parameter usage, return values, or error conditions, leaving significant gaps for an agent to infer.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters1/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 0% schema description coverage, the description should explain all 4 parameters. It only vaguely refers to 'element or coordinates', leaving x, y, ref, and stealth completely unexplained.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('move the cursor') and the target ('element or coordinates'), and mentions the hover events fired. This distinguishes it from sibling tools like 'move_to' which may not fire events, though not explicitly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description lists typical use cases (dropdowns, tooltips, menus) but does not specify when not to use this tool or compare it to alternatives like 'move_to' or 'click'. Usage context is implied but not exhaustive.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

move_toA

Move the cursor to an element ([ref] from read_page) or to absolute viewport x/y along a human-like path. Does not click. stealth:true delivers trusted events via the debugger driver.

ParametersJSON Schema
NameRequiredDescriptionDefault
xNo
yNo
refNo
stealthNo

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries full burden. It discloses 'human-like path' and stealth mode behavior via debugger driver, but lacks details on implications like triggering hover effects or cursor path specifics.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences front-load the core purpose and add a key behavioral detail. No redundant information; every part earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given no output schema and no annotations, the description covers main use cases and common parameters. Missing details on coordinate system origin and path behavior, but sufficient for basic operation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 0% schema coverage, the description adds meaning for all parameters: ref from read_page, x/y as absolute viewport, and stealth as trusted events. However, it does not specify coordinate origin or ref format.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool moves the cursor to either an element reference or absolute viewport coordinates, and explicitly states it does not click. This distinguishes it from sibling tools like click and hover.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description indicates when to use the tool (move cursor without clicking) but does not provide explicit guidance on when not to use it or mention alternatives beyond stating it does not click.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

press_keyA

Press a single key on the focused element: Enter, Escape, Tab, Backspace, Delete, ArrowUp/Down/Left/Right, Home, End, PageUp, PageDown, Space, or a single character. Use to submit (Enter), dismiss dialogs (Escape), or tab between fields. stealth:true delivers a trusted key event via the debugger driver.

ParametersJSON Schema
NameRequiredDescriptionDefault
keyYes
stealthNo

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It discloses the stealth parameter's effect (trusted key event via debugger driver) and lists supported keys. It does not detail side effects, but for a key press action this is sufficient.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with purpose, no redundant information. Every word adds value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple action with 2 parameters and no output schema, the description is fairly complete. It covers key, stealth, and use cases. Minor omissions like behavior for unsupported keys or return value do not significantly hinder an agent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema has 0% description coverage, so the description must compensate. It adds significant meaning by listing allowed values for key and explaining stealth's effect. However, it does not specify exact format (e.g., case sensitivity) or all possible values.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it presses a single key on the focused element and lists supported keys (Enter, Escape, Tab, etc.) and common use cases. This distinguishes it from sibling tools like click and type.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly mentions when to use it (submit, dismiss dialogs, tab between fields). It does not list alternatives, but the context from sibling tool names implies other tools for other actions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

read_pageA

Read the current page: interactive elements with stable [ref] handles, their roles/names and on-screen rectangles, plus visible text. Call before clicking or typing by ref.

ParametersJSON Schema
NameRequiredDescriptionDefault
includeTextNo
maxElementsNo

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations provided, so description carries the burden. It discloses it returns refs, roles, names, rectangles, and text, and implies a read-only operation. However, it does not mention response format, size limits, or rate limits, leaving gaps in behavioral understanding.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Single sentence that front-loads the main action and key outputs. Every word adds value, no redundancy or filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No output schema, so description must compensate. It lists returned data (ref, role, name, rectangle, text) but omits structure, ordering, or parameter effects. Adequate but not comprehensive for a tool with no output schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters1/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, yet the description does not explain either parameter (includeText, maxElements). The hint 'plus visible text' vaguely relates to includeText but is insufficient. The description adds no value beyond the schema's type definitions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it reads the current page and lists specific outputs (interactive elements with ref handles, roles/names, rectangles, visible text). It also implies its role as a prerequisite for actions like clicking/typing, distinguishing it from sibling tools that perform actions.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly says 'Call before clicking or typing by ref', providing clear when-to-use guidance. It lacks explicit when-not-to-use or comparison with other read tools like find or status, but the context is strong.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

screenshotA

Capture the visible tab as an image, scaled so 1 image pixel = 1 click coordinate. SEE the page, then click(x,y)/move_to(x,y) at coordinates read off the image. This is the vision loop (screenshot -> decide coords -> click -> screenshot) and needs no DOM refs.

ParametersJSON Schema
NameRequiredDescriptionDefault
formatNo

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations provided, but description fully discloses key behavior: 1:1 pixel-to-coordinate scaling and independence from DOM, enabling informed agent decision-making.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences, front-loaded with critical information (scaling, usage context), every sentence earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given no annotations and no output schema, description covers all essential aspects for a simple tool with one optional parameter. No gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Single parameter (format) has enum in schema; description does not elaborate on it. Schema coverage is 0%, but enum values are self-explanatory. Minimal added value.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description explicitly states the tool captures the visible tab as an image with coordinate scaling, clearly distinguishing it from DOM-based sibling tools like click, read_page, etc.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Describes the vision loop pattern (screenshot -> decide coords -> click) and explicitly states it needs no DOM refs, guiding when to use vs alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

scrollC

Scroll the page by dy (and optional dx) pixels in eased human steps.

ParametersJSON Schema
NameRequiredDescriptionDefault
dxNo
dyYes
stealthNo

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description mentions 'eased human steps' suggesting smooth scrolling, but fails to explain the effect of the 'stealth' parameter or behavior when scrolling exceeds page bounds. Without annotations, the description carries the full burden and is insufficient.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence with no superfluous words. Every part adds value, making it highly efficient for a simple tool.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a basic scroll tool with three parameters and no output schema, the description is adequate but has gaps: it does not explain the 'stealth' parameter or boundary behavior. Sibling tools like drag or click might be alternatives, but no comparison is given.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, so the description should document parameters. It only partially explains dx and dy (pixels) but omits the 'stealth' parameter entirely. Units are implied but no details on sign conventions or limits.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states that the tool scrolls the page by dy and optional dx pixels, with eased human steps. This distinguishes it from siblings like navigate (page navigation) or drag (scrolling by dragging). However, it does not explicitly mention that it scrolls within the current viewport.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use scroll versus alternatives like drag or clicking on a scrollbar. The description does not mention any prerequisites or context for effective use.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

statusA

Return current MCP server status, driver in use (extension or os), whether the browser bridge is connected, and the active tab URL if available. Use for health checks in long-running tests, CI workflows, and agent monitoring.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

A4.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations present, so description carries full burden. It describes what is returned but does not explicitly state it is read-only or non-destructive. Behavioral traits are implied but not detailed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences: first defines output, second gives usage context. No fluff, every word earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with no parameters and no output schema, the description covers return fields and use cases thoroughly. Minor gap: exact data types not specified, but sufficient for the task.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

No parameters exist, so baseline is 3. Description adds context about return values but not strictly about parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description explicitly states it returns MCP server status, driver, bridge connection, and active tab URL. It clearly distinguishes from sibling tools like click or navigate which are action-oriented.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides explicit use cases: health checks, long-running tests, CI workflows, agent monitoring. No ambiguity about when to use this tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

typeB

Type text with human key timing. If a ref is given, the input is human-clicked to focus first. stealth:true uses the debugger driver.

ParametersJSON Schema
NameRequiredDescriptionDefault
refNo
textYes
stealthNo

TDQS

B3.1/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries full burden. It discloses human key timing and stealth driver usage, but omits details like exact timing behavior, error handling, return values, or side effects, leaving significant gaps.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences with no extra words. Main purpose is front-loaded, and all key points are covered without redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 3-param tool with no annotations, no output schema, and no param descriptions in schema, the description is too brief. Missing details on input constraints, stealth behavior nuances, error conditions, and return values make it incomplete for reliable use.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, so description must explain parameters. It explains ref (focus/click first), text (implied as content to type), and stealth (debugger driver). This adds meaning beyond the bare schema, though more detail would help.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states 'Type text with human key timing,' clearly indicating the verb (type) and resource (text). It distinguishes from sibling tools like press_key and click by mentioning human-like timing and focus behavior.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No explicit guidance on when to use this tool vs. alternatives like press_key or click_text. The note about ref and stealth provides context but no comparative direction.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

wait_forA

Wait until an element [ref] appears or some visible text is present (or specific condition), up to timeoutMs (default 10000). Supports condition: 'exists' | 'visible' | 'text'. Use in testing and automation flows for resilience on dynamic sites.

ParametersJSON Schema
NameRequiredDescriptionDefault
refNo
textNo
conditionNo
timeoutMsNo

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description must cover behavioral traits. It discloses supported conditions and default timeout, but does not explain behavior on timeout (e.g., error or return value), whether the function blocks, or its return type. This leaves some uncertainty for an AI agent.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences, front-loading the purpose and conditions. Every sentence adds value without redundancy. It is appropriately sized for the tool's complexity.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with 4 parameters, no output schema, and no annotations, the description is somewhat complete: it covers the main conditions and use case. However, missing details on return value, error handling, and parameter interactions leave gaps for accurate usage.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, and the description explains 'ref' (element), 'condition', and 'timeoutMs' but not 'text' explicitly. It does not clarify the relationship between 'ref' and 'text' (mutually exclusive or combinable), nor what happens if both are provided. This ambiguity hinders correct invocation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool waits for an element or text condition, with specific verb 'Wait' and resources (element, text). It distinguishes from sibling tools like click, find, etc., by focusing on waiting for conditions rather than performing actions.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description states 'Use in testing and automation flows for resilience on dynamic sites,' providing clear context for when to use it. However, it does not explicitly exclude scenarios or mention alternatives among the listed sibling tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 15 tool updatesv0.2.9
    • First observedclick
    • First observedclick_text
    • First observeddrag
    • First observedfind
    • First observedget_url
    • First observedhover
    • First observedmove_to
    • First observednavigate
    • First observedpress_key
    • First observedread_page
    • First observedscreenshot
    • First observedscroll
    • First observedstatus
    • First observedtype
    • First observedwait_for
Deploy Server
One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

TDQS

A3.7/5.0

Scored across 15 tools

Disambiguation5/5

Each tool has a clearly distinct purpose: clicking methods separate from finding, typing from pressing keys, scrolling from navigation, etc. Even similar tools like click and click_text are differentiated by whether a ref or text is used.

Naming Consistency3/5

Naming conventions are mixed: single verbs (click, drag, hover) coexist with underscore-separated verb_noun (click_text, press_key) and verb_preposition (move_to, wait_for). While all lowercase, the lack of a uniform pattern may cause slight confusion.

Tool Count5/5

15 tools is well-scoped for browser automation, covering navigation, interaction, page reading, waiting, and screenshots without being bloated or sparse.

Completeness4/5

Core browser automation features are present (click, type, navigate, scroll, read page, wait). Minor gaps like tab management, cookies, or back/forward navigation are absent, but the set handles common workflows effectively.

Maintenance

ActivityMaintained
ResponsivenessNo issues

Resources

Related MCP Connectors

Related MCP Servers

  • Browserbeam MCP Server

    nyku
    A
    license
    Not graded
    quality
    D
    maintenance
    Enables real browser automation as tools in Cursor, Claude Desktop, Windsurf, and any MCP-compatible client, allowing AI agents to interact with web pages through natural language.
    8 npm
    MIT

  • humanized-browser

    stoneliving
    F
    license
    Not graded
    quality
    C
    maintenance
    This MCP server enables natural, humanized control of a real Chrome browser from Cursor agents, simulating natural mouse paths, typing cadence, and scroll pauses for click, type, scroll, and navigate actions via a Chrome extension.
    -

  • Perfect

    Gtarafdar
    A
    license
    Not graded
    quality
    B
    maintenance
    Unofficial open-source MCP bridge that gives Cursor agents hands in your real Chrome browser. Enables navigation, clicking, filling, screenshots, and other browser actions with permission modes.
    12 npm
    MIT