Pulse
Maps and drives a Linux desktop: the desktop adapter reads computer-use-linux accessibility snapshots and screenshots, places accessible controls on a normalized grid, clicks mapped controls behind a stale-screen check, and replays verified desktop routes in a single call. Tesseract is used only to place a control whose accessibility bounds are missing.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@Pulse replay the saved login route and verify the dashboard loads"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
Pulse
A screen map and verified-workflow store for computer-use agents.
Pulse records what a page or desktop looks like. It connects screens with transitions it watched succeed. Then it replays a whole route in one call.
Python MCP server. No runtime dependencies.
The problem
An agent driving a UI pays the same cost on every step. Observe, decide, click, observe again to confirm. A route it solved yesterday is solved from scratch today.
Pulse keeps the map between runs. The learned route replays as one call, and the replay still verifies every state it crosses.
flowchart LR
O[observe] --> M[("map<br/>screens + anchors")]
M --> T[("transitions<br/>verified edges")]
T --> R[shortest route]
R --> P[replay]
P -->|new screens| MRelated MCP server: dsh-computer-use
With Pulse and without
flowchart TB
subgraph W["Without Pulse: every step re-observes"]
direction LR
W1[observe] --> W2[click] --> W3[observe] --> W4[click] --> W5[observe] --> W6[click] --> W7[observe]
end
subgraph P["With Pulse: one verified route"]
direction LR
P1["replay call"] --> P2{"verify each state"} --> P3["final state returned"]
end
W ~~~ PThe saving tracks how much observation a route removes. Six guarded clicks become one call. A single click saves one screenshot digest and nothing else.
Measured results
Every number below comes from docs/benchmark.md, on local fixtures and public
sites, on 2026-10-24.
One call against many
Workflow | Baseline | Replay | Saved | Evidence |
6 clicks, in process | 4,328 ms | 1,177 ms | 72.8% | 3 trials each |
3 clicks, in process | 309.0 ms | 95.9 ms | 69.0% | 7 trials each |
2 desktop clicks | 3,364.9 ms | 1,047.7 ms | 68.9% | 7 trials each |
4 transitions, model in loop | 42.76 s | 17.39 s | 59.3% | 3 paired trials |
Whole workflows, live sites
Workflow | Calls | Baseline | Replay | Change |
Add two elements | 2 → 1 | 751 ms | 177 ms | −76.4% |
Toggle a checkbox | 1 → 1 | 440 ms | 231 ms | −47.4% |
Scroll a long page | 1 → 1 | 364 ms | 275 ms | −24.4% |
Complete a todo | 3 → 3 | 903 ms | 773 ms | −14.5% |
Nine workflows through each harness
Harness | Baseline p50 | Replay p50 | Speed increase | Runs |
Claude Code | 17.83 s | 12.49 s | 42.8% | 35/35 correct |
Codex | 34.55 s | 23.00 s | 50.2% | 36/36 correct |
The six-click workflow alone speeds up 278.2% through Claude Code and 276.2% through Codex. Both harnesses hit that target on sequential work. Neither reaches it on random or single-step requests.
The same runs inside agent desktops
Each harness ran in its own pane, on its own agent desktop, with its own Chromium profile, CDP port and Pulse state per trial. The host focus never moved.
Workflow | Harness | Baseline p50 | Replay p50 | Speed increase | Runs |
Six clicks | Claude Code | 49.97 s | 17.38 s | 187.5% | 4/4 correct |
Six clicks | Codex | 85.03 s | 23.35 s | 264.2% | 4/4 correct |
Two categories | Claude Code | 19.73 s | 14.57 s | 35.3% | 4/4 correct |
Two categories | Codex | 36.45 s | 26.15 s | 39.4% | 4/4 correct |
Nine workflows ran in the same conditions. Nine workflows moved the overall p50
from 28.94 s to 18.64 s through Claude Code, a 55.3% speed increase. Nine
workflows moved 39.40 s to 26.43 s through Codex, 49.1%. Nine workflows include
six clicks at 354.1% and 256.4%, two Claude regressions, and two Codex refusals
that stopped on a stale start screen rather than acting. Every trial recorded
deepseek/deepseek-v4.1-flash-ultrafast in its own row. The per-workflow table
is in docs/benchmark.md.
xychart-beta
title "p50 wall time, nine live workflows"
x-axis ["baseline", "replay"]
y-axis "seconds" 0 --> 40
bar [17.83, 12.49]Good cases
Pulse pays for itself when a route removes observation.
Case | Why it gains | Measured |
Many sequential steps | one guarded step leaves per transition | 6 clicks: 72.8% saved in process, 278.2% faster in Claude |
A route you run again | learning cost spreads over every rerun | nine workflows: 42.8% and 50.2% |
Deterministic DOM | screen ids match across runs | most workflows 2/2 or 3/3 |
Desktop with a settled tree | accessibility map replaces screenshots | 68.9% saved over 2 clicks |
Base cases
Pulse gains little or nothing here. Each limit has a reason.
Case | Why it does not gain | Measured |
A single step | both paths observe twice, so a route saves one digest | scroll: 25.0% slower in Claude |
A short workflow | the CLI launch floor, 5.7 s Claude and 11.4 s Codex, dominates | two clicks: 23.3% faster |
A page typed value reappears on | the page renders the value as mapped text | TodoMVC |
Lazy-loading content | settled height differs per run | replay stopped 1/3, then 3/3 after a settle wait |
Action reported only in a | no mapped anchor changes, so no edge forms | demoqa buttons |
Replay pays one settle wait per step. Replay pays 631 ms for the call wrapping a 4.6 ms read. Replay needs that wait to record an async step correctly, so the cost is a correctness decision rather than a tuning knob.
Install
python3 -m pip install -e .
pulse-mcpThe default data directory is ~/dev/pulse-data when ~/dev exists. Otherwise
it is $XDG_STATE_HOME/pulse, or ~/.local/state/pulse. Set PULSE_STATE_DIR
to choose another directory.
Connect
Pulse speaks MCP over stdio. pulse-mcp reads JSON-RPC on stdin and writes it
on stdout, so each client launches its own child process. Pulse opens no port
and runs nothing between sessions.
claude mcp add --scope user pulse -- pulse-mcp# ~/.codex/config.toml
[mcp_servers.pulse]
command = "pulse-mcp"Restart the harness after adding the server. Both harnesses expose the same tools.
Tools
Tool | Purpose |
| Save an observation, return its grid |
| Save an edge, verified or not |
| Read a map, or one screen with its grid |
| Shortest route over verified edges |
| Save and read routine candidates |
| Map visible controls in a local Chromium tab |
| List tabs on one CDP port, and which a call picks |
| Click a mapped control after a stale-screen check |
| Type into one mapped field, guarded, no implicit submit |
| Send |
| Scroll a bounded delta and re-map |
| Run a verified route in one call |
| Map a Linux desktop from accessibility and screenshots |
| Click a mapped desktop control, guarded |
| Run a verified desktop route |
Anchor bounds are normalized. x, y, width and height each use the
viewport as a 0–1 scale, and an anchor sits in the grid cell holding its center.
An edge becomes route-eligible only after it is seen to change screen state.
Start a browser
chromium --headless=new --no-sandbox --remote-debugging-port=9222 \
--user-data-dir=/tmp/pulse-chromium about:blankThe default port is 9222. Tools can pass another local port, or set
PULSE_CDP_PORT. Connect only a browser profile you intend an agent to control.
A call without target_url takes the newest page target, which may not be the
page under test. A call with target_url stays on the tab it names. A call
should follow pulse_browser_targets when one port serves several tabs, because
that tool reports what the port holds and which tab a call would pick. A call
that captures a stalled background tab reports no digest, and the observation
still returns its anchors.
What a map holds
Holds | Never holds |
roles, labels, selected state | form values |
normalized bounds, viewport size | screenshot bytes |
screenshot digest | URL query or fragment |
Map files are written atomically with owner-only permissions. A fill keeps its value out of every map, tool result and saved routine. A fill reports only the length it typed. A fill refuses password, file, hidden, checkbox, radio, disabled and read-only fields. A fill confirms by reading the field back, and sends no key, so it cannot submit.
Two rules govern focus and line endings:
A fill drops focus once it confirms, because a focused field's text reaches the next observation's anchor labels. Pass
keep_focusbeforepulse_browser_key.A single-line field loses a trailing newline, so the entry is trimmed and confirmed against the trimmed text.
A scroll position takes part in the screen id. Two positions in one long page therefore have separate ids, and a stale-screen check cannot pass wrongly.
The fill guard has one leak. A value reaches the map when the page displays it. A value arrives as an anchor label like any other text. A value cannot be redacted without dropping real page content. A value never belongs in a secret field the page shows, and TodoMVC shows exactly that: a typed title becomes the row's checkbox label.
Desktop
The desktop adapter reads computer-use-linux accessibility snapshots and
screenshots. The desktop adapter places accessible elements on the same
normalized grid. The desktop adapter needs computer-use-linux mcp on PATH,
and PULSE_COMPUTER_USE_BIN picks a specific binary. The desktop adapter calls
Tesseract only to place a control whose accessibility bounds are missing.
Browser routes and desktop routes share one planner.
Limits
Pulse does not execute saved routines in this version. Routine steps reject
literal values, so a step carries an input_ref and a future driver supplies
the value.
A paragraph is not mapped. A page that reports an action only in a <p> changes
no screen id, so the map looks unchanged and no route learns.
Development
PYTHONPATH=src python3 -m unittest discover -s tests -v
python3 -m compileall -q src testsResults, the four defects live sites exposed, and the scope of each number are in docs/benchmark.md.
Available Tools
17 toolspulse_browser_clickB
Click a mapped browser control only if the current screen still matches the expected map state. Write and irreversible controls require explicit authorization.
Inputs: anchor_id (required), expected_screen (required), port, app_id, +2 more.
pulse_browser_fillA
Type a value into one mapped browser field. The value applies to this call only, is never stored or returned, and never submits the form. Requires authorized_effects:["write"]. Password, file, hidden, checkbox, radio, disabled and read-only fields are refused. Set keep_focus when a fill is followed by pulse_browser_key, because a fill drops focus by default to keep the value out of the map.
Inputs: value (required), anchor_id (required), expected_screen (required), authorized_effects (required), +4 more.
pulse_browser_keyB
Send one allowlisted key, enter or escape, to the focused element on the expected screen. Enter submits the form that holds the focused field. Requires authorized_effects:["write"].
Inputs: key (required), expected_screen (required), authorized_effects (required), port, +2 more.
pulse_browser_observeB
Observe the active local Chromium page, map visible DOM controls into the spatial grid and store the screen.
Inputs: port, app_id, target_url.
pulse_browser_replayB
Execute a previously verified route in one browser call. Each step checks its expected screen and stops on divergence. A completed route returns the final screen_id, title, viewport, anchor_count, anchors and screenshot_sha256, so no confirmation observe is needed. Write and irreversible effects require explicit authorization.
Inputs: start (required), target (required), port, app_id, +2 more.
pulse_browser_scrollC
Scroll the current browser viewport by a bounded pixel delta and map the resulting screen. The scroll acts only on the viewport of the expected mapped screen.
Inputs: expected_screen (required), dx, dy, port, +2 more.
pulse_browser_targetsA
List the page tabs one CDP port serves and report which one a call would choose. Pass target_url on other browser calls to pin the page under test when a port holds more than one tab.
Inputs: port, target_url.
pulse_desktop_clickA
Click an anchored desktop control only when the live desktop still matches the expected mapped screen. Write effects require explicit authorization.
Inputs: anchor_id (required), expected_screen (required), app_id, application, +1 more.
pulse_desktop_observeC
Capture the active Linux desktop through computer-use-linux, map accessibility controls to a normalized screen grid, and use OCR for anchors when accessibility bounds are missing.
Inputs: app_id, application.
pulse_desktop_replayB
Execute a verified desktop route in one call. Each step checks the expected mapped screen and stops on divergence. Write effects require explicit authorization.
Inputs: start (required), target (required), app_id, application, +1 more.
pulse_map_getB
Read a stored screen map, optionally including one screen's full grid.
Inputs: app_id (required), screen_id.
pulse_map_record_screenC
Record a screen observation with normalized anchors and return its grid.
Inputs: url (required), title (required), app_id (required), anchors (required), +4 more.
pulse_map_record_transitionC
Record a transition. Only verified transitions are used by route planning.
Inputs: app_id (required), source (required), target (required), verified (required), +1 more.
pulse_map_routeC
Find the shortest verified route from a mapped screen to a screen or anchor label.
Inputs: start (required), app_id (required), target (required).
pulse_routine_getB
Read a saved routine candidate by id.
Inputs: routine_id (required).
pulse_routine_listB
List saved routine candidates, optionally filtered by application.
Inputs: app_id.
pulse_routine_saveC
Save a successfully verified routine as a reviewable candidate; this does not execute it.
Inputs: id (required), name (required), steps (required), app_id (required), +2 more.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
TDQS
Scored across 17 tools
The four domain prefixes (map, browser, desktop, routine) plus action suffixes make most tools clearly separable: browser_click vs desktop_click vs map_get vs map_route are all obviously different operations. The main soft spots are pulse_browser_observe vs pulse_map_record_screen (both capture a screen into the grid, differing mainly in source and side effects) and pulse_browser_replay vs pulse_map_route, which an agent could confuse as plan-vs-execute.
Every tool follows a pulse_<domain>_<action> pattern with a stable set of domain prefixes, which makes grouping predictable. The only deviation is argument ordering inside the name (map_get/record_screen are noun-then-verb while browser_observe/desktop_click are verb-only), but this is minor and readable.
17 tools is slightly above the comfortable range but justified by the four distinct sub-surfaces (mapping, browser, desktop, routines), each of which earns two to seven tools. Nothing appears gratuitous, though the map and routine groups are thin enough that some consolidation might be possible.
The browser surface is rich (observe, targets, click, fill, key, scroll, replay) but the desktop surface only offers observe/click/replay with no direct fill, key, or scroll primitive, and map/routine groups lack listing of screens and any delete/update for saved routines or transitions. These gaps force agents to work around missing operations or rely entirely on replay for desktop input.
Maintenance
Related MCP Connectors
Run multi-step tasks in a real Chrome browser: persistent environments, live view, human takeover.
Connect your agents to any website. The cheaper, faster and more reliable alternative to browser use
Discover, compare, route, and execute machine-accessible capabilities for AI agents.
Let your AI agent operate any website through a real, logged-in browser session.
Related MCP Servers
- AlicenseNot gradedqualityCmaintenanceEnables governed browser automation for AI agents with human-in-the-loop approval, audit logging, and read-only replay, supporting both ASK and AUTO autonomy modes.MIT
- AlicenseCqualityCmaintenanceEnables agent-native desktop control by driving mouse, keyboard, and interface elements through accessibility-first semantic operations, with fallback to screen coordinates when accessibility cannot reach a target.30MIT
- AlicenseNot gradedqualityFmaintenanceEnables AI agents to record desktop workflows from human demonstrations, save them as validated semantic skills, and replay them in an isolated Linux sandbox with deterministic verification and evidence-driven self-healing.1MIT
- AlicenseAqualityBmaintenanceEnables an agent to observe and operate desktop apps and browsers through an accessibility-tree-first view, with clicking, typing, scrolling, dragging and waiting tools. Screenshot handling, coordinate mapping and fail-closed safety rules (risk confirmation, deny-listed apps, stale-observation refusal) are enforced inside the server.9Apache 2.0