orchords-web-pilot
OfficialClick on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@orchords-web-pilot Navigate to https://example.com, snapshot the page, and capture a screenshot."
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
Orchords Web Pilot
⭐ If you like Orchords Web Pilot or find it useful, please consider starring this repository. It helps more people discover the project.
Interested in sponsoring ORCHORDS? Sponsorships start at US$1,000. Depending on the sponsorship level, sponsors may receive public recognition, logo and website placement, sponsor updates and early previews, roadmap-feedback briefings, priority issue triage, and engineering or integration discussions. Sponsorship does not buy control of the roadmap or guarantee feature implementation. Contact crm@orchords.com.
Independent software studio founded in 2025.
A Model Context Protocol server that gives coding agents a real browser — navigate, observe, interact, and capture proof on any web page through one transport-agnostic surface.
Orchords Web Pilot is a Model Context Protocol server that exposes a Playwright-backed browser session as a set of agent-friendly tools. Point your agent at it and it can navigate, observe, interact, and capture proof on any web page — same model across local and hosted browsers, same surface across stdio and Streamable HTTP transports.
Start here
If you need… | Start with |
Install and first run | |
Wire it into a desktop agent | |
Run it as a hosted service | |
Use a remote browser grid | |
Available tools and parameters | |
Security guidance | |
Contributing process |
Related MCP server: browser-devtools-mcp
Capabilities
Real Chromium session — local via Playwright or remote via any WebSocket endpoint (Browserless, hosted Chrome, your own grid)
Small, well-typed tool surface — navigate, snapshot, click, type, screenshot, drag, press-key, hover, evaluate JS, capture console + network, fill forms, manage tabs
One transport for everything — stdio for desktop agents, Streamable HTTP for hosted and multi-user setups
Auto-detection — set
BROWSER_WS_ENDPOINTto flip to a remote browser; everything else stays the sameAccessibility-first — every interaction accepts an element target from the agent's snapshot, not raw pixel coordinates
Install
npm install -g orchords-web-pilotOr run it straight from a checkout:
git clone https://github.com/ORCHORDS/OrchordsBrowserPilot.git
cd OrchordsBrowserPilot
npm install
npm run build
npm startCompatibility
Node.js:
>=20.10.Playwright:
>=1.59.1 <2.0.0. Web Pilot uses the AI-mode accessibility snapshot APIs introduced in the Playwright 1.59 line; startup fails with an actionable error if package-manager overrides force an older or unsupported major version.Browser binaries: Playwright browser revisions are coupled to the installed Playwright package. After changing Playwright versions, install the matching browser with
npx playwright install chromium(ornpx playwright install --with-deps chromiumon a fresh Linux runner).Verification policy: every
mainpush exercises the declared minimum Playwright 1.59.1 with a real browser and also installs the packed npm artifact into a clean fixture. The scheduled Daily Build deliberately overrides the lockfile with the current stableplaywright@latestto detect upstream compatibility drift before the supported range is changed.
Web Pilot does not claim support for arbitrary older Playwright installations just because npm can be forced to resolve them. The declared dependency range, runtime guard, package smoke test, and CI compatibility lanes are the support contract.
Usage
stdio (default — desktop agents)
Add to your MCP client config (Claude Desktop, ZCode, etc.):
{
"mcpServers": {
"orchords-web-pilot": {
"command": "orchords-web-pilot",
"env": {
"PILOT_HEADLESS": "true"
}
}
}
}Streamable HTTP (hosted / shared)
PILOT_TRANSPORT=http \
PILOT_HTTP_HOST=0.0.0.0 \
PILOT_HTTP_PORT=8788 \
PILOT_HTTP_PATH=/mcp \
npm startThen point your client at http://<host>:8788/mcp.
Local or remote browser
By default Web Pilot launches a local Chromium via Playwright. To use a remote browser (Browserless, hosted Chrome, your own grid), set:
export BROWSER_WS_ENDPOINT="wss://chrome.browserless.io?token=..."The server auto-detects: if BROWSER_WS_ENDPOINT is set it connects to that;
otherwise it spins up a local browser.
Tools
Tool | What it does |
| Open a URL. |
| Return an accessibility tree (preferred for agent planning). |
| Click by |
| Type into the focused element. |
| Set an input value directly. |
| Press a key (Enter, Tab, Escape, arrow keys…). |
| Hover an element. |
| Drag from one element to another. |
| Choose an |
| Capture PNG (returns base64 or saves to disk). |
| Run a JS expression in the page context. |
| Read console messages captured for this session. |
| List network requests captured for this session. |
| Wait for text, selector, or a fixed duration. |
| Plug into an external captcha-solving service. |
Snapshot refs
browser_snapshot returns an accessibility tree in which every interactive node carries a ref token, e.g.:
- generic [active] [ref=e1]:
- heading "Settings" [level=1] [ref=e2]
- button "Save" [ref=e3]
- textbox "Email" [ref=e4]Pass the token back into any interaction tool: browser_click({ "ref": "e3" }). Refs are resolved to scoped Playwright locators (role + accessible name + occurrence index), so they work even when CSS selectors would be brittle.
Ref lifetime. Refs are valid from the moment browser_snapshot returns until the next navigation (or another snapshot replaces them). After a browser_navigate, all previous refs are invalidated. Using an invalidated ref returns a structured error (Ref 'eN' is no longer valid…) rather than clicking the wrong element — take a fresh browser_snapshot and retry.
Fallback. Every interaction tool also accepts a raw CSS selector (and browser_click accepts x/y coordinates). Prefer refs for agent flows; use selectors only when you already know the DOM.
Sessions. Each MCP session owns its own page, console buffer, and network buffer. On the HTTP transport, sessions are keyed by the Mcp-Session-Id header; on stdio there is a single session for the process lifetime. Console and network diagnostics never leak across sessions.
The captcha-solver tool is a hook: it reads
PILOT_CAPTCHA_SOLVER_URLandPILOT_CAPTCHA_SOLVER_TOKENand forwards the challenge. Wire in your own provider (2Captcha, AntiCaptcha, your own microservice) — the MVP does not ship a default to keep the licensing surface clean.
HTTP transport security
The HTTP endpoint is hardened by default (#43):
Origin validation — any request carrying an
Originheader must match the allowlist (loopback variants by default) or it is rejected with403. This blocks malicious websites from driving your local browser viafetchagainstlocalhost:8788, including sandboxednullorigins.Host validation —
Hostmust match the bound host or the allowlist, defeating DNS-rebinding attacks.Public-bind refusal — binding to a non-loopback address (including
0.0.0.0) fails at startup unlessPILOT_HTTP_ALLOW_PUBLIC_BIND=trueis set. There is no authentication yet (#42); expose the server only through an authenticating reverse proxy.Rate limiting — fixed-window per client IP (default 120/min) with
429+Retry-After.Body cap + timeouts — requests above the body cap get
413; each request has a hard timeout (504).Security headers —
X-Content-Type-Options: nosniff,Cache-Control: no-store,X-Frame-Options: DENY.
Configuration
All config is via environment variables. See .env.example.
Variable | Default | Notes |
|
|
|
|
| HTTP transport only. |
|
| HTTP transport only. |
|
| HTTP transport only. |
| loopback | Comma-separated Origin allowlist. |
| bind host | Comma-separated Host allowlist. |
|
| Requests/minute per client IP. |
|
| Max JSON body size. |
|
| Hard per-request timeout. |
|
| Honor |
|
| Required to bind beyond loopback. |
|
| Set |
| (unset) | Set to use a remote browser. |
| (unset) | Captcha solver endpoint. |
| (unset) | Captcha solver bearer token. |
Development
npm install
npm run dev # tsx watch
npm run lint
npm testSecurity
Web Pilot is opt-in remote-capable. The HTTP transport binds to 127.0.0.1 by default; change to 0.0.0.0 only behind auth. Never expose it on the public internet without putting a reverse proxy + auth in front.
See SECURITY.md.
Documentation boundary
Product-specific engineering guidance lives in this repository. Company-wide public engineering, security, governance, and operational documentation is maintained in ORCHORDS/docs.
Contributing
Issues and PRs welcome. See CONTRIBUTING.md.
Brand
ORCHORDS — BUILD DIFFERENT.
License
Licensed under the Apache License 2.0.
Available Tools
15 toolsbrowser_captcha_solveB
Forward a captcha challenge to the configured solver (PILOT_CAPTCHA_SOLVER_URL).
Inputs: pageUrl (required), siteKey (required), type.
browser_clickB
Click an element identified by a snapshot ref, CSS selector, or x+y coordinates. Prefer ref from browser_snapshot.
Inputs: x, y, ref, selector.
browser_consoleB
Read console messages captured for the current session.
Inputs: level, limit.
browser_dragC
Drag from one element to another. Accepts snapshot refs or selectors.
Inputs: toRef, fromRef, toSelector, fromSelector.
browser_evaluateB
Run a JS expression in the page context and return its JSON-serializable result.
Inputs: expression (required).
browser_fillC
Replace an input value via Playwright's fill() — accepts a selector or a snapshot ref.
Inputs: value (required), ref, selector.
browser_hoverB
Hover an element. Accepts a snapshot ref or a selector.
Inputs: ref, selector.
browser_networkB
List network requests captured for the current session.
Inputs: limit, static.
browser_pressA
Press a non-text key (Enter, Tab, Escape, arrows, etc.).
Inputs: key (required).
browser_screenshotB
Capture a PNG screenshot. Returns base64 or saves to disk if savePath is set.
Inputs: element, fullPage, savePath.
browser_selectB
Select an by value or label. Accepts a snapshot ref or a selector.
Inputs: ref, label, value, selector.
browser_snapshotA
Return the AI-oriented accessibility snapshot for the current page. Each node carries a ref token (e.g. [ref=e23]) that can be passed back into browser_click/browser_type/etc. Refs are invalidated on navigation.
Inputs: no named parameters.
browser_typeB
Type text into the focused, matched, or ref-targeted element.
Inputs: text (required), ref, slowly, submit, +1 more.
browser_waitB
Wait for a selector/text or a fixed duration.
Inputs: text, time, selector, textGone.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
TDQS
Scored across 15 tools
Each browser_ tool serves a distinct, well-defined action—navigation, snapshot, click, type, fill, press, hover, drag, select, screenshot, evaluate, wait, console, network, and captcha solving. Even the text-entry tools (type vs. fill) are clearly differentiated by their descriptions, eliminating ambiguity.
All tools consistently use the 'browser_' prefix followed by a descriptive verb (or verb+noun) in lowercase snake_case, such as browser_navigate, browser_screenshot, browser_captcha_solve. This uniform pattern makes the tool names predictable and easy to learn.
With 15 tools, the set is appropriately scoped for a comprehensive browser automation server. Each tool covers a specific aspect of web interaction without being redundant or overwhelming, staying within the ideal range for a focused purpose.
The tool surface covers the full lifecycle of browser interactions: navigation, inspection, element actions, waiting, screenshotting, JavaScript execution, and special handling like captchas. Minor gaps like tab/multi-page management or dialog handling exist, but they are non-essential for most web-pilot tasks.
Maintenance
Related MCP Connectors
Control a real browser from AI agents: navigate, click, type, screenshot, read the page.
Official Browserless MCP server: browser automation, scraping, screenshots and PDFs for AI agents.
Stealth web browser for agents: search, fetch, click, download and type in persistent MCP sessions.
Web scraping, browser automation, screenshots and AI data extraction for AI agents via MCP.
Related MCP Servers
- FlicenseBqualityDmaintenancePlaywright wrapper for MCP that enables LLM-powered clients to control a browser for automation tasks.101-
- AlicenseNot gradedqualityNot gradedmaintenanceA Playwright-based MCP server that exposes a live browser as a traceable, inspectable, debuggable and controllable execution environment for AI agents.5,213 npm57-
- AlicenseAqualityDmaintenanceEnables MCP clients to automate a real Chrome browser via Playwright, supporting session sharing and tools for navigation, clicking, typing, and more.112MIT
- AlicenseCqualityDmaintenanceExposes a remote browser as MCP tools via Playwright, enabling AI agents to navigate and interact with web pages through DOM snapshots, clicks, typing, and form operations.4014 npm14Apache 2.0