autopilot-mcp

A browser-automation MCP server. It hands an LLM a generic set of browser tools — navigate, screenshot, read text, run JS, click, type — that work on any URL, with each site backed by its own persistent Camoufox browser profile. Logins are filled straight from a Bitwarden vault so passwords never enter the model's context, and any sequence of steps that works can be saved as a "playbook" for one-call replay next time.

Tools

Browser (free-roam)

Each registrable domain (eTLD+1) gets its own persistent browser profile under data/profiles/<profile>/. navigate(url) auto-derives the profile from the URL; everything else takes profile explicitly.

For parallel work on the same site, use isolated browser instances — each clones the site's base profile so concurrent sessions don't collide:

Example: spawn_instance(url="https://accounts.google.com/...", clone_from_profile="google.com") lets each Gmail cleanup branch use its own cloned Google session. Always call close_instance(instance_id) when the branch is finished; timed-out instances are also cleaned up automatically.

Credentials (Bitwarden, fill-don't-reveal)

Local file server (uploads)

For sites that ask the user to upload a local file. Two paths:

1. Standard <input type="file"> — use attach_file(profile, selector, path). Works even when the input is hidden inside a custom dropzone widget; target the input itself, not the visible drop area.

2. Pure-JS uploader (no real input element) — use the local CORS file server below. The MCP publishes the file at an unguessable URL on 127.0.0.1; the LLM uses run_js to fetch() it inside the page, wrap the Blob in a File, and dispatch a synthetic drop event (or set it on a hidden input via DataTransfer).

Security envelope: server binds 127.0.0.1 only; tokens are uuid4 hex (122 bits of entropy); one token = one file path (no directory traversal); idle entries reaped on every request. Override the bind via AUTOPILOT_FILE_SERVER_HOST / AUTOPILOT_FILE_SERVER_PORT env vars.

Playbooks

Related MCP server: Playwrightium

Workflow

1. list_playbooks(url_match) — is there already a playbook for this task?

2. run_playbook(name) — if yes, run it. Done.

3. Otherwise: navigate(url) → screenshot / get_text → run_js / click / type_text.

4. On a login page: fill_login(url) — Bitwarden injects creds directly. If the form needs 2FA: get_totp(vault_item) then type_text(profile, code).

5. For SMS 2FA: navigate("https://messages.google.com/web/") and read the code from Google Messages.

6. After the task succeeds, save_playbook(...) so next time is one call.

7. Just signed up somewhere new? upsert_login(url, username, password) stores it and Bitwarden sync pushes to your other devices.

Credentials setup

The MCP unlocks Bitwarden via a master password stashed in the OS keyring (DPAPI on Windows). See docs/bw_setup.md for the one-time setup: install bw, bw login, keyring.set_password, smoke-test.

Subsequent MCP starts call the keyring, bw unlock --raw, and cache the session token in RAM only. Idle-expires after 15 minutes; re-locks on shutdown. Master password never hits disk outside the OS keyring.

Initial browser session setup

Each profile gets one persistent browser profile the first time it's opened. For sites where you want the session pre-established (to handle 2FA challenges / "remember me" outside the MCP flow):

uv run python scripts/manual_login.py <url>

A visible Camoufox window opens at the URL. Log in, complete 2FA, check "remember me", close the window. The profile at data/profiles/<eTLD+1>/ persists across headless MCP invocations.

Environment variables

All optional — defaults are sane for local use.

Credentials are pulled from Bitwarden — there are no per-site username/password environment variables.

Development

uv sync --extra dev
uv run camoufox fetch
uv run ruff check .
uv run pytest
uv run python server.py    # stdio mode