ComputerPilot MCP

Supervised computer use and developer automation over the Model Context Protocol.

ComputerPilot MCP gives MCP-capable AI agents a local runtime for files, terminal and process execution, Git, testing, code intelligence, browser automation, durable jobs, crash recovery, and multi-step workflows. On Windows it also exposes native screenshots, input, and semantic UI Automation.

It is designed for developers who want an agent to do real work on a machine without collapsing everything into one unstructured shell tool. Capabilities are exposed as typed MCP tools, platform-specific features are registered only when supported, long-running work can survive MCP restarts, and ambiguous mutations are never replayed blindly.

Security: this project intentionally has broad local capabilities. Treat access to the MCP endpoint as equivalent to granting a development agent access to the OS account running it. See SECURITY.md.

Why ComputerPilot MCP

• One supervised runtime for coding and computer use. Files, code search, LSP, Git, diagnostics, processes, browser automation, jobs, workflows, recovery, and system inspection share one MCP surface.

• Durable instead of request-bound. Long-running commands can use persistent jobs with idempotency keys, bounded concurrency, cancellation, incremental output, and restart survival.

• Recovery is explicit. Interrupted mutations can become uncertain and require evidence-based reconciliation or operator acknowledgement instead of blind replay.

• Cross-platform core, capability-gated edges. Windows, Linux, and macOS share the portable core; native Windows desktop/UI Automation is only registered on supported hosts.

• Verified remote tunnel bootstrap. Tunnel mode downloads the official OpenAI Secure MCP Tunnel runtime into local state and verifies release metadata, checksums, and binary version before publication.

• Release evidence is reproducible. CI, platform validation, deterministic packaging, checksums, Doctor checks, benchmarks, and release validation live in the repository.

Related MCP server: rvn

Quick Start

The fastest way to try the project without tunnel credentials is loopback-only local HTTP mode.

Windows

git clone https://github.com/dibbed/computerpilot-mcp.git
Set-Location computerpilot-mcp
$env:MCP_START_MODE = "local-http"
.\START_MCP.bat

Linux / macOS

git clone https://github.com/dibbed/computerpilot-mcp.git
cd computerpilot-mcp
MCP_START_MODE=local-http ./start_mcp.sh

The launchers create or reuse .venv, validate Python and dependencies, run cached/full startup checks, and start the supervisor.

• MCP endpoint: http://127.0.0.1:8765/mcp

• Local control panel: http://127.0.0.1:8766/

For Secure Tunnel mode, browser setup, release archives, and platform-specific notes, see Installation.

What You Can Do

See Tooling and workflows for the public tool model and practical examples.

Architecture

flowchart TD
    A[MCP client / AI agent] -->|Local HTTP| B[ComputerPilot MCP]
    A -->|OpenAI Secure MCP Tunnel| T[Managed tunnel runtime]
    T -->|local stdio| B

    B --> D[Typed tool domains]
    D --> F[Filesystem / project / LSP]
    D --> P[Terminal / process / system]
    D --> G[Git / testing]
    D --> R[Jobs / recovery / workflows]
    D --> U[Browser / desktop]

    B --> S[Supervisor + lifecycle]
    S --> H[Health / control panel]
    S --> L[.agent_state]
    R --> L

Key runtime invariants:

• local HTTP and the control panel bind to loopback by default;

• filesystem mutations use scoped locks and guarded edit primitives;

• durable jobs keep persistent state and disk-backed output;

• uncertain side effects are not automatically replayed;

• Windows process ownership uses Job Objects, while POSIX uses sessions/process groups;

• browser sessions use isolated contexts with bounded pool/session lifetimes;

• generated runtime state lives under .agent_state/ and is excluded from Git.

A deeper component map is in Architecture.

Platform Support

The release workflow builds five target archives from the same tracked source. Platform claims and limitations are documented in Platform support.

Practical Agent Workflows

Inspect → change → verify

code_context
    ↓
apply_patch / rename_symbol / replace_exact
    ↓
affected_tests
    ↓
verify_changes

Run a long task without tying it to one MCP request

submit_job
    ↓
job_wait
    ↓
job_output
    ↓
cancel_job (when needed)

Recover an interrupted mutation

list_uncertain_operations
    ↓
inspect_uncertain_operation
    ↓
reconcile_operation
    ↓
acknowledge_uncertain_operation (only when evidence cannot resolve it)

Durable multi-step automation

workflow_plan
    ↓
workflow_start
    ↓
workflow_execute
    ↓
workflow_status / workflow_operations
    ↓
workflow_reconcile when a side effect is uncertain

Built-in workflows include implement_and_verify, safe_git_commit, and prepare_release. They do not push, tag, or deploy automatically.

Tool Profiles

The default full profile preserves the complete catalog. Smaller profiles reduce the active domain set for specialized agents:

minimal, coding, git, testing, desktop, browser, operations, full.

discover_tool_domains reports the current profile and platform capabilities. recommend_tools ranks tools that are actually registered in the active profile.

Set a profile with:

MCP_TOOL_PROFILE=coding

Secure Tunnel Mode

Tunnel mode is the default launcher mode. It requires a control-plane API key. A profile can be supplied explicitly or resolved by the runtime configuration.

Recommended secret file:

.secrets/control_plane_api_key.txt

Or set the key in the environment:

$env:CONTROL_PLANE_API_KEY = "<your-key>"
.\START_MCP.bat

The project does not vendor tunnel-client or Cloudflared executables in Git or release archives. The managed updater downloads the matching official openai/tunnel-client runtime, validates the GitHub asset digest and upstream SHA256SUMS.txt, verifies the reported version, and publishes the selected runtime under .agent_state/tunnel-runtime/.

See Binary provenance.

Browser Automation

Browser tools are optional.

python -m pip install -r requirements-browser.txt
python -m playwright install chromium
python -m scripts.doctor --mode local-http --browser

Playwright sessions use isolated contexts. Compatible sessions can share a browser process, and idle sessions/pools are bounded by runtime budgets.

Local State

Generated machine-local state is kept out of Git.

Never commit .agent_state/, .secrets/, .venv/, logs, local databases, or generated output.

Validation and Quality Gates

The repository's normal CI matrix runs on Windows, Linux, and macOS with Python 3.10 and 3.12. Each job installs development dependencies and runs:

• compileall;

• Ruff;

• mypy;

• MCP startup check;

• MCP health smoke;

• full pytest.

Release preparation also has:

• Platform Release Validation on Windows, Linux, and macOS with Python 3.12;

• Release Packaging Smoke that builds all five archives and validates SHA256SUMS.txt;

• exact-SHA publication gates for release commits.

Release-specific evidence is stored under docs/V*-VALIDATION.md. For v0.3.0, see release validation.

Run the common checks locally:

python -m pytest
python -m ruff check .
python -m mypy core tools scripts tests
python -m scripts.health_check --json

See Contributing for the full development workflow.

Release Artifacts

Published releases contain deterministic source/runtime-controller archives for:

• Windows amd64

• Linux amd64

• Linux arm64

• macOS amd64

• macOS arm64

• SHA256SUMS.txt

Each archive includes RELEASE-MANIFEST.json. Packaging is built from tracked Git blobs and fails if forbidden runtime, secret, virtualenv, test-cache, or generated-output paths are tracked.

Browse the latest release.

Documentation

Historical release notes and validation records remain under docs/. Internal local planning directories are ignored by Git and are not part of the public documentation surface.

Known Limits

• Native desktop screenshot/input and semantic UI Automation are Windows-only.

• Linux and macOS support is still labeled preview.

• Linux arm64 and macOS amd64 are package targets without a dedicated native hosted-runner claim in v0.3.0.

• Browser automation requires Playwright and installed browser runtimes.

• LSP tools require a trusted pyright-langserver --stdio command already available on PATH; the MCP does not install it implicitly.

• The local HTTP endpoint and control panel are intended for loopback use, not direct exposure to untrusted networks.

• Built-in workflows deliberately avoid blind push/tag/deploy behavior.

Contributing

Contributions should be made on focused branches, validated locally, and merged through pull requests with CI passing.

Read CONTRIBUTING.md before changing platform, recovery, release, or tunnel behavior.

License

Apache License 2.0. See LICENSE, NOTICE, and THIRD_PARTY_NOTICES.md.

Author: Ali Khalili