Agent Computer

Muse-style computer use, inside ChatGPT.

Want the computer-use side of Muse or dots in your existing ChatGPT?

Agent Computer is an open-source project that connects ChatGPT to a self-hosted Linux desktop. It gives ChatGPT a browser, terminal, and workspace files for concrete work: research a topic, run a script, or build and check a small web page. Watch the work, click or type directly on the same desktop, and download the result. ChatGPT drives the tasks; this project provides the computer.

File editing, shell commands, and code execution are powered by Coding Tools MCP, also built by xyTom. One MCP connection brings those tools and the browser into the same workspace.

Early preview · One trusted owner · Apache-2.0

GitHub Actions preview / 限时体验 · English quickstart · 简体中文上手 · Deployment · Contributing · Roadmap · Changelog

Demo videos and v0.1.0 release. The demo is a scripted MCP sequence with sample data. The setup video covers optional Docker self-hosting.

What can you use it for?

• Research with a saved result. Visit official sources, compare options, and save a short report with links.

• Work with files. Read workspace files, transform sample data with a script, and save the output for download.

• Build something small. Write an HTML page or a personal tool, open it in Chromium, and check how it looks.

• Improve the result. Ask for a change, check the revised file in the same environment, and download it when it is ready.

These are task ideas, not success-rate benchmarks. Start with public information and sample files.

Related MCP server: browser-use-relay-mcp

How it works

ChatGPT → OAuth + MCP → Agent Computer
                       ├─ Linux desktop + Chromium
                       ├─ Coding Tools MCP → files, terminal, code
                       └─ Web viewer → watch + use the desktop, download files

You run the computer. ChatGPT supplies the model, plans the task, and calls its tools. The server provides the desktop and execution environment; it has no background model loop or scheduler. When the client stops calling tools, the server does not continue reasoning on its own.

Browser automation and desktop control use the same visible Chromium session. With Docker self-hosting, files live in /data/workspace on a persistent volume. GitHub Actions preview files are temporary. Other MCP clients can connect if they support Streamable HTTP and the required OAuth flow; account-specific compatibility still needs testing.

Get started

Check your client before deploying. Computer use needs custom MCP write tools. OpenAI currently lists full MCP write access for Business, Enterprise and Edu; Pro access is read/fetch-only. Account/workspace permissions vary. See OpenAI's current requirements (checked October 5, 2026).

Try it with GitHub Actions

For this first release, start with the temporary development/testing preview (Chinese guide). Fork the repository, enable Actions, set your owner secret, and run Launch MCP preview. The guide covers OAuth setup and a small first task. You do not need a local Docker installation or a Cloudflare account for the Quick Tunnel route.

The default session lasts 60 minutes after readiness, within GitHub's job limits. When it ends or is cancelled, the desktop, files and browser sessions are deleted. Download results first. URLs change between runs; account quotas and GitHub Actions terms apply. Use this for developing and testing Agent Computer.

For a persistent computer on your own host, follow the existing Docker path below.

1. Start locally (optional self-hosting)

You need Git, Python 3.11+, a running Docker installation with Compose, and at least 3 GB available for the container plus host overhead. Linux x86-64 is the validated target; macOS/Windows Docker Desktop and ARM are not certified.

git clone https://github.com/connbot/desktop-bridge.git
cd desktop-bridge
python3 scripts/setup.py
python3 scripts/doctor.py
docker compose up --build -d
python3 scripts/doctor.py --running

The first build downloads Chromium and system packages. If the last check runs before startup finishes, wait and rerun it. Open http://localhost:8080 on the Docker host. Sign in with BRIDGE_OWNER_TOKEN from your local .env file. Keep that token out of chats, source control and logs; it is for owner login and OAuth approval, not a bearer token to paste into the MCP client.

2. Connect ChatGPT over HTTPS

ChatGPT cannot directly reach your laptop's localhost. Follow the HTTPS deployment walkthrough to set BRIDGE_PUBLIC_URL, configure a reverse proxy and check your domain. Keep Docker's port 8080 on loopback. Your direct MCP endpoint is https://YOUR_HOST/mcp.

Create a custom app named Agent Computer, select OAuth and, if asked, dynamic client registration. Scan tools, sign in on your computer's domain, review the callback and approve access. No static client secret is required. Select the app in a regular ChatGPT chat and keep the viewer open beside it.

The step-by-step quickstart covers account setup, local vs. remote addresses, authorization expiry and troubleshooting. Refresh-capable clients can renew one-hour access tokens automatically within a 30-day authorization (7-day refresh inactivity limit); existing connections need new client registration and one new approval (usually by recreating the app connection). Revocation/restarts still end authorization, and this does not extend a temporary Actions run. No model API key is needed for this direct connection. The server does not include a model subscription.

3. Get a result you can check

Start with a small task that needs no external account:

Use Agent Computer to open http://127.0.0.1:8080/static/demo.html in its browser. Fill Project note with “Hello from Agent Computer”, click Save note, and verify the message on the page. Create hello-agent-computer.txt in the workspace with the same sentence and read it back. Do not sign in to any external account.

In the viewer, expand Files & activity, click Refresh under Your files, and download hello-agent-computer.txt, and open it. You should see the browser interaction, the saved file and its verified contents. Then try the build-and-revise tutorial to create a budget calculator and check it in Chromium. The default safe command policy may deny shell operations; see the tutorial before attempting that extra step.

One shared computer

Log in and the desktop opens first. Click or type whenever you need to; there is no Take control / Hand back cycle. Watching alone does not pause the AI. Setup, files/activity, and optional preferences/tasks are collapsed below the desktop.

• Use it directly: owner input waits for the current AI action to finish, then runs between actions. Held keys or mouse buttons block new AI actions until released. The next AI action needs a fresh observation; the client handles this, not an owner mode switch.

• Pause AI: explicitly block new AI actions and attempt to stop tracked managed shell processes. Resume AI releases that pause.

• Go private: also block AI observations and tool access until you explicitly choose Resume AI. You can still use the desktop. Inspect running apps before entering secrets; unmanaged background processes may remain.

• Disconnect & revoke: revoke all owner viewer sessions and OAuth grants.

Ordinary clicking and typing do not cancel running commands. A long current action may delay your input; use Pause AI when you actually want work stopped. Shared input is serialized with bridge actions, not isolated from background apps or shell processes that can keep changing the desktop. Explicit pause/private cannot undo submitted forms, completed purchases or other external effects. Action receipts and fresh-observation checks remain in place; the client still handles user approval. The legacy read-only /desktop/view and explicit control APIs remain for compatible clients.

The screen reconnects automatically after a transport interruption. This is not persistent hosting: Actions runs still expire, temporary tunnel URLs can change, and server restarts still end viewer/OAuth sessions and client registration.

Keep your work

The Docker volume preserves workspace files, the browser profile, and action receipts across container restarts. A restart does not restore running processes or resume a model task.

docker compose stop    # Stop the computer
docker compose start   # Start it again

Download or back up files you want to keep. Avoid docker compose down -v unless you intend to delete the volume and its data.

For temporary development sessions, the repository also includes an on-demand GitHub Actions preview. It is disposable, bounded hosting: download results before it ends. Temporary tunnel URLs change between runs.

Optional memory and tools

The desktop works without these features. Both are off by default:

• Personal context: save selected preferences, goals, task progress, and result references in an explicitly configured local or Postgres-compatible store. The client must read and update them; a saved task does not start itself. Task status is author-reported, and exporting context does not export the linked files.

• External MCP tools: expose an explicit allowlist of tools from owner-configured remote MCP servers. Accounts, credentials, service compatibility, and permission for consequential actions require separate setup. Included examples do not mean a live service is connected.

The default Coding Tools command policy is safe; some shell operations require a deliberate owner policy choice. Do not switch modes just to bypass a denied command. See command permissions.

Configuration changes require a restart. No model API key is needed for the ChatGPT connection above. Optional third-party services may require their own credentials.

Security and current limits

Use this with one trusted owner and low-sensitivity data. The browser, gateway, and shell share one container. This is not a security boundary against malicious code, a hostile agent, or independent tenants.

• Chromium runs without its inner sandbox inside a non-root container. The container shares the host kernel, and network egress is not restricted.

• Shell commands can reach container-local services. Environment scrubbing and private ports do not isolate valuable credentials from arbitrary code in that container.

• The persistent volume includes browser sessions and tool receipts, which may contain sensitive output. Protect it like other private data.

• Use test accounts and minimal permissions. Avoid high-value personal accounts and untrusted code. The server cannot decide whether an action is a purchase or whether the user has approved it.

• Website compatibility, client permissions, task success, and behavior on unverified host platforms vary.

Verification and implementation

The verification record documents Linux x86-64 Docker tests for real browser/desktop control, OAuth, Coding Tools operations, takeover, view-only enforcement, and restart persistence. The owner also reported successful ChatGPT connection and live tool calls on October 4, 2026. These are integration checks and a specific client report, not an autonomous-task benchmark or a guarantee for every account.

Architecture · English / 中文 quickstarts · HTTPS deployment · Optional Fly deployment

Need help or want to improve the project? Start with contributing and support. Report security concerns using the security guide; do not put tokens or private workspace data in public issues.

Credits and license

• Coding Tools MCP by xyTom: files, shell, processes, and code editing through private stdio.

• Cua: desktop screenshots, pointer, and keyboard control.

• Playwright: structured browser actions in the same Chromium session.

• noVNC and x11vnc: the live shared desktop and legacy server-enforced read-only channel.

The project's source is Apache-2.0. Dependencies retain their own licenses; see third-party notices. This is an independent project with no OpenAI, Muse or Cua affiliation. The Muse/dots comparison describes the computer-use experience, not feature parity or endorsement.

Dependencies are pinned where practical, including the reviewed Coding Tools commit identified in pyproject.toml. Base images and Debian packages can change, so rebuilds are not claimed to be byte-for-byte identical.

Share the project

Chinese and English community launch copy, title options, and demo storyboard.

本项目已链接认可 LINUX DO 社区。

Optional web setup wizard (local review build)

A Chinese, beginner-oriented initializer is available in initializer/. GitHub is required; Cloudflare and an existing active domain are optional. Quick mode needs neither, while named mode uses only the user's domain. It includes real provider adapters, a separately labeled offline demo and regression tests. It is not deployed or production-accepted; see the exact setup, security and real-provider/browser validation gates in the initializer guide.