DroidLab MCP

Android emulator under full control of an AI agent. DroidLab is an MCP (Model Context Protocol) server that lets any MCP client — Claude Desktop, OpenCode, Cursor, a custom agent — boot an Android emulator, drive its UI (tap / swipe / type / keys), read the screen (screenshot + UI element tree), install APKs, read logs, and use the clipboard. While the agent works, a human can watch the live screen in a regular browser over the local network.

Agent first. The MCP server is the product; the browser is the observation deck. An agent can set up and use the entire toolset without any human in the loop.

[MCP client / agent] ──stdio──> [MCP server] ──adb──> [Android emulator]
                                     │
[Browser / human] <──WebSocket──> [Node.js bridge] <──┘  (video + input relay)
   H.264 via WebCodecs

Table of contents

• Requirements

• Agent quick start

• Tools

• Resources

• Human quick start

• Architecture

• Security model

• Configuration

• WebSocket protocol

• Latency model

• Troubleshooting

• License

Related MCP server: mobila

Requirements

Linux, macOS and Windows are supported. The primary H.264 path works everywhere; the WebP fallback (browsers without WebCodecs) is Linux-only.

Self-bootstrap: missing pieces are downloaded on demand. env_start creates a missing AVD by itself — it derives the API level from the AVD name (API33 → system-images;android-33;google_apis;<host ABI>, arm64 hosts get arm64-v8a), downloads the image via sdkmanager (pending SDK licenses are auto-accepted, 30-min cap) and runs avdmanager create avd -d pixel_7. If sdkmanager/avdmanager are absent, the official cmdline-tools package is fetched into <sdk>/cmdline-tools/latest; when no system java exists, Android Studio's bundled JBR is wired into JAVA_HOME/PATH. Missing scrcpy is downloaded to ~/bin/scrcpy/ (release v4.1 asset for the platform + scrcpy-server jar) before the bridge starts; SCRCPY/SCRCPY_SERVER env vars override the lookup. Downloads need network access.

Agent quick start

git clone https://github.com/cirkasssian/Droid-Lab-MCP.git droidlab && cd droidlab
bash scripts/install-mcp.sh

The installer is safe by design: it locates node at known absolute paths and never invokes brew upgrade/brew reinstall (a bare brew operation can collateral-upgrade unrelated apps — this is exactly how an improvised install broke opencode on 2026-09-14: brew reinstall node → brew replaced the opencode binary → every prompt failed with "Failed to send prompt"). If node is missing, it installs it with collateral-upgrade guards, smoke-tests the MCP handshake, and registers the server in ~/.config/opencode/opencode.json[c] (idempotent, with a backup).

Manual registration (any MCP client) — always use the absolute node path; GUI clients do not inherit the interactive shell PATH, so a bare "node" silently fails there:

{
  "mcpServers": {
    "droidlab": {
      "command": "/opt/homebrew/bin/node",
      "args": ["/absolute/path/to/droidlab/mcp/server.mcp.mjs"]
    }
  }
}

opencode (opencode.jsonc) uses the "mcp" block format: "command": ["/opt/homebrew/bin/node", "/path/to/mcp/server.mcp.mjs"] plus "env": { "PATH": "/opt/homebrew/bin:/usr/local/bin:/usr/bin:/bin" } as a fallback.

A typical first session:

1. env_start — boots Xvfb (Linux), the emulator (cold boot) and the bridge; blocks until Android is up.

2. screenshot / ui_dump / wait_for — see the screen, locate elements by text or resource-id.

3. tap / swipe / text / key — drive the UI in native pixels (default screen 1080×2400).

4. install_apk / push_file / pull_file / logcat / clipboard_get — install and inspect.

5. access_start — when a human needs to watch: returns a tokenized LAN URL.

6. env_stop — shut everything down when done.

All long operations (boot, image download, APK install, file transfer) support MCP cancellation (notifications/cancelled) and report progress (notifications/progress).

There is no raw adb shell tool by design: an arbitrary command is an injection vector and a bypass of tool annotations. The 33 tools below cover the needed surface.

Tools

33 tools. Status tools (env_status, device_state, env_list, system_images_list) also return structuredContent (MCP 2025-06-18). All tools declare MCP annotations (readOnlyHint / destructiveHint / idempotentHint).

Lifecycle

SDK / AVD management

Device interaction

Network access (for humans)

Resources

Human quick start

Start the stack manually (or just ask the agent: "start the emulator and let me watch"):

# 1. Xvfb (Linux, WebP fallback only)
Xvfb :99 -screen 0 1080x2400x24 &

# 2. Emulator
~/Android/Sdk/emulator/emulator -avd API33 -no-window -no-audio -no-snapshot &

# 3. Bridge (starts scrcpy/ffmpeg on its own)
node web/server.js

Open http://<host>:8090 in a browser. The agent controls the stack over MCP and is the only party that opens network access (access_start) or unlocks browser input (set_dev_input).

Headless machine? Tunnel instead of exposing the port:

ssh -L 8090:localhost:8090 user@headless -N
# then: http://localhost:8090/?token=<accessToken>

Browser controls: click = tap, drag = swipe, wheel = scroll, keyboard = device input (printable text, Backspace, Enter, arrows, Esc), plus Back / Home / Recents / fullscreen buttons. APKs can be dragged into the window (adb install -r -t); other dropped files land in /sdcard/Download/. Ctrl+C / Ctrl+V bridge the host clipboard with the device.

Architecture

Primary video path — raw scrcpy-server, two instances per session:

• a video instance (control=false) for the stream, and a ctrl instance (video=false) for input/clipboard — with video=true the server maps touch coordinates through the video frame and display pixels never arrive;

• the bridge pushes scrcpy-server.jar, connects the sockets via adb reverse localabstract:scrcpy_<scid> and splits the stream into access units [u64 pts_flags][u32 size] (bit 62 = config, bit 61 = keyframe);

• the browser decodes H.264 through WebCodecs VideoDecoder; the stream is content-driven (no frames on a static screen) and has no recording time limit.

Fallbacks: VIDEO_SRC=screenrecord (180 s limit, auto-recycled at 170 s) and a WebP path (scrcpy → Xvfb → ffmpeg x11grab) for browsers without WebCodecs (?codec=webp).

If the emulator dies, the bridge restores the stream automatically once the device is back.

Security model

• Tokens. The bridge requires an access token (WEB_ACCESS_TOKEN, generated per bridge start): HTTP and WS without ?token= get 401. Browser input is gated by a separate WEB_CONTROL_TOKEN, so an observer cannot unlock input by itself. A manual start without WEB_ACCESS_TOKEN runs unauthenticated (trusted LAN only).

• Role separation. The agent works through MCP; the developer watches (and taps only after set_dev_input(true)). While the agent works, browser input is blocked server-side.

• No shell injection surface. No raw adb shell tool; the network surface is one port with token auth.

Configuration

Environment variables (all optional):

End-to-end test: npm run e2e:mcp (boots the stack, exercises the toolset over real MCP stdio).

WebSocket protocol

Client → server, the first message picks the codec:

{"type":"init","codec":"h264"}

h264 is selected automatically when window.VideoDecoder exists; force it with ?codec=h264 or ?codec=webp.

Input commands:

{"type":"tap","x":540,"y":1200}
{"type":"swipe","x1":540,"y1":10,"x2":540,"y2":1440,"ms":300}
{"type":"key","code":4}
{"type":"text","text":"hello"}

Server → client: binary frames [1 byte flag][payload] (bit 0 = keyframe for H.264 AUs, Annex-B; full WebP frames in fallback mode), {"type":"res","name":"486x1080"} on resolution changes, and a 1 Hz ping (RTT probe for ABR).

Latency model

Resolution is managed automatically (ABR) from network throughput; the ladder is 324x720 → 486x1080, 1004x2231 is available via API only.

The emulator's software encoder holds real-time only up to ~486×1080 when the emulator runs with software rendering (no hardware GPU passthrough), so the full tier is deliberately downscaled on both paths. Measured tap → visible change: ~120–400 ms (H.264), ~200 ms (WebP). Native 1080×2400 is not real-time with a software encoder (1.2–5 s) and is not used.

A resolution switch does not flicker: the scrcpy video host restarts with the new max_size (the input channel stays up), the client resizes the canvas immediately, filters frames of the old size and covers the canvas until the first frame of the target size arrives.

Troubleshooting

• A system image does not boot — some images are finicky about the host virtualization stack; if one fails to reach sys.boot_completed, try another API level (API 33 is a stable default; emulators.json carries per-AVD notes).

• Native H.264 (1080×2400) is not real-time with a software-rendered emulator — by design, see Latency model.

• env_stop waits up to 25 s for a graceful emulator exit before a verified kill; SIGKILL mid-shutdown can wedge qemu in kernel D-state and leave stale AVD locks, which env_start/env_stop clean up themselves.

• Emulator killed externally (e.g. by the OOM killer) — the bridge restores the stream after the emulator is back; restart it manually if needed.

License

MIT © Shamil (cirkasssian)