DroidLab MCP

Android emulator under full control of an AI agent. DroidLab is an MCP (Model Context Protocol) server that lets any MCP client — Claude Desktop, OpenCode, Cursor, a custom agent — boot an Android emulator, drive its UI (tap / swipe / type / keys), read the screen (screenshot + UI element tree), install APKs, read logs, and use the clipboard. While the agent works, a human can watch the live screen in a regular browser over the local network.

Agent first. The MCP server is the product; the browser is the observation deck. An agent can set up and use the entire toolset without any human in the loop.

[MCP client / agent] ──stdio──> [MCP server] ──adb──> [Android emulator]
                                     │
[Browser / human] <──WebSocket──> [Node.js bridge] <──┘  (video + input relay)
   H.264 via WebCodecs

Table of contents

• Requirements

• Agent quick start

• Tools

• Resources

• Human quick start

• Architecture

• Security model

• Configuration

• WebSocket protocol

• Latency model

• Troubleshooting

• License

Related MCP server: phone-mcp-server

Requirements

Linux, macOS and Windows are supported. The bridge serves HTTPS (self-signed cert, generated on first start) because WebCodecs VideoDecoder requires a secure context. Open the access URL once and accept the certificate warning ("Continue to droidlab.local / unsafe").

Self-bootstrap: missing pieces are downloaded on demand. env_start creates a missing AVD by itself — it derives the API level from the AVD name (API33 → system-images;android-33;google_apis;<host ABI>, arm64 hosts get arm64-v8a), downloads the image via sdkmanager (pending SDK licenses are auto-accepted, 30-min cap) and runs avdmanager create avd -d pixel_7. If sdkmanager/avdmanager are absent, the official cmdline-tools package is fetched into <sdk>/cmdline-tools/latest; when no system java exists, Android Studio's bundled JBR is wired into JAVA_HOME/PATH. Missing scrcpy is downloaded to ~/bin/scrcpy/ (release v4.1 asset for the platform + scrcpy-server jar) before the bridge starts; SCRCPY/SCRCPY_SERVER env vars override the lookup. Downloads need network access.

Agent quick start

git clone https://github.com/cirkasssian/Droid-Lab-MCP.git droidlab && cd droidlab
bash scripts/install-mcp.sh

The installer is safe by design: it locates node at known absolute paths and never invokes brew upgrade/brew reinstall (a bare brew operation can collateral-upgrade unrelated apps — this is exactly how an improvised install broke opencode on 2026-09-14: brew reinstall node → brew replaced the opencode binary → every prompt failed with "Failed to send prompt"). If node is missing, it installs it with collateral-upgrade guards, smoke-tests the MCP handshake, and registers the server in ~/.config/opencode/opencode.json[c] (idempotent, with a backup).

Manual registration (any MCP client) — always use the absolute node path; GUI clients do not inherit the interactive shell PATH, so a bare "node" silently fails there:

{
  "mcpServers": {
    "droidlab": {
      "command": "/opt/homebrew/bin/node",
      "args": ["/absolute/path/to/droidlab/mcp/server.mcp.mjs"]
    }
  }
}

opencode (opencode.jsonc) uses the "mcp" block format: "command": ["/opt/homebrew/bin/node", "/path/to/mcp/server.mcp.mjs"] plus "env": { "PATH": "/opt/homebrew/bin:/usr/local/bin:/usr/bin:/bin" } as a fallback.

A typical first session:

1. env_start — boots the emulator (cold boot) and the bridge; blocks until Android is up.

2. screenshot / ui_dump / wait_for — see the screen, locate elements by text or resource-id.

3. tap / swipe / text / key — drive the UI in native pixels (default screen 1080×2400).

4. install_apk / push_file / pull_file / logcat / clipboard_get — install and inspect.

5. access_start — when a human needs to watch: returns a tokenized LAN URL.

6. env_stop — shut everything down when done.

All long operations (boot, image download, APK install, file transfer) support MCP cancellation (notifications/cancelled) and report progress (notifications/progress).

shell (raw adb shell) is included deliberately for full control and diagnostics; it is annotated destructiveHint: true. Prefer dedicated tools when they cover the task — annotations and structured output make them safer and easier to parse.

Tools

44 tools. Status tools (env_status, device_state, env_list, system_images_list, list_devices) also return structuredContent (MCP 2025-06-18). All tools declare MCP annotations (readOnlyHint / destructiveHint / idempotentHint).

Many tools accept a required confirm: true argument. It has no functional effect — it exists to prevent a known LLM failure mode: when a tool's arguments are all optional, some models emit a bare { (truncated JSON) instead of {} for empty calls, which the MCP client rejects with JSON parsing failed: Text: {. Requiring confirm forces the model to generate a complete {"confirm":true} object, eliminating the truncation.

Lifecycle

SDK / AVD management

Device interaction

Network access (for humans)

Configuration

On the first env_start (when config.json does not exist yet), the reply includes a note offering to customize the defaults via mcp_config.

Resources

Human quick start

Start the stack manually (or just ask the agent: "start the emulator and let me watch"):

# 1. Emulator
~/Android/Sdk/emulator/emulator -avd API33 -no-window -no-snapshot &

# 2. Bridge (starts scrcpy on its own)
node web/server.js

Open https://<host>:8090 in a browser (accept the self-signed cert warning once). The agent controls the stack over MCP and is the only party that opens network access (access_start) or unlocks browser input (set_dev_input).

Headless machine? Tunnel instead of exposing the port:

ssh -L 8090:localhost:8090 user@headless -N
# then: https://localhost:8090/?token=<accessToken> (localhost is a secure context — no cert warning)

Browser controls: click = tap, drag = swipe, wheel = scroll, keyboard = device input (printable text, Backspace, Enter, arrows, Esc), plus Back / Home / Recents / fullscreen buttons and a sound toggle (device audio is streamed as opus; the browser starts muted — autoplay policy). APKs can be dragged into the window (adb install -r -t); other dropped files land in /sdcard/Download/. Ctrl+C / Ctrl+V bridge the host clipboard with the device. Touch coordinates are mapped to the device's native screen size (queried via wm size), so tablets and phones both work correctly. The header shows FPS and the actual downlink bitrate (KB/s or MB/s, max across active h264 clients, updated 1×/sec). The canvas is hidden until the device's real aspect ratio is known (via /state), so the placeholder silhouette matches the actual device (tablet/phone).

Architecture

Primary video path — raw scrcpy-server, two instances per session:

• a video instance (control=false, audio=opus) for the stream, and a ctrl instance (video=false) for input/clipboard — with video=true the server maps touch coordinates through the video frame and display pixels never arrive;

• the bridge pushes scrcpy-server.jar, connects the sockets via adb reverse localabstract:scrcpy_<scid> and splits the stream into access units [u64 pts_flags][u32 size] (bit 62 = config, bit 61 = keyframe);

• the browser decodes H.264 through WebCodecs VideoDecoder; the stream is content-driven (no frames on a static screen) and has no recording time limit.

Fallback: VIDEO_SRC=screenrecord (180 s limit, auto-recycled at 170 s) when raw scrcpy is unavailable.

If the emulator dies, the bridge restores the stream automatically once the device is back.

Security model

• TLS. The bridge serves HTTPS with a self-signed cert (generated on first start, cached in ~/.local/state/droidlab/). WebCodecs requires a secure context — plain HTTP hides VideoDecoder. Accept the cert warning once per browser.

• Tokens. The bridge requires an access token (WEB_ACCESS_TOKEN, generated per bridge start): HTTP and WS without ?token= get 401. Browser input is gated by a separate WEB_CONTROL_TOKEN, so an observer cannot unlock input by itself. A manual start without WEB_ACCESS_TOKEN runs unauthenticated (trusted LAN only).

• Privileged actions. APK install / file push is not available to observers: when WEB_CONTROL_TOKEN is set, /push additionally requires ctoken=<WEB_CONTROL_TOKEN> — use the MCP install_apk/push_file tools instead. Without WEB_CONTROL_TOKEN (manual mode) the browser drag&drop works as before.

• Hardened input path. WS input messages are rate-limited (200 msg/s per connection) and every coordinate/keycode is validated as a finite number before it can reach adb; token comparisons are constant-time (crypto.timingSafeEqual).

• Role separation. The agent works through MCP; the developer watches (and taps only after set_dev_input(true)). While the agent works, browser input is blocked server-side.

• No shell injection surface. No raw adb shell tool; the network surface is one port with token auth.

Configuration

Environment variables (all optional):

End-to-end test: npm run e2e:mcp (boots the stack, exercises the toolset over real MCP stdio). CI runs a lighter smoke test on every push (node scripts/check-tools.mjs — tools/list + annotations, no emulator needed; see .github/workflows/ci.yml).

WebSocket protocol

Client → server, the first message picks the codec:

{"type":"init","codec":"h264"}

h264 is selected automatically when window.VideoDecoder exists; force it with ?codec=h264.

Input commands:

{"type":"tap","x":540,"y":1200}
{"type":"swipe","x1":540,"y1":10,"x2":540,"y2":1440,"ms":300}
{"type":"key","code":4}
{"type":"text","text":"hello"}

Server → client: binary frames [1 byte flag][payload] (bit 0 = keyframe for H.264 AUs, Annex-B; flag 0x02 = opus audio packet, 0x03 = OpusHead config — both raw from the device, 48 kHz stereo), {"type":"res","name":"486x1080"} on resolution changes, {"type":"bitrate","kbps":N} every second (actual downlink bitrate, max across active h264 clients), and a 1 Hz ping (RTT probe for ABR).

Latency model

Resolution is managed automatically (ABR) from network throughput; the ladder is 324x720 → 486x1080, 1004x2231 is available via API only.

The emulator's software encoder holds real-time only up to ~486×1080 when the emulator runs with software rendering (no hardware GPU passthrough), so the full tier is deliberately downscaled. Measured tap → visible change: ~120–400 ms (H.264). Native 1080×2400 is not real-time with a software encoder (1.2–5 s) and is not used.

A resolution switch does not flicker: the scrcpy video host restarts with the new max_size (the input channel stays up), the client resizes the canvas immediately, filters frames of the old size and covers the canvas until the first frame of the target size arrives.

Troubleshooting

• A system image does not boot — some images are finicky about the host virtualization stack; if one fails to reach sys.boot_completed, try another API level (API 33 is a stable default; the emulator registry carries per-AVD notes).

• Old API levels (14–19, 27) fail self-bootstrap on x86_64 hosts — ensureAvd derives the ABI from host arch (x86_64) but those API levels only have google_apis/x86 (32-bit) images. Create the AVD manually via avd_create with an x86 package, or set a device profile that matches.

• Native H.264 (1080×2400) is not real-time with a software-rendered emulator — by design, see Latency model.

• env_stop waits up to 25 s for a graceful emulator exit before a verified kill; SIGKILL mid-shutdown can wedge qemu in kernel D-state and leave stale AVD locks, which env_start/env_stop clean up themselves.

• Emulator killed externally (e.g. by the OOM killer) — the MCP watchdog auto-restarts it with the original arguments (guarded: max 5 restarts per 5 min, then it gives up and logs to crash.log), and the browser shows a crash banner until the stream recovers.

License

MIT © Shamil (cirkasssian)