# voice-android-mcp

An MCP server that lets Claude **see** your Android phone and **press its buttons by voice**:

1. Claude takes a **screenshot over ADB** and reads the screen (numbered boxes + element list).
2. Text that needs typing is typed **over ADB**.
3. To press a button, Claude **says it out loud** ("click Send") through the **PC speakers**.
   The phone hears it with Google **Voice Access** (accessibility) and taps the button.
4. Claude takes another screenshot to confirm (Voice Access shows a toast like "Tapped 6").

Plain ADB tapping (no voice) is also available: set `TAP_MODE=adb`.

## Phone setup (once)

1. On the phone: *Settings → About phone →* tap **Build number** 7 times.
2. *Settings → Developer options →* turn on **USB debugging**.
3. Plug in by USB, unlock the phone, accept **Allow USB debugging**.
4. Check: `adb devices` shows your phone as `device`.
5. Install and turn on **Voice Access** (Google): *Settings → Accessibility → Installed apps → Voice Access*.
   Its notification should say "Tap to pause" (= listening). Set its language to English.
6. Put the phone close to the PC speakers and turn the volume up.

Wi-Fi instead of USB: turn on *Wireless debugging*, then ask Claude to `connect_wifi("IP:PORT")`.

To type Arabic / emoji / accented text, install [ADBKeyboard](https://github.com/senzhk/ADBKeyBoard)
on the phone (`adb install ADBKeyboard.apk`). Plain ASCII text works without it.

## Install

Needs [uv](https://docs.astral.sh/uv/), adb, and **ffmpeg** on PATH (decodes the neural voice).

```bash
uv sync
```

## Use with Claude Code

This folder has a `.mcp.json`, so Claude Code opened here picks up the server automatically
(approve it when asked). To use it from any folder:

```bash
claude mcp add voice-android --scope user -e TAP_MODE=voice -e TTS_OUTPUT=Realtek -- uv --directory "/path/to/voice-android-mcp" run voice-android-mcp
```

Then ask, e.g. *"open Settings and turn on Bluetooth"*, or use the **phone_task** prompt
(in Claude Code: `/mcp__voice-android__phone_task open YouTube and search for lofi music`).

## Tools

| Tool | What it does |
|---|---|
| `screenshot` | Screen image with numbered boxes + element list (`[3] button "Send" at (528,1205)`) |
| `voice_tap` | Say "click <name>" / "click <Voice Access number>" so Voice Access presses it |
| `voice_command` | Say any Voice Access command: "go back", "scroll down", "open WhatsApp", "show numbers" |
| `voice_access_status` | Check Voice Access is installed and enabled |
| `type_text(text, submit)` | Type into the focused field over ADB |
| `get_ui_elements` | Element list only (no image, cheaper) |
| `tap_element(n)` / `tap(x,y)` | ADB taps (used when `TAP_MODE=adb`) |
| `long_press`, `swipe`, `scroll`, `press_key` | ADB gestures and keys |
| `open_app`, `list_apps`, `open_url`, `current_app` | Apps and links |
| `wake_and_unlock_swipe`, `wait`, `adb_shell` | Utilities |
| `list_devices`, `select_device`, `connect_wifi` | Device management |
| `speak`, `list_speakers` | Say anything on the PC speakers / pick the speaker |
| `listen`, `list_microphones` | Optional: your voice → text (PC mic or phone mic) |

## Settings (environment variables)

| Variable | Default | Meaning |
|---|---|---|
| `TAP_MODE` | `adb` | `voice` = press buttons by speaking to Voice Access (`.mcp.json` sets this) |
| `TTS_OUTPUT` | Windows default | Speaker for the voice: index or name part (`list_speakers`), e.g. `Realtek` |
| `TTS_ENGINE` | `edge` | `edge` = Microsoft neural voice (online, best recognised); `sapi` = built-in Windows voice |
| `EDGE_VOICE` | `en-US-AriaNeural` | Neural voice name (`en-US-GuyNeural`, …) |
| `VOICE_CMD_RATE` | `-2` | Speed of spoken commands, -10 (slow) … 10 (fast) |
| `TTS_LEAD_SECONDS` | `0.5` | Silence before each phrase so the speaker doesn't clip the first word |
| `ANDROID_SERIAL` | – | Device to use when several are connected |
| `ADB_PATH` | `adb` on PATH | Full path to adb |
| `SCREENSHOT_MAX_SIDE` | `1280` | Screenshot size sent to Claude (smaller = cheaper) |
| `VOICE_LANG` | `en-US` | `listen` language: `en-US`, `fr-FR`, `ar-MA`… |
| `VOICE_MIC` | auto | `listen` input: `phone` or a PC mic index / name part |
| `VOICE_ENGINE` | `google` | `listen` recogniser: `google` (online) or `whisper` (`uv sync --extra whisper`) |

## Tips (from testing on a Galaxy S23 Ultra)

- Say **"click"**, not "tap": "tap …" was misheard as the app "Tap Tile" and opened it.
- The old Windows voices (David/Zira) were rarely understood; the neural voice worked reliably.
- A bare number ("6") was not recognised; "click 6" was.
- Voice Access numbers (grey pins on the phone) differ from the coloured box numbers in our
  screenshot. `voice_tap(index=…)` speaks the element's *name*; `voice_tap(va_number=…)` speaks the grey pin.
- While Voice Access is on, Android gives it the phone's microphone, so `listen` with `VOICE_MIC=phone`
  hears nothing. The phone-mic option uses the bundled [scrcpy](https://github.com/Genymobile/scrcpy)
  server (Apache-2.0, v5.0.1, SHA-256 checked), pushed to `/data/local/tmp` on the phone.
- Banking apps and DRM video often block screenshots (black image) — that's Android, not a bug.

## License

MIT — see [LICENSE](LICENSE). The bundled `src/voice_android_mcp/vendor/scrcpy-server-v5.0.1`
is from [scrcpy](https://github.com/Genymobile/scrcpy) by Genymobile and stays under the
[Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0).