# Auto Phone

**English** | [简体中文](https://github.com/Rynque/auto-phone/blob/main/README.zh-CN.md)

<!-- mcp-name: io.github.rynque/auto-phone -->

Auto Phone is a command-line tool that controls an Android phone over adb and outputs JSON, along with an MCP server and an agent skill.

This project is developed with vibe coding.

## Requirements

- Python 3.10 or later.
- `adb` installed and on your PATH; `PHONE_ADB` can point at it instead.
- USB debugging enabled on the phone, authorized after connecting to the computer.
- Installing ADBKeyboard on the phone is recommended.

## Install

Install straight from GitHub (this needs `git` on your machine). The CLI and the MCP server come from the same Python package:

```bash
# pipx (recommended)
pipx install "auto-phone[mcp] @ git+https://github.com/Rynque/auto-phone"

# uv users
uv tool install "auto-phone[mcp] @ git+https://github.com/Rynque/auto-phone"

# or pip
pip install "auto-phone[mcp] @ git+https://github.com/Rynque/auto-phone"
```

`[mcp]` is the recommended install: it includes the MCP server dependency and Pillow. Pillow handles screenshot downscaling.
If you only use the command line and don't need downscaling, drop `[mcp]`.

After installing, run the self-check:

```bash
phone doctor
```

It checks, item by item: the adb executable, device connection, SDK version, screen state, foreground app, ADBKeyboard, Pillow and the model endpoint.
The model endpoint is tested only when `PHONE_BASE_URL` is set. Anything missing comes with a way to deal with it.

## Quick start

```bash
phone screenshot shot.png
phone tap 540 1200
```

The first command saves a screenshot to `shot.png`. Open the image, find the target, and pass its pixel coordinates to `phone tap`.
Screenshots are full resolution by default, so pixels in the image are screen coordinates and can be used as arguments directly.
If you pass `--max-width` when capturing, convert using `orig_width` and `width` from the returned JSON.

Take another screenshot after an action to confirm the result. If the page is still loading or an animation hasn't finished, wait first:

```bash
phone wait 1.5
```

## Output format

Except for `phone run` and `phone resume`, subcommands print one line of JSON. On success:
```json
{"ok": true, "data": {"tapped": [540, 1200]}}
```

On failure:
```json
{"ok": false, "error": {"code": "DEVICE_NOT_FOUND", "message": "No online device", "hint": "Check the USB connection and developer options"}}
```

Failures exit with code 1. `hint` says what to do next and is also written to stderr. `phone run` and `phone resume` print human-readable progress; with `--json-events` that becomes a JSONL event stream.

## Agent integration

### Agent skill

```bash
npx skills add Rynque/auto-phone
```

A skill only provides usage instructions — the package above has to be installed first.

### MCP server

Register `phone-mcp` as a stdio server in your MCP host:

```json
{
  "mcpServers": {
    "auto-phone": {
      "command": "phone-mcp"
    }
  }
}
```

If the command isn't on your PATH, the host can run `python -m auto_phone.mcp_server` instead.

MCP tool names match the CLI subcommands, plus a `sequence` tool for running several actions in a row. The model loop behind `phone run` is not exposed over MCP; the host's own model decides the actions.
Coordinates refer to the image returned by the latest `screenshot`, and the server converts them to screen coordinates automatically.

## phone run

`phone run` starts the built-in vision-model loop: take a screenshot, decide the next action, execute it, verify with another screenshot. It needs an OpenAI-compatible vision endpoint:

```bash
export PHONE_BASE_URL=https://api.example.com/v1
export PHONE_API_KEY=sk-...
export PHONE_MODEL=your-vision-model

phone run "open Settings and set the screen timeout to 10 minutes"
```

When a session is interrupted, its context is saved to `.phone-session.json` in the current directory; `phone resume` continues it.

Common options:
- `--json-events` switches the output to a JSONL event stream, easier for programs to consume.
- `--point-confirm` draws the coordinates onto the screenshot for confirmation before executing; more accurate, at the cost of one extra model call per step.
- `--max-steps` caps the number of steps, 50 by default.

## Commands

```bash
phone screenshot [path] [--max-width N]
phone ui
phone current
phone size
phone devices

phone tap X Y
phone double-tap X Y
phone long-press X Y
phone swipe X1 Y1 X2 Y2 [--duration-ms 300]
phone drag X1 Y1 X2 Y2
phone slide X1 Y1 X2 Y2
phone key back [name ...]
phone type "text"
phone clear-text

phone launch alias-or-package
phone stop alias-or-package
phone clear-bg

phone wait 1.5
phone run "task"
phone resume [instruction]
phone doctor
```

In detail:
- `ui` prints the list of interface elements with their center coordinates; use it as a fallback when screenshots fail.
- `current` prints the foreground app and screen state.
- `swipe` is a straight-line swipe for scrolling and page turns; `--duration-ms` defaults to 300.
- `drag` is the same move at a lower speed, for reordering and sliders.
- `slide` is for CAPTCHA sliders only; its path includes overshoot and jitter.
- `key` accepts several key names at once and presses them in order. Names include `home`, `back`, `enter`, `del`, `tab`, `space`, `vol-up`, `vol-down`, `power`, `menu`, `esc`, a single letter, a single digit, and numeric keycodes.
- `launch` and `stop` accept package names as well as built-in aliases. Aliases work in either language, e.g. `微信` and `wechat`, `淘宝` and `taobao`, `抖音` and `douyin`, `设置` and `settings`.

## Environment variables

- `PHONE_DEVICE` sets the device serial. It is required when more than one device is connected; otherwise commands fail with `AMBIGUOUS_DEVICE`.
- `PHONE_ADB` sets the path to the adb executable. PATH is searched first, and common SDK install locations are probed if that fails.
- `PHONE_BASE_URL`, `PHONE_API_KEY` and `PHONE_MODEL` are the model endpoint, key and model name that `phone run` needs.
- `PHONE_ADBKEYBOARD_APK` points at the ADBKeyboard apk file; once set, it is installed automatically when needed.
- `PHONE_MAX_STEPS` overrides the step cap for `phone run`, 50 by default.

## License

Released under the MIT license.