android-jev

Control your Android phone through Jev.

You say what you want done — open the Play Store, turn on aeroplane mode, take a photo — and this does it. It plugs into any MCP client as a tool called run_task; inside, it reads the phone's own screen, asks Jev what to do next, does it, checks the result, and tells you honestly whether it worked.

Everything runs over adb. Nothing is installed on the phone.

run_task("open the Play Store")   →  achieved: true   done, 3 steps
run_task("search for YouTube")    →  achieved: true   done, 5 steps
run_task("open YouTube")          →  achieved: true   done, 3 steps

Quick start

What you need

• A phone attached by USB, with a data-capable cable. It stays plugged in while it works.

• USB debugging on — one setting, below.

• macOS with Python 3.11+, and uv. The picture route uses macOS Vision, which installs on macOS as a dependency; on Linux the loop works and reads the accessibility tree, and leaves the fallback out.

• Verified on a Pixel 8a running Android 17. The project has only been measured on that one, so treat other phones as untested rather than unsupported.

1. Connect the phone

Plug it in, and on the phone:

1. Settings → About phone → tap "Build number" seven times. Developer options appear.

2. Settings → System → Developer options → USB debugging → on.

3. Replug the cable. The phone shows Allow USB debugging? — tick Always allow and tap Allow. Nothing works until that tap happens, and it is deliberate: a computer cannot authorise itself.

Check the connection:

adb devices          # the phone should say "device", not "unauthorized"

2. Install the server

git clone https://github.com/FZ2000/android-jev ~/src/android-jev
cd ~/src/android-jev
uv venv --python 3.12 .venv
uv pip install --python .venv/bin/python -e .

The distribution is android-jev, the import is phone_control and the command is phone-control: three names, each read somewhere different. The command also answers to android-jev, so that uvx android-jev — the spelling a registry client derives from the distribution name — starts the server.

No adb? PHONE_CONTROL_ADB points at one, and the usual SDK locations are searched:

curl -sSL -o /tmp/pt.zip \
  https://dl.google.com/android/repository/platform-tools-latest-darwin.zip
unzip -q /tmp/pt.zip -d ~/.local/opt

3. Give it a decision model

run_task works without one, using a keyword fallback, and is markedly better with Jev: goals that do not name a control — turn off Bluetooth, open the settings app — need it.

Either of two keys works, and the key's own shape says which gateway serves it: an OpenRouter key beginning sk-or-, or a TypeSafe key beginning apikey_. Nothing in the configuration says which you are using, because sending one gateway's model to the other is a 400.

The server reads it from JEV_API_KEY — or OPENROUTER_API_KEY, if that is the name already in use here — or from a file named jev-api-key or openrouter-api-key in the secrets directory of whichever harness starts it. In DeepSeek Harness that is $DSH_HOME/secrets/:

printf %s "$JEV_API_KEY" > "$DSH_HOME/secrets/jev-api-key"
chmod 600 "$DSH_HOME/secrets/jev-api-key"

Whichever way it reaches the server, the key never appears in a state, a log, a run folder or a transcript.

4. Mount it in your agent

In Claude Code, this repository is also a plugin, which mounts the server and installs the skill in one step — steps 4 and 5 both:

claude plugin marketplace add FZ2000/android-jev
claude plugin install android-jev@android-jev

It asks for the Jev key once and keeps it in the system's credential store; leave it empty to use JEV_API_KEY or OPENROUTER_API_KEY from your environment instead. The server runs from the plugin's own copy of this repository through uv, so what it does always matches the skill that describes it. It still needs uv and adb, and the phone set up as in step 1.

For any other client, or to run from your own clone:

Any MCP client can mount this server; it speaks MCP over stdio. Send the agent this, with the path changed to wherever you cloned the repository — it says what the repository is and where its own documentation lives, because an agent that has to guess either of those will guess wrong. A prompt rather than a snippet per client: each client keeps its MCP configuration somewhere different, and changes the format between versions.

The repository at ~/src/android-jev is an MCP server that drives an Android phone attached by USB, together with the instructions for using it. Mount it for me, and tell me when you have.

• Start it as a local stdio server: the absolute path ~/src/android-jev/.venv/bin/python, the single argument -m phone_control, run from that directory.

• Call it android.

• It needs the Jev key in its environment, under JEV_API_KEY — or OPENROUTER_API_KEY if that is the name already in use here. Either an OpenRouter sk-or-… key or a TypeSafe apikey_… one: the shape of the key decides which gateway serves it. If your MCP bridge scrubs KEY, PASSWORD, SECRET or TOKEN out of a child process's environment, name the variable in the server's own env rather than expecting it to be inherited.

• Give it a generous per-call timeout, a couple of minutes: one of its tools waits for the phone by design.

• Read README.md at the repository root first. Its quick start covers connecting the phone, enabling USB debugging, installing the server and supplying the key; docs/ has the longer explanations. If the phone is not attached or the server will not start, read that rather than guessing.

• The instructions for using it are skills/android-phone-control/SKILL.md, with every tool and argument in references/tools-reference.md beside it.

When it is mounted, list its tools and tell me whether I need to start a new session before you can see them.

Three things catch people out, and the prompt is written the way it is to survive all three:

• The interpreter path has to be absolute. A client starts the server itself and does not inherit this project's virtualenv, so a bare python -m phone_control resolves to the wrong interpreter or to none at all.

• The key cannot always be inherited. DeepSeek Harness, for one, strips every variable whose name matches /KEY|PASSWORD|SECRET|TOKEN/i from a stdio child, so the key has to be named in the server's own env or it arrives empty.

• A mounted server is not a visible one. Most clients need a new session, or a restart, before a model sees the tools at all — which is why the prompt asks.

The tools surface under whatever namespace your client uses — mcp__android__run_task in DeepSeek Harness, something else elsewhere. Nothing in these instructions depends on the exact spelling, and neither should you.

5. Teach your agent how to use it

The skill is one body of text, skills/android-phone-control/SKILL.md, and this repository already ships it rendered for every harness that reads instructions from the working directory — which file each one reads is in The skill, for any agent below. If you work in this repository, your agent has it already and there is nothing to install.

For any other harness, send it this:

Install the phone-control skill in ~/src/android-jev for yourself, so that you can drive my phone in any session. The instructions are skills/android-phone-control/SKILL.md, with one reference file beside it at references/tools-reference.md. Put both wherever you read skills from, and tell me whether you need to be restarted before it takes effect.

If your harness has nowhere to keep a skill — a hosted assistant with an instructions box and nothing else — paste this instead, which is the part that matters:

The phone is driven by an MCP server mounted as android. To do anything on it, call run_task with the outcome you want in plain language — "open the Play Store", "turn on aeroplane mode". Ask for one thing at a time and wait for each result. Read achieved and reason from the reply, and relay them rather than retrying.

6. Sanity test: take a photograph

With the phone unlocked and the screen on, ask your agent:

run_task("open the camera app and take a photo")

Then check the phone's own evidence rather than the report:

adb shell ls -t /sdcard/DCIM/Camera | head -3      # a new file at the top

A new file and a report that it could not confirm the goal are both correct here, and this is the most useful thing the quick start can show you. A photograph leaves no trace on the screen — measured, the tree is byte-identical before and after the shutter — so the run can see that the camera opened and cannot see the picture. It says so instead of claiming a success it did not verify.

Run against the phone this was written on, through a fresh clone of this repository, that is what it looks like: five steps, three of them pressing the shutter, and then

"achieved": false,   "outcome": "stalled",
"reason": "3 actions in a row left the screen exactly as it was, so the run was
           going round rather than on"

and three new files in DCIM/Camera. It took the photograph and could not know it had, so it stopped rather than pressing the shutter forever. That is the honest answer to a goal whose result is off-screen, and it is why the last line of this test is one whose result is on the screen:

run_task("open the calculator app")     # achieved: true, done, 3 steps, ~13s

Related MCP server: android-mcp

Where to go next

• docs/engineering-decisions.md — why the code is the way it is, with the measurement that forced each decision.

• docs/code-map.md — every module, what it is for, and how they depend on each other.

• docs/android-apis.md — what this phone will and will not tell you, measured, including the reliable ways to prove an end state.

• docs/debugging.md — one run failing, and what to read.

• docs/tool-contract.md — the tool surface as a contract, including what is deliberately not built.

• docs/what-we-got-wrong.md — what was tried and did not work.

• CONTRIBUTING.md — the rules the tests live by.

Why it is built this way

Most adb wrappers expose tap(x, y), which is useless to an agent that cannot see. This one exposes what is actually on screen:

#5 ImageButton labeled "Send message" at (970,2170) [tappable]
#4 EditText labeled "Type a message" at (450,2170) [text field, focused]

Four decisions follow from that, and each cost something to learn.

The reading is structured, not a screenshot. read_screen reports every window with its type, whether the keyboard is up, and a typed description of every control — its category, label, bounds and a point to tap. An image is the fallback for genuinely visual screens, not the default. See docs/android-apis.md for what is extractable and what is not.

It reports what has focus, not what is merely running. When a system window such as the lock screen or the notification shade is in front, status says so and reports no foreground app — rather than naming the app behind it, which is what mFocusedApp would have said.

A control is addressed by name, then by number, and only then by coordinate. Every action tool re-reads the screen before it acts, so a name or a number is resolved against the screen as it is now; a coordinate is not. Read the reply — it names exactly what was acted on.

The agent states the goal; the server owns the loop. run_task("open the email app") is one call. The server reads its own screen, decides each step, checks its own work, and repeats until the goal is reached or cannot be — then reports what it did and why it stopped. An agent needs to know nothing about accessibility trees, windows or coordinates to use it.

Each step is several typed questions, asked together. Inside that loop, Jev, TypeSafe's System One decision model, answers four questions in one request and only the answer belonging to the chosen operation is used:

Splitting them is the part that mattered. One flat list holding operations, app names and screen controls made the decider unsure on a launcher screen — it answered give_up at 0.63 — and the reason is mechanical: confidence measures how concentrated a choice is, so overlapping options always read as doubt, whichever of them is right. Once the operations had a question of their own, open_app won the same screen at 0.74.

Only what can be carried out is offered. Typing needs a focused field, scrolling needs something that scrolls, a website needs a value the device can resolve, a tap needs a control to tap. An option the loop cannot execute is a guaranteed dead step and worse than that — it reads as the model being unsure when the model was never at fault. A web address with no scheme was offered once; Android could not resolve it, the action did nothing, and the 0.41 confidence on that step was read as doubt for days.

The state is facts computed from the phone, not a dump to interpret. What the focused field holds, where each control sits in coarse terms, what tells two identical labels apart, what has already been tried on this screen. This is what makes a threshold unnecessary rather than merely wrong: asked whether a goal was complete, Jev scored 0.42 to 0.47 on a screen where it genuinely was and 0.51 on one where it was not, because in neither case did the state say whether the query had been sent or was still in the box. No number separates those two; a state that says what is on the screen does.

A low number is not a reason to stop, and the loop does not decide. Every rule that used to refuse an answer for scoring low is gone, because each one was measured ending a run that was working. What is left is a stop rule about the phone — three actions in a row that left the screen as it was, or two already taken on the same screen, means the run is going round rather than on — and both err toward running on, because a futile run costs a minute and a working run cut short costs the task.

One exception, and it is about safety rather than sureness. An action that would have a material effect — sending, deleting, paying, granting a permission — is not carried out unless the goal explicitly asks for it. That gate exists because a real run tapped Allow Chrome to record audio while pursuing an unrelated goal.

When the loop stops short, a reader takes over. With no reader configured a stop ends the run as it always did. With one, it reads the screen and says whether the goal is met, and may set one next move for the decider to aim at — never choosing the action itself. The reference implementation measured 0.39 becoming 0.92 on the same capture once a focus was set, which is the argument in one number: the fix belongs in what the decider is told, not in a threshold.

The phone is read once per step. Reading it was 84% of a step, half of that a second read whose only job was to see what the action had done. The post-action observation is now the next step's reading:

before   step 7.29s   reading 2.91s   watching_the_result 3.18s   deciding 0.26s
after    step 3.75s   reading 2.18s   watching_the_result 0.00s   deciding 0.24s

What is sent is the source, not a summary of it. Jev's state carries the accessibility tree's own field names and values, and each option is a word (open_app) rather than our numbering. The measured prompting evidence is blunt: the same programs scored 0.894 as source, 0.598 as a syntax tree and 0.530 as a control-flow graph, even though the graph carried more information for 2.5× the tokens.

Running it directly

.venv/bin/python -m phone_control

It speaks MCP over stdio and waits. Nothing to configure; the tools appear as mcp__android__* once a client mounts it, as described in the quick start.

The tools

Full argument reference: skills/android-phone-control/references/tools-reference.md.

The skill, for any agent

skills/android-phone-control/ teaches the loop: start with the goal rather than the taps, when to read the tree rather than take a screenshot, and how to read Jev's confidence. The body names no agent, so the same instructions work anywhere.

Each tool reads a different file, so the skill is rendered into all of them from one canonical source — skills/android-phone-control/SKILL.md:

Edit the canonical skill and re-render; a test fails if any copy has drifted:

.venv/bin/python scripts/render_skill_for_agents.py

For DeepSeek Harness, symlink or copy skills/android-phone-control/ into a scanned skill root — $DSH_HOME/skills/ for every session, or .dsh/skills/ in a project. Discovery is live; no restart.

Environment

Known limits

• Whether a screen can be read at all varies moment to moment. This is the limit to know about, because it produces failures that look like the model's fault and are not. Measured on one page, read twice in a row: 2.9 seconds and 29 controls, then 73.8 seconds and a failure. The page cannot be dumped while it is doing something, and a page carrying a live figure is doing something most of the time and not all of it. So the same goal can succeed on one attempt and fail on the next, with nothing changed but the timing. Everything else below is a consequence or a neighbour of this one.

• A screen hosting embedded Android Views inside Compose can read as almost empty. The interop wrapper is marked NO_HIDE_DESCENDANTS, which hides its whole subtree from every accessibility client — no flag or setting overrides it, and Appium's workaround exists only because it ships its own dumper inside an installed APK. Measured, this is the common case on this device's Settings pages rather than the exception.

• When the tree will not describe a page, a photograph of it is read instead. That is the second route, and it reaches the tools as well as the loop: a page that cannot be dumped still comes back with its words. What it cannot do is give controls — the reading is lines of text with no tappable targets on them — so on such a page a run can conclude the goal and leave, and cannot act further. A page whose dump fails and whose photograph says nothing either is reported as a failure, not as an empty screen.

• dumpsys activity top is not a foreground oracle. It reads as though it should be, and a probe built on it will silently read another app's state: asked while the Settings Battery page was in front, it reported fragments belonging to Photos. dumpsys window names the page that is actually in front, and for a Settings page reached by tapping that is the generic SubSettings — the page's own title, read from the tree, is the reliable identity.

• A sleeping screen reads as empty. The display going off leaves one bare node. read_screen says which of asleep, locked or canvas-only it is.

• Non-ASCII typing. input text goes through the virtual keyboard's character map, which returns null for anything it cannot generate — AOSP's own comment says "For robust text entry, do not use this function." Non-ASCII is routed through the clipboard and a paste instead. The proper fix is ACTION_SET_TEXT.

• dumpsys output is internal and drifts between releases. Parsing is tested against dump shapes captured from real reports and exercised on a real device; docs/android-apis.md separates what was confirmed from what the device corrected.

Documentation

Testing

scripts/check.sh --as-ci                        # what CI runs: no phone, no key
scripts/check.sh                                # everything, with the 95% coverage gate
.venv/bin/python -m pytest -q -m "not device"   # everything that needs no phone
.venv/bin/python -m pytest -m device -q         # everything that needs a real phone

--as-ci is the one to run before pushing. It runs the fast suite on a machine that has never been set up — no adb, no decision-model key, no home directory holding either — because that is what the runner is, and the difference is not obvious from a machine that has both. The suite passed here for weeks while main was red for exactly that reason.

The default suite runs without a device; device tests skip themselves when none is attached, and when the phone is asleep or locked — that is a precondition, not a code property. The same is true of three other things the suite needs and cannot provide: a Jev key, a network that holds for the length of a run, and macOS's Vision framework, which is the only way to read a picture of a screen. A test that needs any of them stands itself aside with the reason rather than failing, because a dropped connection is not a fact about this code.

That last one is worth knowing when reading a build: on the Linux runner, every test that photographs a screen stands aside, so the picture route is verified on a Mac rather than in CI. scripts/check.sh --as-ci reproduces the missing adb and the missing key, because it runs on a Mac it cannot reproduce the missing framework.

The counts are deliberately left out of this list. They change with every commit that adds a test, and a number in a README is stale the moment after it is written — these commands print the real ones. What does not change is the shape: the majority of the suite needs no phone, and the device suite is the part that cannot be faked.

Coverage has to be read from both suites together, against a gate of 95%: most of this package only executes with a phone attached, so the fast suite alone fails a gate on code it had no way to reach. scripts/check.sh runs both. CI runs the fast suite only, because a hosted runner has no phone.

The usage scenarios

tests/scenarios/ is the specification: all 47 scenarios, each a thing a person asks a phone to do, from one launch to goals no phone can reach. Each carries its own starting state, its own ground truth read from the phone, and a screenshot at every stage.

scripts/run_scenarios.py --tier 1            # a tier at a time
scripts/run_scenarios.py --id bluetooth-off  # or one, by name
scripts/run_scenarios.py --all               # about half an hour

Artefacts land in artifacts/scenarios/<id>/. See docs/debugging.md for how to read them.

The rules that keep the tests honest

scripts/install_git_hooks.sh      # once: refuse a commit whose suite is red
.venv/bin/python -m pytest tests/test_no_xfail.py -q   # no test may be allowed to fail

Four times in this project a fake agreed with its author and the belief was wrong: that a bare domain opens a browser, that am start -d resolves it, that cmd clipboard set places something, that a settings toggle is checkable. Each cost real work. Hence two rules rather than good intentions — a test allowed to fail reports green while the behaviour it describes does not work, and a belief about Android belongs in tests/test_what_the_phone_actually_does.py, where the phone gets asked.

Watching a run afterwards

An MCP reply is one shot and a run leaves nothing behind unless you ask it to:

export PHONE_CONTROL_RUNS=~/.phone-control/runs

Every run then writes a folder — the report, the exact request and answer for every step, and where the seconds went — and the tool's reply names it. A recorded step can be re-decided offline, without the phone:

.venv/bin/python scripts/replay_run.py ~/.phone-control/runs/<folder> --step 0

Status

Working and device-validated: opening apps, reading and acting on screens, scrolling, typing, window and keyboard state, screenshots.

Measured, on a Pixel 8a running Android 17

tests/test_jev_does_the_job.py asks for things a person would ask for and then checks the phone rather than the report. Every goal below was independently confirmed against the focused window, which the decision never sees:

No false successes: nothing claimed a result the phone did not reach. The completion check agreed with the chosen action every time, at confidences of 0.97, 1.00 and 0.99.

One honest failure. Asked to "order a large pepperoni pizza", which no phone can do, it correctly reported the goal unmet — but it spent all ten steps exploring Chrome's interface first, including tapping "Allow Chrome to record audio". It explores before it concedes, and some of that exploration is consequential. Bounding that is open work.

Not yet done, and named rather than implied: the tool contract migration described in docs/tool-contract.md, and any device run of the Jev decision path, which needs a Jev key.