phone-mcp-server
Allows CrewAI agents to control Android phones via MCP, enabling screen capture, tapping, typing, swiping, app management, and device info retrieval.
Allows Google Gemini to control Android phones via HTTP API, using function calling for screen capture, tapping, typing, swiping, app management, and device info.
Allows LangChain agents to control Android phones via HTTP API, integrating phone actions as tools for screen capture, tapping, typing, swiping, app management, and device info.
Allows OpenAI agents (GPT models, Codex CLI) to control Android phones via HTTP API, providing tools for screen capture, tapping, typing, swiping, app management, and device info.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@phone-mcp-servertake a screenshot of the current screen"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
phone-mcp-server
English | 中文
Standalone MCP + HTTP server for controlling Android phones from any AI agent.
Works with Claude Desktop, Claude Code, OpenAI Codex CLI, GPT agents (via OpenAI API), Gemini, LangChain, AutoGen, CrewAI, Open Interpreter, or any HTTP client.
How It Works
┌──────────────────────────────────────────────────┐
│ Any AI Agent │
│ │
│ Claude ──── MCP (stdio) ──┐ │
│ Codex ──── MCP (stdio) ───┤ │
│ ▼ │
│ ┌──────────────┐ │
│ │ MCP Server │ │
│ │ mcp_server │ │
│ └──────┬───────┘ │
│ │ │
│ GPT ──── HTTP ────┐ │ │
│ Gemini ── HTTP ───┤ │ │
│ Custom ── HTTP ───┤ │ │
│ ▼ ▼ │
│ ┌─────────────────┐ │
│ │ phone_control │ │
│ │ (core package) │ │
│ └────────┬────────┘ │
│ │ │
│ ADB ────┤──── Appium (optional) │
│ │ │
├───────────────────────┼──────────────────────────┤
│ Android Emulator │ │
└───────────────────────┴──────────────────────────┘Related MCP server: scrcpy-mcp
Requirements
Python 3.10+
Android SDK Platform Tools (
adbon PATH)A running Android emulator or device
Optional (for Unicode text input and WebView support):
Appium (
npm install -g appium)Appium Python client (
pip install Appium-Python-Client)
Optional (for apps with an empty accessibility tree):
macOS 13+ and Xcode Command Line Tools. Build the local Vision OCR helper:
mkdir -p ~/.phone-mcp/bin && swiftc phone_control/phone_ocr.swift -o ~/.phone-mcp/bin/phone-ocr -framework Vision -framework ImageIO
Install
git clone https://github.com/Ctrl-Creeper/phone-mcp-server.git
cd phone-mcp-server
pip install .
# With Appium support
pip install ".[appium]"
# Optional: install Hermes Phone Agent for secure Unicode/special-character input
curl -fL -o hermes-phone-agent-v0.2.1.apk \
https://github.com/Ctrl-Creeper/hermes-phone-agent/releases/download/v0.2.1/hermes-phone-agent-v0.2.1.apk
adb install -r hermes-phone-agent-v0.2.1.apkQuick Start
Claude Desktop
Add to ~/Library/Application Support/Claude/claude_desktop_config.json:
{
"mcpServers": {
"phone-control": {
"command": "python",
"args": ["/path/to/phone-mcp-server/mcp_server.py"]
}
}
}Claude Code
claude mcp add phone-control python /path/to/phone-mcp-server/mcp_server.pyOpenAI Codex CLI
codex --mcp-config codex-mcp.jsonCreate codex-mcp.json:
{
"mcpServers": {
"phone-control": {
"command": "python",
"args": ["/path/to/phone-mcp-server/mcp_server.py"]
}
}
}OpenAI API / GPT Agents
Start the HTTP server, then fetch the tool schema:
python http_server.pyimport requests, openai
tools = requests.get("http://localhost:8080/openai/tools").json()
response = openai.chat.completions.create(
model="gpt-4o",
messages=[{"role": "user", "content": "Open Settings on the phone"}],
tools=tools,
)
tool_call = response.choices[0].message.tool_calls[0]
result = requests.post("http://localhost:8080/openai/call", json={
"name": tool_call.function.name,
"arguments": tool_call.function.arguments,
}).json()Google Gemini
import requests, google.generativeai as genai
tools_schema = requests.get("http://localhost:8080/openai/tools").json()
# Convert to Gemini format
gemini_tools = []
for t in tools_schema:
f = t["function"]
gemini_tools.append(genai.types.Tool(
function_declarations=[genai.types.FunctionDeclaration(
name=f["name"],
description=f["description"],
parameters=f["parameters"],
)]
))
model = genai.GenerativeModel("gemini-2.0-flash", tools=gemini_tools)
chat = model.start_chat()
response = chat.send_message("Open the camera app")
# Execute the function call
fc = response.candidates[0].content.parts[0].function_call
result = requests.post("http://localhost:8080/openai/call", json={
"name": fc.name,
"arguments": dict(fc.args),
}).json()LangChain
import requests
from langchain_core.tools import StructuredTool
def phone_action(action: str, **kwargs):
return requests.post(f"http://localhost:8080/phone/{action}", json=kwargs).json()
# Or dynamically load from schema
tools_schema = requests.get("http://localhost:8080/openai/tools").json()Any HTTP Client (curl)
# Capture UI hierarchy
curl -s localhost:8080/phone/capture -d '{"mode":"hierarchy"}' | jq .
# Tap element #3
curl -s localhost:8080/phone/tap -d '{"element":3}' | jq .
# Type text
curl -s localhost:8080/phone/type -d '{"text":"hello world"}' | jq .
# Get device info
curl -s -X POST localhost:8080/phone/device_info | jq .
# Fetch OpenAI tool schema
curl -s localhost:8080/openai/tools | jq .Exposed Tools (18)
Tool | Description |
| Capture screen (hierarchy / screenshot / both) |
| Tap by element index or coordinates |
| Double-tap |
| Long-press (configurable duration) |
| Swipe by direction or coordinates |
| Type text (Unicode via Appium hybrid) |
| Clear text field |
| Clear + type new text |
| Send key event (BACK, HOME, ENTER, etc.) |
| Launch app by package name |
| Force-stop app |
| List installed apps |
| Get foreground app |
| Device model, screen size, Android version |
| Wait N seconds |
| Find and open a WeChat conversation by title with OCR search and 80% matching |
| Reply to a verified WeChat conversation with delivery confirmation and bounded recovery |
| Scroll, OCR, deduplicate, and collect bounded WeChat history |
The WeChat tools do not depend on a fixed list position. They search when the
conversation is not visible, ignore volatile member-count suffixes, verify the
opened title and input area, and recover conservatively after transient UI failures.
phone_capture(mode="hierarchy") automatically falls back to local OCR when an
app exposes no usable accessibility nodes.
Configuration
Environment Variable | Description | Default |
|
|
|
| Device serial (auto-detected if one device) | — |
| Appium server port (hybrid backend) |
|
| Path to phone-policy.yaml | auto-search |
| MCP SSE server port |
|
| HTTP server port |
|
Policy Engine
The phone policy (phone-policy.yaml) controls what actions the agent can perform on which apps. Place it at ~/.hermes/phone-policy.yaml or set PHONE_POLICY_PATH.
See the virtual-phone-agent repo for the full policy reference and examples.
Security
All ADB commands use argument-list subprocess (no shell injection)
install_apkandshellare blocked over HTTP APIPolicy engine enforces per-app action restrictions
Phone content is untrusted data — never treated as instructions
Input sanitization: shell metachar rejection, keycode allowlist, coordinate bounds, text length limits
License
AGPL-3.0
This server cannot be deployed
Maintenance
Related MCP Connectors
Control real Android and iOS devices with LLM agents — tap, swipe, type, automate flows.
Melaya is a remote MCP server. It gives an assistant hands on your own Android phone and browser: it reads the screen through the accessibility tree, then taps, types and navigates inside the apps and sites you allow-list, with no per-app API. It also builds, schedules and runs agent pipelines across 6k+ connected tools. OAuth 2.1, nothing to install.
Give AI agents real phone numbers, messages, and voice calls via MCP.
Automate 1,000+ services from any MCP-compatible AI agent: build Applets, run actions and queries.
Related MCP Servers
- AlicenseNot gradedqualityCmaintenanceEnables full Android control from any AI agent via 7 MCP tools, including screen capture, touch interaction, app management, and system control.MIT
- AlicenseAqualityAmaintenanceMCP server that gives AI agents full vision and control over Android devices via ADB and scrcpy. Supports screenshots, input, apps, UI automation, shell, files, and clipboard.38302 npm96MIT
- AlicenseNot gradedqualityDmaintenanceEnables MCP-compatible agents to control an Android device over the network via ADB, providing tools for shell commands, screen capture, UI inspection, file operations, and input simulation.5 npmMIT
- AlicenseNot gradedqualityCmaintenanceEnables AI agents to control Android devices via ADB, providing tools for screen capture, tap/swipe/input, app management, and diagnostics. Works locally over USB or WiFi and integrates with MCP clients like Claude and Cursor.1MIT