This server lets AI agents automate and visually verify Android apps rendered entirely inside an opaque OpenGL/Vulkan/Metal surface (e.g. Cocos2d-x, Unity, libGDX), using screenshots plus a pluggable vision provider instead of the accessibility tree.
Take screenshots and ask short visual questions (screenshot_ask).
Tap, swipe, long-press, hold, drag, and release, each optionally followed by a screenshot + vision question (tap_and_ask, swipe_and_ask, long_press_and_ask, hold_and_ask, swipe_hold_and_ask, release_and_ask).
Perform actions without vision calls for cheaper steps (tap, hold, swipe_hold, release).
Record timed screenshot sequences to check animations (record_and_ask) and mid-drag behavior (record_swipe_and_ask).
Type text, send Android key events, and read filtered logcat (type_text, press_key, logcat_grep).
Track vision spending with built-in per-call alerts and daily caps (vision_spend_report).
Plug in different vision models via providers: runware, openai, openrouter, or any OpenAI-compatible endpoint.
Provides tools for automated testing and interaction with Android apps that render their UI entirely within a single OpenGL surface, including combined action-and-observe operations like tap, swipe, long-press, screenshot capture, vision-based question answering, text input, key events, and logcat analysis.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type @ followed by the MCP server name and your instructions, e.g., "@mobile-mcp-opengl tap at (120, 450) and describe the screen"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
An MCP server for AI coding agents (Claude Code, Cursor, etc.) to test Android apps whose entire UI is drawn inside a single opaque OpenGL/Vulkan/Metal surface — Cocos2d-x, Unity, Unreal, raw OpenGL, libGDX, and similar engines.
adb shell uiautomator dump and every accessibility-tree-based automation tool (including most MCP mobile-automation servers) work by inspecting the native Android view hierarchy — buttons, labels, their text and coordinates. That works great for a normal Android UI built out of native views.
It does not work for a game or app that renders its entire UI as textures inside one GLSurfaceView. From the accessibility tree's point of view, there is exactly one opaque view on screen with no children, no labels, no coordinates for anything inside it. There is nothing to inspect — the screen is a black box, no matter how much UI is actually on it.
The only real observation channel left is screenshots. This server is built around that fact as the normal case, not as an occasional fallback.
Unlike mobile-next/mobile-mcp (a good default for normal native apps, accessibility-tree first with a screenshot fallback), this server skips the accessibility-tree attempt entirely — it's pointless for OpenGL-canvas apps — and routes every vision call through a pluggable, separate provider (see "Bring your own model" below) rather than the calling agent's own vision, so screenshot bytes never enter the agent's context — only the provider's short answer does.
Related MCP server: Android-MCP
Tool | What it does | Vision call? |
| Screenshot, then ask a short question about it | Yes |
| Tap (x, y), wait, screenshot, ask | Yes |
| Swipe/drag (x1,y1)→(x2,y2), wait, screenshot, ask | Yes |
| Press down at (x, y) and HOLD (no release), wait, screenshot, ask | Yes |
| Drag to (x2, y2) and HOLD (no release) - continues an existing hold if one is active | Yes |
| Release whatever touch is currently held, wait, screenshot, ask | Yes |
| Tap (x, y) - no screenshot, no vision call | No |
| Press down at (x, y) and HOLD (no release) - no screenshot, no vision call | No |
| Drag to (x2, y2) and HOLD (no release) - no screenshot, no vision call - continues an existing hold if one is active | No |
| Release whatever touch is currently held - no screenshot, no vision call | No |
| Long-press (x, y) for a duration, wait, screenshot, ask | Yes |
| Optional action, then N screenshots spaced apart in time, ask the same question about each frame | Yes (N calls) |
| Like | Yes (N calls) |
| Type into the currently focused field | No |
| Send an Android | No |
| Read recent logcat, optionally filtered by regex | No |
| Report today's cumulative vision spend and thresholds | No |
See "Tools usage" below for how the _and_ask/quiet pairs, drag/hold composition, and the two multi-frame tools actually work together.
git clone <this repo>
cd mobile-mcp-opengl
npm install
cp .env.example .env
# edit .env: at minimum set RUNWARE_API_KEY - see "Bring your own model" below
# for what the default VISION_PROVIDER=runware actually talks to, and how to
# switch providers/modelsRequires adb on PATH (or ADB_PATH set in .env), and a running/connected device or emulator. If more than one is attached, set ADB_DEVICE_SERIAL (see adb devices).
Add a .mcp.json in your project root (this file is typically project-local and git-ignored, since it usually points at a machine-specific path or holds machine-specific env overrides):
{
"mcpServers": {
"mobile-opengl": {
"command": "node",
"args": ["/absolute/path/to/mobile-mcp-opengl/src/server.js"]
}
}
}Claude Code picks this up automatically for the project. The server reads its own .env (next to package.json in this repo) for all configuration — the calling agent never needs to know or pass any API key itself.
AGENTS.md is a short, portable set of behavioral rules for any agent using this server — how to use the tools well, not what they do (that's the "Tools" table above). Either link to it from your own CLAUDE.md/AGENTS.md/equivalent, or copy its content in and adapt as needed. Don't duplicate the tool reference or cost/model tables here into your own instructions — link back instead, so they don't drift out of sync.
No built-in provider has a reliable confidence signal an agent could threshold on to decide when to double-check (confirmed directly — GPT-5 rejects logprobs, Runware's imageCaption returns no confidence field, and verbalized self-confidence is known to be unreliable in general). AGENTS.md's substitute: always check the saved screenshot yourself when an answer matters, not just when a model claims low confidence.
Response length drives cost, not image size. Measured empirically against runware:150@2 (LLaVA-1.6-Mistral-7B, the runware provider's default — see "Bring your own model" below): the same question asked with a forced one-word answer cost the same ($0.0006) across image sizes from 360×360 up to 1600×2400 (retina-class). The same 1024×1024 image with an open-ended "describe this" prompt cost $0.0013–0.0019 — 2-3x more — purely because the model wrote a longer answer, not because the image was bigger.
Practical implications:
Don't bother downsampling screenshots before sending them — it doesn't meaningfully reduce cost, and you lose detail you might need.
Always phrase questions to force short answers: yes/no, a number, a short label, a tiny JSON object with a couple of fields. Every tool in this server appends a short-answer instruction automatically, but a vague open-ended question ("what do you see?") can still push the model toward a longer answer than a specific one ("is the error dialog visible? yes/no").
What a 500-call QA session costs, by provider (500 calls is a realistic session — a few dozen test scenarios, each a handful of check-ins):
Provider (default model) | Cost/call (short answer) | 500-call session |
| ~$0.0006 | ~$0.30 |
| ~$0.006 | ~$3.11 |
The openai provider costs about 10x more per call for a meaningfully bigger model. In this server's own small side-by-side comparison it read as somewhat more reliable, but not by enough to obviously justify that 10x — see "Switching models if accuracy is insufficient" below for the actual test results, and run your own comparison against your app's screens before assuming that holds for your case. runware is the recommended default here; reach for openai for a specific case where the cheap default is demonstrably failing you, not as a blanket upgrade. Both numbers roughly double or triple if your questions are open-ended instead of short (same response-length effect as above) — budget accordingly for whichever provider you're running.
Every vision call is logged to .data/vision-log.jsonl (JSONL, one entry per call: timestamp, question, answer, cost) — same gitignored .data/ directory as the saved screenshots (see "Screenshot files" below). Two independent protections sit on top of that log, both provider-agnostic (they work off whatever costUsd a provider reports):
Per-call alert (VISION_ALERT_USD, default $0.0015): if a single call comes back above this, the tool's response includes a [COST ALERT] note telling you the model likely ignored the short-answer instruction — a signal to reformulate the question, not something to silently eat.
Daily cap (VISION_SESSION_CAP_USD, default $2.00): once today's cumulative logged spend reaches this, every further vision call is refused outright (before it reaches the provider) until the cap is raised or the day rolls over. This is a hard stop against a runaway loop, not just a warning.
Call vision_spend_report any time to check today's total without making a device or vision call.
If a provider can't report a cost (see openai-compatible in "Bring your own model" below), calls from it are logged with costUsd: null and never trigger the alert or count toward the cap — the guardrails simply can't protect spend they have no visibility into.
A naive design exposes tap, screenshot, and ask as three separate tools. That forces the calling agent to orchestrate a multi-step loop for every single interaction: tap → take a screenshot → hand it to a vision step → read the result → decide what to do next. Each of those is a separate tool call and a separate turn — burning tokens on coordination instead of on the actual test logic, and giving more surface area for the agent to drop a step, misorder them, or reason about stale state between calls.
Instead, this server exposes combined tools — tap_and_ask, swipe_and_ask, long_press_and_ask — that perform the action, wait briefly, take the screenshot, ask the vision provider, and return one short answer, all as a single tool call. A multi-step test scenario ends up costing roughly one agent turn per meaningful check, not three or four.
Plain screenshot_ask (observe only, no action) and cheap non-vision tools (type_text, press_key, logcat_grep) are also available for the parts of a test flow that don't need this pattern.
Prefer logcat_grep over a vision call whenever what you need is already in a log line (crashes, your own debug prints, network errors) — it's free and exact, a vision call is neither. Same idea for tap/hold/swipe_hold/release: use these instead of their _and_ask counterparts for any step in a gesture you don't need to observe — e.g. release instead of release_and_ask once record_swipe_and_ask already answered what you needed mid-drag, or hold for a setup step before the leg you actually want to check. They share the same held-touch state as the _and_ask tools (see "Composing a multi-step gesture" below), so a hold → swipe_hold_and_ask → release chain works exactly like an all-_and_ask chain, just without paying for observation you don't need.
A single adb shell input swipe/draganddrop call, or a hand-rolled loop of separate adb shell input touchscreen motionevent DOWN/MOVE.../UP invocations with no timing control, is not guaranteed to register as a real drag on every engine's custom touch handler — each separate adb shell call is its own short-lived process, so the events can arrive at the input dispatcher as disconnected single-shot injections rather than one continuous touch stream. Confirmed directly: on one tested Cocos2d-x app with a custom drag handler, neither approach produced a recognized drag while a real finger did.
The fix, in src/adb.js: the same low-level motionevent DOWN/MOVE/UP primitives, driven from a single JS-side timer loop within one async function call, pacing MOVE events roughly every 20ms (tunable via MOVE_STEP_MS) against wall-clock target times. All drag-capable tools go through this (adb.drag()), not a single-shot call.
Case study — drag-and-drop test on a Cocos2d-x game: a composed touchDown → continueDrag → continueDrag → touchUp sequence (the same primitives swipe_hold_and_ask/release_and_ask use) produced a real touch-end callback on the app side 10/10 runs, confirmed via temporary debug logging added to the app's touch handlers for this investigation — including across multi-second gaps between tool calls while a touch was held. Two apparent "sometimes silently does nothing" failures turned out not to be this server's fault: (1) device-pixel coordinates computed from a displayed (scaled-down) screenshot instead of the real image dimensions, landing the touch outside any tappable element's bounds — see AGENTS.md; (2) adb logcat -d -t 500 only returning the most recent 500 lines, scrolling a debug marker out of the tail before a slower test script (with real vision-call round trips in between) could read it — use an unbounded logcat -d read instead. A third, genuine app-side animation bug was also found and isolated this way, filed with the app's own maintainers rather than fixed here (out of scope for an MCP server repo) — the touch itself was received, tracked, and released correctly throughout.
Composing a multi-step gesture across tool calls. Eight tools share one module-level "is a touch currently held, and where" variable in adb.js (a plain in-memory variable — this server is one long-lived process per session, so it doesn't need to survive a restart or be shared across sessions):
hold_and_ask / hold — press down at (x, y) and hold. Starts a new held touch; fails if one is already held.
swipe_hold_and_ask / swipe_hold — drag to (x2, y2) and hold. If a touch is already held, x1/y1 are ignored and the drag continues from wherever that touch currently is — the app sees one unbroken finger-down stream across both tool calls. If nothing is held, x1/y1 are required and this starts a fresh touch.
release_and_ask / release — sends UP wherever the currently-held touch is. Fails with a clear error if nothing is held.
record_swipe_and_ask — screenshots at independently-specified intervalMs/frameCount while a drag is still in flight (see below); optionally leaves the touch held at the end via holdAtEnd for a following swipe_hold_and_ask/swipe_hold/release_and_ask/release.
The _and_ask and quiet (no-suffix) versions of each are fully interchangeable mid-chain — they read and write the same held-touch state, so hold → swipe_hold_and_ask → swipe_hold → release_and_ask is one valid continuous gesture, not four independent taps: whichever variant you pick per step, only pay for a screenshot+vision call on the steps you actually need to observe. Calling any hold/swipe-hold variant with nothing held and no x1/y1, or any release/continuing swipe-hold variant with nothing held at all, raises a clear error rather than silently starting a wrong gesture.
record_and_askSingle-frame tools can't tell you whether something animates correctly (does an indicator pulse smoothly, does a label fly up and fade out, does a sprite spring back to its start position). record_and_ask performs one optional action (tap or swipe, or neither), waits waitMs (same meaning as waitMs in tap_and_ask/swipe_and_ask — time for the UI to start reacting before the first frame), then captures frameCount screenshots spaced intervalMs apart, and returns one short answer per frame — the calling agent gets a timeline in one tool call instead of orchestrating N separate screenshot+ask round trips itself.
Why one vision call per frame, not one call with all frames bundled together. Runware's imageCaption turns out to accept an undocumented inputImages array (plural) alongside the documented single inputImage — tested directly against the API. It works cleanly for exactly 2 images (a same-request before/after comparison came back correct and coherent). At 3+ images in one request, both that array parameter and a manually-composited side-by-side "filmstrip" image produced truncated or malformed answers in testing — the small 7B vision model apparently loses coherence past a certain combined visual+instruction load in one call. Sequential single-image calls (this tool's approach) were reliable at any frame count tested, and aren't meaningfully more expensive: cost is dominated by response length (see "Cost model" above) rather than call count, so N short sequential answers costs about the same as, or less than, one long multi-image answer. If your own provider handles multi-image requests more reliably, this is an obvious place to optimize — see "Bring your own model".
record_swipe_and_askrecord_and_ask samples frames before/after a one-shot action; it can't tell you what happened while a finger was still moving — does a dragged piece track the finger smoothly, does a drop-zone light up partway through the drag rather than only on release. record_swipe_and_ask starts a real continuous drag (the same adb.drag() implementation swipe_and_ask uses — see "Drag/hold support" above) from (x1, y1) to (x2, y2) over durationMs, and concurrently — while that drag is still in progress — captures frameCount screenshots spaced intervalMs apart, independent of the drag's own internal MOVE cadence. If frameCount * intervalMs exceeds durationMs the drag finishes early and the later frames capture the held/released end state rather than genuine mid-drag frames, so keep the two in proportion if every frame specifically needs to be mid-gesture. Pass holdAtEnd: true to leave the touch down afterward (e.g. to continue with swipe_hold_and_ask) instead of releasing.
Every screenshot a vision call actually looked at is saved to .data/screenshots/ (gitignored), named by capture timestamp (2026-08-25T18-04-31-288Z.png) so a directory listing is already in chronological order. Every *_and_ask tool's response includes the absolute path of the screenshot(s) it used — both the calling agent and the human can open that exact file directly, independent of what the vision model said about it.
This matters because of a real failure mode worth naming plainly: the vision provider's text answer is not always correct. In practice it has been observed confidently answering wrong about color, presence/absence of an element, and even returning malformed JSON for a screen that was, by direct pixel inspection of the saved file, completely different from the answer. Don't treat a *_and_ask answer as ground truth for anything you're about to act on with consequence — when an answer looks surprising or a bug report doesn't reproduce as described, open the saved screenshot file yourself (or have the calling agent read it) before trusting the text.
.data/screenshots/ is not cleaned up automatically — nothing in this server deletes old files, ages them out, or caps the directory's size. Clear it yourself (rm -rf .data/screenshots, or delete individual files) whenever a session's screenshots are no longer needed; a long QA session can accumulate a lot of them.
Vision analysis goes through src/providers/visionProvider.js, which picks a provider by name from VISION_PROVIDER in .env. Four are built in:
runware (active in .env.example, default model runware:150@2) — talks to Runware.ai's task-specific imageCaption endpoint, using AIR id runware:150@2 (LLaVA-1.6-Mistral-7B) by default — the cheapest option, ~$0.0006/short-answer call, and the recommended default (see "Switching models" below for the accuracy tradeoff against the pricier openai option). This endpoint only accepts a small, undocumented set of AIR ids — general Runware chat/vision model ids (Gemini, GPT-5.6, ...) are not valid here and return invalidCaptionModel; for those, use openai below instead.
openai (default model GPT-5.6 Terra) — despite the name, does not require an OpenAI account by default: it talks to Runware's own /v1/chat/completions endpoint (same RUNWARE_API_KEY, no separate account, reaches Runware's full chat/vision catalog under their AIR id format creator:family@version — a different model registry than the runware provider's imageCaption task above). Point OPENAI_BASE_URL at https://api.openai.com/v1 with a real OPENAI_API_KEY instead for genuinely OpenAI-hosted inference (implements the standard shape directly; not verified against that specific host, only against Runware's). ~10x the cost of the runware default per call — see "Switching models" below before reaching for this as your default.
openrouter — talks to OpenRouter's catalog under its own slug format (provider/model, e.g. qwen/qwen3-vl-30b-a3b-instruct) — a large, cheap selection including Qwen3-VL, which isn't available as a vision model through either Runware endpoint above (Runware's chat catalog only has Qwen3.5 as text-only, no vision, as of when this was checked). Not live-tested — see src/providers/openrouterProvider.js for why (no funded OpenRouter account was available to verify against) and please open a PR if you try it and it needs a fix.
openai-compatible — a generic escape hatch for anything else speaking the OpenAI chat-completions vision format (image_url content parts): a local Ollama/LM Studio server, Groq, Together.ai, or OpenRouter itself if you'd rather hand-configure it here instead of using the dedicated openrouter provider. Configure OPENAI_COMPATIBLE_BASE_URL, OPENAI_COMPATIBLE_API_KEY, OPENAI_COMPATIBLE_MODEL in .env. Most OpenAI-compatible APIs report token usage rather than a flat dollar cost; set OPENAI_COMPATIBLE_PRICE_PER_1M_INPUT/_OUTPUT if you want this provider to estimate costUsd from that (otherwise cost tracking/guardrails are inert for this provider, per the note in "Built-in spend guardrails" above).
The runware provider's models are small vision models, and small vision models get things wrong on real screenshots — not rarely. Directly reproduced against this server's own saved screenshots (see "Screenshot files" above): asked "what color is the vertical bar on the left edge?" against a screenshot with an orange fill inside a pale-blue tube, the older runware:152@2 (Qwen2.5-VL-7B) confidently answered "Black" — not a garbled response, a clean, wrong, confident one — and on a separate question it emitted a run of garbage repeated tokens instead of a real answer. runware:150@2 (the current default for that provider) did noticeably better across the same test cases and costs the same, but is still the same weight class of model — don't skip "Screenshot files" above if you haven't read it yet; a wrong-looking answer is common enough with either runware-provider model that checking the saved file yourself needs to be routine, not a last resort.
If the runware provider isn't accurate enough for what you're checking, switch to openai (bigger model, same Runware account, no new signup) or openrouter (much cheaper per token, but not verified here — see above). The same test image was compared across several models reachable through openai pointed at Runware's chat endpoint:
Model (Runware AIR id) | Answer | Cost/call (this test) | Notes |
| "Black" | ~$0.0006 | Wrong. The old default; also produced garbage repeated tokens on a separate test question. |
| "Blue" | ~$0.0006 | Correct-ish, same price as 152@2 — a different model, not a config change. |
| "White" | low | Wrong. |
| "Blue" | ~$0.006 | Correct-ish (tube is pale blue), but used ~600 completion tokens on a one-word question. |
| "Blue" | ~$0.006 | Correct-ish, ~4 completion tokens — respects the short-answer instruction, fastest (~1s). |
| "Blue" | ~$0.015 | Correct-ish, same token discipline as Terra, no accuracy edge over it in this test — costs more for no observed benefit here. |
| "Blue" | ~$0.014 | Correct-ish, but ignored the short-answer instruction entirely — ~975 reasoning tokens and 10+ seconds for a one-word question. |
None of these are infallible — a follow-up multi-question comparison against other saved screenshots from the same session had both Terra and Gemini 3.6 Flash answer wrong on 2 of 3 questions (miscounting visible icons, missing a small colored marker) despite both being "the good tier" in the table above. Practical takeaway from this project's own (small, informal) comparison: openai/Terra did read as somewhat more reliable than the runware default, but not by enough to obviously justify ~10x the cost per call — that's why runware stays the recommended default here rather than openai. This is not a claim that the two are equivalent in general, just what held in this project's own side-by-side runs — run your own comparison against your app's actual screens (the table above is a template for how) before deciding it applies to your case too. The practice in "Screenshot files" (verify against the saved file when it matters) applies regardless of which model or provider you're running, including the expensive ones.
.env.example has the runware provider active by default. The openai provider is documented (commented) with openai:gpt@5.6-terra as its own default if you enable it, google:gemini@3.6-flash and google:gemini@3.1-pro given as further commented alternatives (swap in their OPENAI_MODEL/_PRICE_PER_1M_* trio to switch), and the openrouter provider is documented (commented) as an unverified but meaningfully-cheaper-per-token alternative if you want to try Qwen3-VL. Whichever you pick, the cost guidance in "Cost model" above still applies: short, specific questions keep even a pricier model's per-call cost down, since response length (not model size or image size) is still the dominant cost driver.
To add a fully custom provider (a self-hosted model, a different API shape entirely), copy src/providers/openaiCompatibleProvider.js as a starting point, implement:
async function ask(imageBuffer, mimeType, question) {
// return { text: string, costUsd: number | null }
}
module.exports = { ask };and register it with a name in src/providers/visionProvider.js's loadProvider().
MIT
Developed by Kinect.PRO
Read the last N logcat lines, optionally filtered by a regex (e.g. your app's tag, or "Exception|FATAL"). No vision call, no cost - prefer this over screenshot_ask whenever what you need is already in a log line (crashes, your own debug prints, network errors).
| Name | Required | Description | Default |
|---|---|---|---|
| lines | No | How many recent lines to fetch (default 200). | |
| filterRegex | No | Optional regex; only matching lines are returned. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It clearly frames the operation as a read ('Read the last N logcat lines'), implying no mutation, and adds resource-related behavior ('No vision call, no cost'). It does not detail empty-result behavior or regex error handling, but for a non-destructive log reader this is sufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two compact sentences, each earning its place: the first states the core operation, the second provides selection guidance and cost context. No redundant phrases or unnecessary details.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given only two optional parameters and no output schema, the description covers the action, filtering, selection criteria, and cost trade-off. It is complete enough for an agent to invoke correctly, though it does not spell out behavior for empty results or invalid regex.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema already documents both parameters with 100% coverage, so the baseline is 3. The description adds practical regex examples ('your app's tag, Exception|FATAL') and clarifies that the filter is optional, providing contextual guidance beyond the raw schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('Read') and a clear resource ('logcat lines'), and it explicitly distinguishes itself from a sibling tool (screenshot_ask) by stating 'No vision call, no cost'. An agent can immediately tell what this tool does and how it differs.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives an explicit when-to-use rule: 'prefer this over screenshot_ask whenever what you need is already in a log line,' followed by concrete examples (crashes, debug prints, network errors). It also explains the cost advantage, making the selection decision clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Long-press at (x, y) for durationMs, wait briefly, take a screenshot, and ask a short question about the result.
| Name | Required | Description | Default |
|---|---|---|---|
| x | Yes | ||
| y | Yes | ||
| waitMs | No | Milliseconds to wait after releasing before screenshotting (default 500). | |
| question | Yes | ||
| durationMs | No | Hold duration in ms (default 800). |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the burden of disclosing behavior. It transparently lists the operation sequence and references default waitMs/durationMs defaults in the schema. However, it does not disclose side effects of long-pressing (e.g., opening context menus or triggering navigation), what the 'ask' returns or to whom, or any required permissions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single sentence that front-loads the core action and includes the full workflow without filler. Every element earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With 5 parameters, no annotations, and no output schema, the description leaves important gaps: the return/response behavior is ambiguous ('ask a short question about the result'), coordinate system is unspecified, and side effects are not mentioned. An agent would need additional implicit knowledge to call this tool confidently.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema descriptions only cover waitMs and durationMs (40% coverage). The description helps by framing x and y as long-press coordinates and question as a short question about the result. Still, it does not specify coordinate units/origin or any constraints on the question, so it only partially compensates for the schema gaps.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific action sequence: long-press at (x, y) for durationMs, wait, screenshot, and ask a question. The long-press gesture clearly differentiates it from sibling tools like tap_and_ask, swipe_and_ask, and screenshot_ask, even without naming them explicitly.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies the use case: perform a long-press and inspect the resulting screen via a screenshot and question. However, it does not explicitly state when to prefer this over tap_and_ask, swipe_and_ask, or other siblings, nor does it mention any exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Send an Android keyevent code (e.g. 4 = BACK, 66 = ENTER, 187 = APP_SWITCH). No vision call.
| Name | Required | Description | Default |
|---|---|---|---|
| keycode | Yes | Android KEYCODE_* integer value. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are present, so the description is the only source of behavior. It discloses the core action (sending a keycode) and that it does not use vision, but it does not clarify whether the key is pressed and released with a single event or describe timing/duration. Lacks details on side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
One compact sentence with two clauses; the key action is front-loaded, and each part (action, examples, vision exclusion) adds value without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Simple tool with one required parameter and no output schema. The description covers what it does and gives examples, so an agent can invoke it correctly. It does not explain return behavior or errors, but those are likely unnecessary for a fire-and-forget key event. Minor missing context about when to use it is covered under usage guidance.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema already documents keycode as an Android KEYCODE_* integer. The description adds specific example values (4=BACK, 66=ENTER, 187=APP_SWITCH), which clarify the range and meaning significantly beyond the schema's generic description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States the specific verb 'send' and resource 'Android keyevent code', gives concrete examples distinguishing it from vision-based siblings like screenshot_ask and tap_and_ask, and explicitly notes 'No vision call.'
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides minimal guidance on when to use; the 'No vision call' implies it is not for visual tasks, but it does not explicitly name alternatives or conditions for selection. The examples imply use for system keys but lack explicit routing.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
For checking an ANIMATION or any effect that plays out over time (e.g. "does the strength indicator pulse smoothly?", "does the XP label fly up and fade out?", "does the sprite return to its start position?"). Optionally performs one action first (tap or swipe, or neither), then takes frameCount screenshots spaced intervalMs apart, and asks the SAME short question about each frame separately (each frame gets its own vision call, with its frame number in the prompt) - returns one answer per frame in order.
Sequential single-frame calls were chosen over sending several frames in one request: Runware's imageCaption does accept an undocumented multi-image array, and it works fine for exactly 2 frames, but degrades noticeably at 3+ (truncated/malformed answers in testing) - sequential calls are both more reliable and, per-frame, no more expensive. Keep frameCount modest (3-6) - each frame is a full separate vision call and cost scales linearly with it.
| Name | Required | Description | Default |
|---|---|---|---|
| x | No | Required for action=tap or action=swipe (swipe start x). | |
| y | No | Required for action=tap or action=swipe (swipe start y). | |
| x2 | No | Required for action=swipe (end x). | |
| y2 | No | Required for action=swipe (end y). | |
| action | Yes | Action to perform before starting the capture sequence. | |
| waitMs | No | Milliseconds to wait after the action before the FIRST screenshot (default 500) - same meaning as waitMs in tap_and_ask/swipe_and_ask, separate from intervalMs which spaces out the frames after that. | |
| question | Yes | The same short question asked about every captured frame (e.g. "Is the indicator visible? yes/no"). | |
| frameCount | Yes | How many screenshots to take, spaced intervalMs apart (2-8; keep modest, see description). | |
| intervalMs | Yes | Milliseconds between each screenshot (i.e. the sampling interval of the sequence). |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden, and it delivers: it reveals that each frame gets its own vision call with the frame number in the prompt, that answers come back one per frame in order, and that sequential calls were deliberately chosen over multi-image requests due to reliability degradation. This gives the agent accurate expectations about cost and behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is longer than average, but every sentence contributes: purpose, action semantics, per-frame behavior, return ordering, rationale for sequential calls, and cost guidance. The key use case is front-loaded, and the engineering rationale is placed where it helps the agent decide rather than adding noise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 9 parameters, no output schema, and no annotations, the description covers the core interaction fully: what triggers the sequence, what each frame does, how the question is applied, what the result order is, and cost implications. The remaining parameter details are already well documented in the input schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3. The description adds extra value by advising frameCount stay modest (3-6), explaining linear cost scaling, and clarifying that waitMs is distinct from intervalMs and has the same meaning as in tap_and_ask/swipe_and_ask. This goes beyond the schema's structural descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific use case—checking an animation or effect that plays out over time—and clearly states the mechanism: perform an optional action, take frameCount screenshots spaced intervalMs apart, and ask the same question about each frame. This distinguishes it from single-shot siblings like screenshot_ask or tap_and_ask without needing to open their schemas.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It explicitly says when to use the tool: for animations or time-based effects. It also explains when the optional action is tap, swipe, or neither. However, it does not explicitly name alternative tools or state when NOT to use this one, so the guidance is clear but not fully contrastive.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Take a screenshot of the current screen and ask a short question about it (e.g. "Is there an error dialog visible?", "How many word icons are on screen?", "What color is the strength indicator?"). Use this when you need to check state WITHOUT performing an action first. Phrase the question so a short answer is possible (yes/no, a number, a short label) - see this server's README "Cost model".
| Name | Required | Description | Default |
|---|---|---|---|
| question | Yes | A short, specific question about the current screen. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the burden of behavioral disclosure. It clearly communicates a non-action read-only behavior and implies the response is short ('so a short answer is possible'). It points to the README for cost details, adding context. While it doesn't describe the exact return format, the answer is implied by the question-asking purpose.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise (about 3 sentences) and front-loaded with the main action. Every sentence earns its place: statement of action, examples, usage guidance, and cost reference. No redundancy or filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a single-parameter, read-only tool without an output schema, the description is largely complete. It covers purpose, usage timing, question phrasing, and cost considerations. It could explicitly mention that the result is an answer to the question, but that is reasonably implied. The description is sufficient for an agent to call it correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% and the parameter description already clarifies the question. The tool description adds value by providing examples of valid questions and guidance on phrasing for short answers, which enriches the parameter's semantics beyond the schema alone.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb-resource pair ('Take a screenshot of the current screen and ask a short question about it') and provides clear examples. It also distinguishes itself from siblings by specifying 'WITHOUT performing an action first', which separates it from action-based tools like tap_and_ask or swipe_and_ask.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It explicitly states when to use this tool ('when you need to check state WITHOUT performing an action first') and gives concrete guidance on phrasing questions for short answers. The reference to the README 'Cost model' provides additional usage context. This fully addresses when to use instead of alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Swipe (or drag, for drag-and-drop UIs) from (x1, y1) to (x2, y2), wait briefly, take a screenshot, and ask a short question about the result - all in one call.
| Name | Required | Description | Default |
|---|---|---|---|
| x1 | Yes | ||
| x2 | Yes | ||
| y1 | Yes | ||
| y2 | Yes | ||
| waitMs | No | Milliseconds to wait after the swipe before screenshotting (default 500). | |
| question | Yes | A short, specific question about the screen after the swipe. | |
| durationMs | No | Swipe duration in ms (default 300; use longer for drag-and-drop hold gestures). |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the transparency burden and does disclose the full ordered behavior: swipe, wait, screenshot, ask. It also notes the duration nuance for drag-and-drop holds. It doesn't detail coordinate units or what 'ask' returns, but the step sequence is clearly communicated.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
One sentence captures the entire workflow with no filler. The core action is front-loaded, and the drag-and-drop nuance is efficiently folded into the gesture description.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description is adequate for a simple composite gesture tool, but given no output schema and no annotations, it omits the return/answer semantics of 'ask', the coordinate system, and any cost or side-effect implications. These gaps prevent it from being fully self-sufficient.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is only 43%, so the description must compensate. It explains x1/y1/x2/y2 as the swipe's start and end points, and clarifies that question should be short and about the post-swipe screen. The optional waitMs and durationMs already have schema descriptions, and the prose adds the 'hold for drag-and-drop' nuance.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb+resource: swipe/drag from one coordinate to another, wait, screenshot, and ask a question. It clearly distinguishes itself from sibling tools like tap_and_ask and screenshot_ask by naming the gesture ('Swipe') and the compound workflow.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The instruction to use drag 'for drag-and-drop UIs' gives some contextual guidance, but there is no explicit when-to-use/when-not-to-use statement or reference to alternatives. The appropriate context is implied by the swipe gesture rather than directly contrasted with siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tap at device screen coordinates (x, y), wait briefly for the UI to react, take a screenshot, and ask a short question about the result - all in one call. Use this for any "tap here, then check what happened" step instead of calling separate tap/screenshot/ask tools.
| Name | Required | Description | Default |
|---|---|---|---|
| x | Yes | X coordinate in device pixels. | |
| y | Yes | Y coordinate in device pixels. | |
| waitMs | No | Milliseconds to wait after the tap before screenshotting (default 500). | |
| question | Yes | A short, specific question about the screen after the tap. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. It describes the sequence of actions (tap, wait, screenshot, ask) and mentions a wait period before screenshotting. However, it does not disclose what happens if the tap fails, the exact return format (e.g., does it return an image, a text answer, or both?), or any side effects like requiring a running app. The description is adequate for basic behavior but lacks depth on error handling or output, which is significant for a composite tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences with no redundancy. The first sentence front-loads the action sequence, and the second sentence provides direct usage guidance. Every word serves a purpose, and the structure is clean and immediately understandable. It avoids jargon and is well-scoped.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a composite tool with 4 parameters and no output schema, the description should cover the response format and any prerequisites. While it clearly states the purpose and usage, it does not mention what the tool returns (e.g., an answer to the question, a screenshot reference) or any necessary preconditions (e.g., the device being interactive). Since there is no output schema, the description's silence on return values leaves an agent without complete information for correctly interpreting the tool's outcome. This is a notable gap, so a 3.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3. The description adds minimal semantic value beyond the schema: it refers to 'wait briefly' which maps to waitMs, and characterizes the question as 'short and specific', but does not explain coordinate units or default wait behavior beyond what the schema provides. The description does not go beyond the schema definitions, so a 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's verb and resource: 'Tap at device screen coordinates (x, y), wait briefly for the UI to react, take a screenshot, and ask a short question about the result - all in one call.' It distinguishes itself from the siblings by defining its specific action (tap) and explicitly contrasting with calling separate tap/screenshot/ask tools. The title also reinforces the composite nature, so there is no ambiguity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit when-to-use guidance: 'Use this for any "tap here, then check what happened" step instead of calling separate tap/screenshot/ask tools.' This clearly defines the usage context and names an alternative (separate tools). It does not explicitly mention sibling tools like swipe_and_ask or long_press_and_ask, but the reference to 'tap' inherently implies a distinction from those. This is strong guidance, but not exhaustive about exclusions from all siblings, hence a 4.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Type text into whatever field currently has focus. No screenshot/vision call - pair with screenshot_ask if you need to confirm the result.
| Name | Required | Description | Default |
|---|---|---|---|
| text | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the behavioral burden. It discloses that typing targets the focused field and that screenshot/vision is not part of the operation, but it does not cover edge cases such as no focused field, whether existing text is replaced, or how special characters are handled.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two tight sentences: the core behavior is front-loaded, and the follow-up guidance about screenshot_ask earns its place. No filler or redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a one-parameter tool, the description is nearly sufficient: it states the target, the action, and the verification route. Missing failure-mode detail, such as what happens when no field has focus, keeps it from a 5.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description does not add meaning beyond the property name 'text'. The parameter is simple, but nothing explains format, newline behavior, limits, or encoding, so the description fails to compensate for the schema's lack of documentation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a concrete action ('Type text') and a specific target ('whatever field currently has focus'), and explicitly warns against treating it as a screenshot/vision operation. This clearly differentiates it from screenshot_ask and the other _and_ask siblings.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives clear operational guidance: use it when a field has focus, and pair it with screenshot_ask when confirmation is needed. It does not explicitly contrast with press_key or other input tools, but the focus-based behavior is enough to guide selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Report the cumulative vision-provider spend for today and the configured alert/cap thresholds, without making any device or vision call.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It openly states 'without making any device or vision call', which discloses side-effect-free behavior, but it does not describe the output format, potential delays, or any other behavioral aspects. This is a reasonable disclosure but not comprehensive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, well-structured sentence that front-loads the verb and resource. Every word adds value, with no redundancy or filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a zero-parameter, no-output-schema tool, the description covers the essential information: what is reported and the guarantee of no side effects. It does not specify the return format or any prerequisites, but given the simplicity, it is sufficiently complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are zero parameters, so the baseline is 4. The description adds meaning about what the report contains (spend and thresholds) beyond the empty schema, which is exactly what is needed for a parameterless tool.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses the specific verb 'Report' and identifies the exact resource ('cumulative vision-provider spend for today') plus the alert/cap thresholds. It is clearly distinct from the sibling action-oriented tools (screenshot, tap, etc.) by stating it makes no device or vision call.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies the tool is for checking spend information and explicitly notes it does not make any device or vision call, but it does not name alternative tools or provide explicit when-to-use guidance. The use case is somewhat obvious given the sibling list, but the guidance is not explicit enough for a higher score.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
v0.1.0logcat_greplong_press_and_askpress_keyrecord_and_askscreenshot_askswipe_and_asktap_and_asktype_textvision_spend_reportScored across 9 tools
Each tool has a clearly distinct purpose: screenshot_ask is passive state-checking, while tap/swipe/long_press_and_ask each combine a specific gesture with screenshot-and-ask. record_and_ask targets animations, type_text and press_key are direct input without vision, logcat_grep handles logs, and vision_spend_report tracks cost. No two tools overlap in function.
All tool names follow a consistent snake_case pattern, with a clear <action>_and_ask convention for vision-verifying interactions and simple verb_noun for the rest. The naming logically separates gesture tools from non-vision tools, making the set easy to navigate.
Nine tools is well-scoped for a mobile automation/verification server. Each tool addresses a concrete need—actions, verification, logging, cost monitoring—and none feel redundant or purely decorative. The count fits the domain without bloat or sparsity.
The tool surface covers the full cycle of mobile UI interaction and verification: direct input (type_text, press_key), gestures (tap/swipe/long_press), visual state checking (screenshot_ask, record_and_ask), log inspection (logcat_grep), and cost governance (vision_spend_report). No obvious dead ends or missing operations for the stated purpose.
Control real Android and iOS devices with LLM agents — tap, swipe, type, automate flows.
Drive real devices from your AI Coding tool. Embed a client SDK (Unity, Godot, Flutter, iOS/macOS, Android, React Native, Web) in your app, then capture screenshots, traverse the UI tree, inject taps and key events, and run automated test tasks on the physical device over a secure relay.
Melaya is a remote MCP server. It gives an assistant hands on your own Android phone and browser: it reads the screen through the accessibility tree, then taps, types and navigates inside the apps and sites you allow-list, with no per-app API. It also builds, schedules and runs agent pipelines across 6k+ connected tools. OAuth 2.1, nothing to install.
AI visual generation agent: multi-pipeline rendering, prompt crafting, and image composition.
