Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type @ followed by the MCP server name and your instructions, e.g., "@twiddle-mcp tap the login button and wait for the dashboard to load"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
Stop your agent twiddling its thumbs. — a Twiddle Thumb Studio tool
Blocking wait_for primitives and semantic UI diffs for the iOS Simulator and Android Emulator. An MCP server that lets coding agents (Claude Code, Cursor, any MCP client) drive and — critically — verify mobile apps without screenshot → sleep → screenshot polling.
Web agents already have this: Playwright MCP's accessibility snapshots and browser_wait_for. On mobile, waiting exists only in fragments — Maestro's flow engine has implicit waits, SilbercueSwift waits on log patterns (iOS only), Appium-based stacks can script visibility waits — but no cross-platform MCP server makes blocking tree-based waits and semantic diffs the primitives agents call directly. This fills that gap.
agent: tap(ref=e4) → "Tapped button 'Load Data'"
agent: wait_for({text: "Welcome back"}) → blocks…
→ MATCHED in 1452ms:
~ text #statusLabel "Loading…" → "Welcome back, Tyler"One tool call. No sleeps, no screenshot diffing, no wasted tokens.
// .mcp.json
{
"mcpServers": {
"twiddle": { "command": "npx", "args": ["-y", "@twiddlethumb/twiddle-mcp"] }
}
}Requirements: macOS with Xcode (iOS) and/or the Android SDK with adb on PATH. First iOS use compiles WebDriverAgent (~30s–3min, then cached). Android bootstraps in ~3s.
Related MCP server: Mobile Device MCP
Verify primitives (the point of this project):
Tool | What it does |
| Block until text/element appears (or |
| Block until the UI stops changing. Volatile elements (clocks, spinners) are auto-detected and ignored. |
| Accessibility tree as text with |
| "What changed since snapshot N?" as short sentences. |
| Accept/dismiss system & in-app dialogs from the tree — handles entrance-animation frames and the tap-jacking cooldown that silently swallows an agent's too-fast taps, with verify-and-retry. |
Driver: list_devices · launch_app · terminate_app · tap (by ref / selector / x,y — refs re-resolve against the live tree; a vanished element returns the current tree instead of a blind tap; scroll:true swipes through content until the selector appears, settle-aware so it never taps a still-moving list) · type_text · swipe · press_button · screenshot (fallback for purely visual checks).
Both platforms expose a localhost HTTP automation server — WebDriverAgent on iOS (bootstrapped from the appium-webdriveragent npm package), appium-uiautomator2-server on Android (APKs installed over adb). One keep-alive connection pool and one shared poll loop per device (~7Hz) serve any number of concurrent waits. Trees are normalized to a platform-agnostic node model; elements get stable refs via identity keys (role+identifier first), so refs survive re-renders; diffs align by those keys and suppress frame jitter and volatile elements.
pnpm install
pnpm test # unit tests (tree engine, wait logic) — no devices needed
pnpm build # dist/index.js
examples/ios-demo/build.sh # build + install the SwiftUI demo on the booted simMeasured against mobile-mcp, Maestro MCP, and the raw screenshot baseline on both platforms (tap → verify a 2s async load; medians over repeated runs, canonical agent patterns, full method in bench/RESULTS.md):
agent turns | context/verification | est. total (Android / iOS) | |
twiddle-mcp | 2 | 1 KB | 11s / 9s |
mobile-mcp | 3–4 (1 poll/turn) | 3 KB | 16s / 14s* |
maestro-mcp | 1 (flow YAML) | 0.3 KB | 10s / 6s |
screenshots + shell (default) | 3 | 117 KB | 12s / ✗ (simctl cannot tap) |
* iOS requires a workaround for mobile-mcp's fractional-coordinate click bug (see RESULTS.md).
On failure (wait_for timeout), the agent gets the full current UI tree to reason over; a failed Maestro flow returns a 288-byte error string, and polling patterns return their last snapshot. Every extra second of app latency costs polling tools another turn; it costs wait_for nothing.
Early but working: iOS + Android verified end-to-end over real MCP stdio (see bench/ and examples/). Roadmap: event-driven AccessibilityService transport on Android, iOS benchmark, agent-level benchmark via headless Claude Code sessions (bench/run.mjs).
MIT
Get the current accessibility tree as structured text with [ref=eN] markers usable in tap/type_text. Much cheaper and more precise than a screenshot.
Inputs: device, filter.
Accept or dismiss a blocking alert/dialog (system permission prompts, 'Open in …?' sheets, in-app modals). Finds the right button from the current UI tree — no coordinates needed. Use buttonText for a specific button. Returns what was tapped and what changed; if no alert is present you get the current tree.
Inputs: action, device, buttonText.
Launch an app by iOS bundle id or Android package name.
Inputs: appId (required), device, relaunch.
List iOS simulators and Android emulators/devices. Booted devices first; use the returned id with every other tool.
Inputs: no named parameters.
Press a device button (home, back [Android], volumeUp, volumeDown, enter).
Inputs: button (required), device.
PNG screenshot — the fallback for purely visual checks (colors, images, layout polish). For text/structure, prefer get_ui_tree or wait_for: cheaper and more precise.
Inputs: device.
Swipe in a direction (from screen center) or between explicit points.
Inputs: to, from, device, direction, +1 more.
Tap an element by [ref=eN] from get_ui_tree/wait_for, by selector, or by x/y. Refs re-resolve against the live tree; if the element is gone you get the current tree instead of a blind tap. With scroll:true, a selector that isn't on screen is searched for by scrolling through the content first. Returns a mini-diff of what the tap changed.
Inputs: x, y, ref, device, +2 more.
Terminate a running app.
Inputs: appId (required), device.
Type text into the focused field (optionally tap a field first via ref/selector). submit presses Enter/Return.
Inputs: text (required), ref, device, submit, +1 more.
Semantic diff of the current UI vs a previous snapshot (default: the last one this server took). Answers 'what changed?' without comparing screenshots.
Inputs: device, sinceSnapshotId.
BLOCK until text or an element appears (or disappears with state:'gone') — use this instead of screenshot+sleep loops. Returns immediately when the condition is met, with elapsed time and a semantic diff of what changed. On timeout it returns matched:false plus the current UI tree (never throws) so you can see what IS on screen.
Inputs: text, state, device, selector, +1 more.
BLOCK until the UI stops changing (animations/loads settled). Volatile elements like clocks and spinners marked noisy are ignored. Use after navigation before reading the screen.
Inputs: device, quietMs, timeoutMs.
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
Scored across 13 tools
Each tool has a distinct purpose: tapping, typing, swiping, waiting, etc. No two tools overlap in functionality.
All tool names follow a consistent verb_noun snake_case pattern (e.g., wait_for, get_ui_tree). No mixing of conventions.
13 tools is well-scoped for mobile UI testing, covering all necessary actions without excess.
Covers core workflows like launching, interacting, waiting, and inspecting UI. Minor gaps like missing explicit scroll or long press, but swipe can compensate.
Cloud iOS simulators and Android emulators your agent can create, drive, and throw away.
Disposable cloud Android emulators for coding agents: run an APK or PR build, tap, type, screenshot.
Phones for your agents: each agent gets its own cloud Android phone over MCP, reads the screen, taps, types and installs your APK. Ready in ~20 s, $0.01/min, parked phones free.
MCP server for Mint — AI-powered QA that runs your app in a real browser on every PR.
