react-native-eye

react-native-eye connects Claude Code, Cursor or any other MCP client to your app through Metro's Chrome DevTools Protocol endpoint. The agent reads the screen, taps and types, and checks logs, network and state. The app needs no SDK and no code changes.

It is built for agent speed. Composite tools run a whole interaction in one call, so the agent does not pay one model round trip per tap.

Contents

• Quick start

• Requirements

• What the agent can do

• A whole flow in one call

• How it works

• Tool reference

• Configuration

• Native driver

• Privacy

• Limitations

• Development

• License

Related MCP server: React Native MCP Server

Quick start

1. Add the server to your MCP client.

Claude Code, from your project folder:

claude mcp add react-native-eye -- npx -y react-native-eye@latest

Cursor (.cursor/mcp.json), Claude Desktop and any other client that takes the standard config:

{
  "mcpServers": {
    "react-native-eye": {
      "command": "npx",
      "args": ["-y", "react-native-eye@latest"]
    }
  }
}

Codex (~/.codex/config.toml):

[mcp_servers.react-native-eye]
command = "npx"
args = ["-y", "react-native-eye@latest"]

2. Start Metro and a dev build. Run the app in an iOS simulator or an Android emulator the way you usually do. Metro on the standard ports (8081–8083, 19000–19002) is found automatically. For any other port, pass --port 8090 in args, set METRO_PORT, or give the port to the connect tool.

3. Ask the agent.

Connect to my app and tell me what is on the screen.

The agent calls connect, then observe. The first call on each device installs the native driver once.

Requirements

The JS paths (layout from the runtime, via: "js" taps and typing) need only a connected app.

What the agent can do

35 tools in five groups. Tools in bold are composite: one call instead of many. Prefer them.

The composite tools:

A whole flow in one call

{ "steps": [
  { "type": { "text": "a@b.c", "into": { "placeholder": "Email" } } },
  { "type": { "text": "hunter2", "into": { "placeholder": "Password" } } },
  { "tap": { "text": "Log in", "via": "os" } },
  { "wait_for": { "text": "Home", "timeout_ms": 3000 } },
  { "expect_not": { "text": "Error" } }
] }

Each step takes exactly one action (settle is a modifier) — a step with two actions is a validation error, not a silently skipped assertion. Steps run back-to-back with adaptive settling (two rendered frames, capped at 1s; override per step with "settle": 200). A redbox or uncaught exception anywhere in the flow aborts it immediately with full context, and steps that never ran are listed as such.

tap and type steps take the same selectors, via and into as the standalone tools — into accepts {testID}, {placeholder} or {label}, and a bare string is still a testID — and each step's result names the path it used. A step that asks for via: "os" gets a real touch even when a JS press was available: use it for anything driven by the responder lifecycle, and leave the default auto everywhere else.

How it works

The server reaches the app two ways and picks the cheaper one for each call.

Through the JS runtime. The server walks React's fiber tree over CDP. It measures elements and calls press and text handlers directly, with no touch involved. A press takes about a millisecond and a full-screen layout about 80 ms.

Through the device. A prebuilt driver ships in the npm package and installs itself on first use: an XCUITest runner on iOS, an instrumentation server on Android. It delivers real touches for gestures, system dialogs and native views, and types any Unicode text on Android. If it cannot start, input falls back to axe or adb.

No silent switching. Every response says which path ran: (input: js), (input: os/runner), (layout: js), (layout: os), with the reason when a path was skipped.

Measured in the project's own field tests against a busy video feed:

Tool reference

Three parameters appear on many tools.

• device — on every device-facing tool (tap, swipe, scroll_to, type_text, press_key, screenshot, get_layout, read_screen_text, observe, run_flow, launch_app, install_app, open_url, set_permission, record_video): a udid or device-name substring. Without it the device is taken from the connected app's target, and two booted devices with no hint is an error rather than a silent guess.

• via — on tap, type_text, scroll_to, swipe, press_key and the tap/type steps of run_flow: "js" | "os" | "auto", default auto.

  • js goes through the app's own fiber tree — the element under the resolved point is found, its nearest onPress/onChangeText is called directly, and the whole round trip is about a millisecond.

  • os delivers a real touch or HID keystroke through the simulator or device.

  • auto tries js and falls back to os, naming the reason.

  • Every response says which path ran — (input: js), or (input: os/axe — no press handler in ancestry) (os/runner, os/axe or os/adb names the OS mechanism that delivered it) — mirroring (layout: js)/(layout: os).

  • Selector, ambiguity and visibility rules are identical on both paths: the JS path replaces delivery, not judgement, so an off-screen or hidden element is refused either way.

• connect — port, target_id (from list_targets), or device.

• tap — duration_ms for a long press (also available on run_flow tap steps). On the JS path a duration_ms of 500 or more calls the element's onLongPress; an element that has none falls back to os, which is what the response then says.

• swipe — direction or x1,y1,x2,y2, duration_ms, via. The JS path scrolls the container under the start point by the gesture's own vector (FlatList.scrollToOffset 0 → 640), so a feed scrolls in a millisecond instead of a 20 s touch; no scroll container under the point falls back to os with the reason.

• scroll_to — {text | testID}, plus via and max_swipes (default 10): bring an element into view. The JS path finds the element's nearest scrollable ancestor and scrolls that container (ScrollView, FlatList/VirtualizedList, FlashList, @legendapp/list; an unrecognised container falls back to os). The OS path swipes toward the element, re-reading the layout after each swipe and stopping the moment it is visible. This is the missing half of tap's "element is off-screen — scroll to it first".

• type_text — into chooses the field: {"testID": "…"}, {"placeholder": "…"} or {"label": "…"} (accessibilityLabel); a bare string is still read as a testID. Omitted, it types into the one TextInput on the focused, topmost screen and refuses — listing the candidates by placeholder and position — when there is more than one. Two instances of the same screen in a stack (each with the same composer) are told apart by react-navigation / react-native-screens state: the field on the focused screen wins and the response says so; a match only on a background screen is typed into with a warning. submit: true fires the field's submit handlers after typing (see press_key enter). verify (default true) reads the field back after typing and fails loudly, naming both strings, when what arrived is not what was sent. via as above; the JS path calls onChangeText, so the app's real pipeline (debounce, request, re-render) runs.

• press_key — key: back | home | enter, via, into. back on the JS path calls the react-navigation container's goBack(); an empty stack is reported as Did not go back: nothing to go back to and deliberately does not fall through to the hardware key (on Android that would background the app). enter fires the field's own onSubmitEditing → onEndEditing → onBlur (JS only; into picks the field, otherwise the focused one).

• navigate — action: stack | navigate | push | back | pop | pop_to_top | reset, with name/params (navigate, push), count (pop), state (reset). The react-navigation container is found by walking the React tree for its ref context (react-navigation 6/7, Expo Router), cached on globalThis.__RNE_NAV__; no app change needed. stack prints the focused branch level by level; every mutating action returns the new stack, and an action no navigator handled is an error carrying react-navigation's own message.

• run_flow — observe controls the trailing snapshot: auto (default) attaches a screenshot and adds the full layout only when the flow failed, full always, none never.

• get_layout — search (terms matched against text/testID), type, match: substring|exact, case_sensitive, or at: {x, y} to identify the element at a point.

• read_screen_text — on-host OCR of the current screen (Apple Vision, nothing leaves the machine): one line per text run as <text> @x,y, coordinates in tap units usable directly with tap {x,y}. search (substring filters), languages (e.g. ["de","en"]; auto-detected by default), compare_tree (default true — lines that also exist in the JS layout are marked [tree]; prefer tap {text} for those). Sees text the layout tree cannot: images, canvases, WebViews, native dialogs.

• screenshot — output_path saves to a file (format: png|jpeg, mask on iOS) and returns the path instead of an inline image.

• get_component_tree — max_depth. The budget counts only components a developer actually authored: host components (RCTView, RCTText, ...) and structural wrappers (forwardRef/memo/context, styling HOCs like NativeWind's cssInterop) are free and can repeat any number of times without spending depth. RN's own dev-only infrastructure (LogBox, the debugging overlay, the root tag context) is dropped from the output entirely, not just made free. A very large tree is capped and marked truncated rather than returned unbounded.

• get_state — path (dot-path) and depth; the value is summarised inside the app, so the result is always valid JSON. found says whether a store exists at all; resolved says whether the path walked all the way to a value — a mistyped segment reports resolved:false with failedAt and availableKeys instead of looking identical to an honest undefined.

• eval_js — timeout_ms for a legitimately slow promise (default 10s). background: true starts the evaluation as a job and returns its id at once, for a measurement that outlives a tool call or a JS thread that is busy right now; the CDP message is queued by Hermes and runs when the thread frees. label names the job.

• get_job — id returns a background job's status and value (or error); without id, lists jobs. profile_renders also runs as a job with background: true, so a profile can be driven with tap/swipe while it collects.

• get_network — search (URL substring to keep), exclude (URL substrings to drop), include_dev (show dev-tooling noise: Metro/devtools ports and /symbolicate, /logs, /status on local hosts are hidden by default — the footer says how many). Query strings are cut short in the list; detail: <seq> has the full URL and bodies.

All four work on a dev build with zero app setup.

• profile_renders — duration_ms (default 5s), top, background. Needs a dev build (the React DevTools hook is what it wraps).

• profile_js — duration_ms (default 5s), top, sampling_interval_us (Hermes may ignore it — the output says so), save (default true, to $RNE_OUTPUT_DIR or ~/Downloads).

• sample_cpu — duration_s (default 3), app_id when the connected target does not name the app.

• get_fps — duration_ms (default 2s), app_id. On iOS only the JS-thread number is available without native code: UI-thread frames need a physical device and xctrace's Core Animation instrument.

• launch_app — terminate_running restarts a running app; env passes launch environment variables (iOS only).

• open_url — opens a deeplink or universal link (simctl openurl on iOS, am start -a android.intent.action.VIEW on Android). No touch is involved, so on iOS it is by a wide margin the cheapest way to reach a screen — see the touch cost in Limitations.

• set_permission — {service, status: "grant" | "revoke" | "reset", app_id?} (simctl privacy on iOS, pm grant/pm revoke on Android). Granting before launch is the reliable way to keep a permission alert off the screen; dismissing one costs a touch, and on iOS some services only pick up the change after the app is terminated — the response says so rather than failing quietly.

• record_video / stop_recording — screen recording; defaults to $RNE_OUTPUT_DIR or ~/Downloads. Android clips cap at 3 minutes (screenrecord limit).

• ping — no arguments, returns pong. It answers without touching Metro, the app or a device, which is exactly what makes it useful: it separates "the MCP server is not running" from "the app is not connected".

Configuration

Native driver

For faster, more reliable OS input the package ships prebuilt on-device drivers; there is nothing to build (scripts/build-drivers.sh is only for contributors). On first use it installs, per device:

If a driver cannot be installed or started, input and layout fall back to axe/adb automatically, and the response names the path that ran. RNE_NATIVE_DRIVER=off opts out entirely.

While the Android server is up it holds UiAutomation, so other UiAutomation clients on that device (Appium, uiautomator dump) fail until it is stopped (adb shell am force-stop dev.rneye.server, or RNE_NATIVE_DRIVER=off).

Privacy

No telemetry. No accounts. No cloud. The only network traffic is to localhost (Metro) and local device drivers. This is enforced by an automated test that fails the build if any external URL appears in the source, and another that fails if a license/account/feedback tool ever shows up in the tool surface.

Limitations

The short version: dev builds only, and the JS path calls handlers rather than performing gestures.

Development

npm install
npm run build        # compile to build/
npm test             # unit tests
npm run dev          # run the server from source

• npm run test:integration runs the device tests. They need a booted iOS simulator (its udid in RNE_INTEGRATION_UDID) or an Android emulator (its serial in RNE_INTEGRATION_ANDROID_SERIAL).

• scripts/build-drivers.sh rebuilds the native drivers under native/ (Xcode for iOS; JDK 17+ and the Android SDK for Android). Any change under native/ changes the driver version.

• scripts/pack-test.sh packs the tarball, installs it into a fresh project and checks that the drivers arrive intact and the server answers.

Pull requests are welcome: see CONTRIBUTING.md. Report security issues privately as described in SECURITY.md.

License

MIT for react-native-eye itself. The native drivers include third-party components under Apache-2.0, BSD and MIT licenses. Attributions are in NOTICE, the Apache-2.0 text is in native/LICENSE-APACHE, and per-driver details are in native/ios-runner/THIRD_PARTY.md and native/android-server/THIRD_PARTY.md.