Fetch the on-screen component tree of a React Native app to locate UI elements and obtain normalized tap coordinates for automated interaction.
Apache 2.0
678,539 tools. Updated 2026-10-19 21:12
"Server for MCP Protocol with On-Screen UI Interaction" matching MCP tools:
- Enumerate the live GUI tree under a root Instance to discover on-screen UI elements, their paths, classes, and text for targeted interaction.MIT
- Interact with any on-screen UI element by supplying its index from get_elements(). Use it to tap buttons, links, or fields after listing available elements.MIT
- Get a JSON array of all UI elements on the current screen, including text, bounds, and clickability, to understand the layout before taking action.MIT
- Define a consistent communication protocol template to prevent AI from using different protocols for different game features, reducing bugs in multiplayer and interaction systems.MIT
- Get the current Windows UI element tree as XML to verify on-screen content, detect UI changes, and find element names or AutomationIds for automation.Apache 2.0
Matching MCP Servers
- FlicenseAqualityDmaintenanceEnables LLMs to capture screenshots and screen recordings through MCP with chunked session-based transfers for reliable image consumption. Supports multi-monitor selection, timeline capture, and compatibility with both vision and non-vision language models.111-
- AlicenseAqualityDmaintenanceAn MCP server that enables AI agents like Claude to capture screen screenshots on demand and maintain a sliding window of recent frames for multimodal Q&A context.61MIT
Matching MCP Connectors
Review supplied evidence of actor identity, principal authorization, tool scope and intended action before an agent acts. Returns missing evidence and review guidance; enforcement belongs to the caller integration. Two MCP-native free trial attempts per network for the campaign, shared daily capacity. No action authorization or identity certification.
Mlola UI components, tokens and design rules for coding agents, with a markup checker
- Retrieve Android screen XML hierarchy to locate elements, read text, or find resource IDs for UI interaction.MIT
- Detects UI changes on Android devices after user actions by comparing screen states. Use to verify interactions, wait for transitions, and confirm expected screen updates during automated testing.MIT
- Get the current screen state: interactive UI elements with label indices, types, names, tap coordinates, and device context. Use vision for annotated screenshot. Call first before interacting.MIT
- Observe the current screen and UI state—geometry, active window, cursor, OCR text, and accessibility elements—to understand what's on screen before acting. Levels adjust detail: minimal, normal, full.MIT
- Wait for a UI element to appear on Android screen by polling the accessibility tree. Use after navigation to ensure the screen is ready before interacting with elements.MIT
- Polls the device UI tree until two consecutive snapshots are identical, indicating animations and loading have completed. Use after navigation or screen transitions to ensure the new screen is ready for interaction.Business Source 1.1
- Dump the current mobile UI hierarchy into structured element data with coordinates, text, and interaction flags. Supports text, TSV, and XML output formats.Apache 2.0
- Locate and classify UI elements in screen regions using YOLOv8n object detection accelerated by NPU or CPU, outputting labeled bounding boxes with confidence scores.MIT
- Click UI elements by element ID or screen coordinates. Supports left, right, and double clicks for precise interaction with desktop apps.MIT
- Lists all inbounds from the 3x-ui panel with their clients, protocol, port, and traffic stats.Apache 2.0
- Capture a mobile screen to inspect current UI state. Use only for visual awareness, not for element interaction; retrieve fresh screenshot each time.Apache 2.0
- List UI elements on the current screen with text, accessibility labels, and pixel coordinates. Use this to identify tappable areas.MIT
- Retrieve a current UI tree with interaction points as @ref tokens for element selection in tests. Supports compact or full format, depth limits, and filtering to interactive elements.MIT
- List on-screen Flutter UI elements with tap coordinates. Filter by text or clickable-only to locate controls precisely without screenshots.MIT
- Wait for a UI element to appear on screen within a specified timeout. Use this tool after navigation, loading screens, or animations to ensure elements are present before interaction.MIT