mobile_dump_ui
Extract UI elements from Android screens as structured JSON with parent-child relationships to enable automated testing and interaction analysis.
Instructions
Get UI elements from Android screen as JSON with hierarchical structure.
Returns a JSON structure where elements contain their child elements, showing parent-child relationships. Only includes focusable elements or elements with text/content_desc/hint attributes.
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
No arguments | |||
Implementation Reference
- main.py:99-100 (registration)Registration of the 'mobile_dump_ui' tool using the @mcp.tool() decorator, including function signature and docstring describing input/output schema.@mcp.tool() def mobile_dump_ui() -> str:
- main.py:100-108 (handler)Handler function for mobile_dump_ui tool, checks device initialization and delegates to internal _mobile_dump_ui.def mobile_dump_ui() -> str: """Get UI elements from Android screen as JSON with hierarchical structure. Returns a JSON structure where elements contain their child elements, showing parent-child relationships. Only includes focusable elements or elements with text/content_desc/hint attributes. """ if device is None: return "Error: Device not initialized. Please call mobile_init() first to establish connection with Android device." return _mobile_dump_ui()
- main.py:110-122 (handler)Core handler logic: dumps UI hierarchy XML, parses it with ElementTree, extracts UI elements using helper, and returns as string.def _mobile_dump_ui(): try: xml_content = device.dump_hierarchy() root = ET.fromstring(xml_content) global current_ui_state ui_coords.clear() ui_elements = extract_ui_elements(root) return str(ui_elements) except Exception as e: return f"Error processing XML: {str(e)}"
- main.py:36-83 (helper)Recursive helper to extract UI elements: filters focusable or text-bearing elements, computes center coordinates, builds hierarchical structure with children, collects global UI coordinates.def extract_ui_elements(element): resource_id = element.get('resource-id', '') text = element.get('text', '').strip() content_desc = element.get('content-desc', '').strip() hint = element.get('hint', '').strip() bounds = parse_bounds(element.get('bounds', '')) focusable = element.get('focusable', 'false').lower() == 'true' has_text = bool(text or content_desc or hint) children = [] for child in element: children.extend(extract_ui_elements(child)) if not (focusable or has_text): return children display_text = text or content_desc or hint if focusable and not display_text: child_texts = get_children_texts(element) display_text = ' '.join(child_texts).strip() element_info = { "text": display_text, "class": element.get('class', ''), "coordinates": {"x": bounds["x"], "y": bounds["y"]} if bounds else None } global ui_coords ui_coords.add((bounds["x"], bounds["y"])) if resource_id: element_info["resource_id"] = resource_id if children: filtered_children = [] for child in children: child_text = child.get("text", "") child_coords = child.get("coordinates") if not (child_text == element_info["text"] and child_coords == element_info["coordinates"]): filtered_children.append(child) if filtered_children: element_info["children"] = filtered_children return [element_info]
- main.py:14-24 (helper)Helper to parse bounds string from XML to center coordinates used in UI elements.def parse_bounds(bounds_str): if not bounds_str or bounds_str == '': return None try: bounds = bounds_str.replace('[', '').replace(']', ',').split(',') x1, y1, x2, y2 = map(int, bounds[:4]) center_x = (x1 + x2) // 2 center_y = (y1 + y2) // 2 return {"x": center_x, "y": center_y, "bounds": [x1, y1, x2, y2]} except: return None