Skip to main content
Glama

get_clickable_elements

Identify all interactive screen elements with coordinates to determine what can be tapped during Android UI testing and debugging.

Instructions

Get all clickable/interactive elements on screen with their coordinates. Perfect for understanding what can be tapped.

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
device_serialNo

Implementation Reference

  • The handler function decorated with @mcp.tool() that implements the get_clickable_elements tool. It fetches the UI hierarchy XML, parses for clickable nodes using regex, extracts properties like text, content-desc, resource-id, bounds (with calculated center and size), and class name. Returns a list of dictionaries for each clickable element.
    @mcp.tool() def get_clickable_elements(device_serial: str | None = None) -> list[dict]: """ Get all clickable/interactive elements on screen with their coordinates. Perfect for understanding what can be tapped. """ xml = get_ui_hierarchy(device_serial) elements = [] # Parse clickable elements pattern = r'<node[^>]*clickable="true"[^>]*>' for match in re.finditer(pattern, xml): node = match.group() element = {} # Extract text text_match = re.search(r'text="([^"]*)"', node) if text_match: element['text'] = text_match.group(1) # Extract content-desc desc_match = re.search(r'content-desc="([^"]*)"', node) if desc_match: element['content_desc'] = desc_match.group(1) # Extract resource-id id_match = re.search(r'resource-id="([^"]*)"', node) if id_match: element['resource_id'] = id_match.group(1) # Extract bounds and calculate center bounds_match = re.search(r'bounds="\[(\d+),(\d+)\]\[(\d+),(\d+)\]"', node) if bounds_match: x1, y1 = int(bounds_match.group(1)), int(bounds_match.group(2)) x2, y2 = int(bounds_match.group(3)), int(bounds_match.group(4)) element['bounds'] = {'x1': x1, 'y1': y1, 'x2': x2, 'y2': y2} element['center'] = {'x': (x1 + x2) // 2, 'y': (y1 + y2) // 2} element['size'] = {'width': x2 - x1, 'height': y2 - y1} # Extract class class_match = re.search(r'class="([^"]*)"', node) if class_match: element['class'] = class_match.group(1) if element: elements.append(element) return elements
  • Helper function get_ui_hierarchy used by get_clickable_elements to dump and retrieve the UI hierarchy XML from the device via uiautomator.
    @mcp.tool() def get_ui_hierarchy(device_serial: str | None = None) -> str: """ Dump the complete UI hierarchy as XML. Shows all visible elements, their properties, bounds, and content descriptions. """ run_adb(["shell", "uiautomator", "dump", "/sdcard/ui_dump.xml"], device_serial) output = run_adb(["shell", "cat", "/sdcard/ui_dump.xml"], device_serial) run_adb(["shell", "rm", "/sdcard/ui_dump.xml"], device_serial) return output

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/rahulkr/r_adb_mcp_server'

If you have feedback or need assistance with the MCP directory API, please join our Discord server