Android Mobile MCP

Instructions

Implementation Reference

• main.py:99-100 (registration)

  Registration of the 'mobile_dump_ui' tool using the @mcp.tool() decorator, including function signature and docstring describing input/output schema.

  @mcp.tool()
  def mobile_dump_ui() -> str:

• main.py:100-108 (handler)

  Handler function for mobile_dump_ui tool, checks device initialization and delegates to internal _mobile_dump_ui.

  def mobile_dump_ui() -> str:
      """Get UI elements from Android screen as JSON with hierarchical structure.
      
      Returns a JSON structure where elements contain their child elements, showing parent-child relationships.
      Only includes focusable elements or elements with text/content_desc/hint attributes.
      """
      if device is None:
          return "Error: Device not initialized. Please call mobile_init() first to establish connection with Android device."
      return _mobile_dump_ui()

• main.py:110-122 (handler)

  Core handler logic: dumps UI hierarchy XML, parses it with ElementTree, extracts UI elements using helper, and returns as string.

  def _mobile_dump_ui():
      try:
          xml_content = device.dump_hierarchy()
          root = ET.fromstring(xml_content)
          
          global current_ui_state
          ui_coords.clear()
  
          ui_elements = extract_ui_elements(root)
          return str(ui_elements)
  
      except Exception as e:
          return f"Error processing XML: {str(e)}"

• main.py:36-83 (helper)

  Recursive helper to extract UI elements: filters focusable or text-bearing elements, computes center coordinates, builds hierarchical structure with children, collects global UI coordinates.

  def extract_ui_elements(element):
      resource_id = element.get('resource-id', '')
      
      text = element.get('text', '').strip()
      content_desc = element.get('content-desc', '').strip()
      hint = element.get('hint', '').strip()
      bounds = parse_bounds(element.get('bounds', ''))
      focusable = element.get('focusable', 'false').lower() == 'true'
      
      has_text = bool(text or content_desc or hint)
      
      children = []
      for child in element:
          children.extend(extract_ui_elements(child))
  
      if not (focusable or has_text):
          return children
      
      display_text = text or content_desc or hint
      if focusable and not display_text:
          child_texts = get_children_texts(element)
          display_text = ' '.join(child_texts).strip()
      
      element_info = {
          "text": display_text,
          "class": element.get('class', ''),
          "coordinates": {"x": bounds["x"], "y": bounds["y"]} if bounds else None
      }
  
      global ui_coords
      ui_coords.add((bounds["x"], bounds["y"]))
  
      if resource_id:
          element_info["resource_id"] = resource_id
      
      if children:
          filtered_children = []
          for child in children:
              child_text = child.get("text", "")
              child_coords = child.get("coordinates")
  
              if not (child_text == element_info["text"] and child_coords == element_info["coordinates"]):
                  filtered_children.append(child)
  
          if filtered_children:
              element_info["children"] = filtered_children
      
      return [element_info]

• main.py:14-24 (helper)

  Helper to parse bounds string from XML to center coordinates used in UI elements.

  def parse_bounds(bounds_str):
      if not bounds_str or bounds_str == '':
          return None
      try:
          bounds = bounds_str.replace('[', '').replace(']', ',').split(',')
          x1, y1, x2, y2 = map(int, bounds[:4])
          center_x = (x1 + x2) // 2
          center_y = (y1 + y2) // 2
          return {"x": center_x, "y": center_y, "bounds": [x1, y1, x2, y2]}
      except:
          return None