Skip to main content
Glama

ui_tree

Read-only

Extracts a window's controls as compact, one-line text with refs so AI agents can inspect desktop UIs; filter lines or include screen rectangles.

Instructions

A window's controls as compact text, one line per control with a ref (e12). Long lists show 25 items; filter prints matching lines only (refs still cover the window); rects adds screen rectangles. If it shows almost nothing (games, Qt, canvas apps), use ui_ocr.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "title": "<title>"
    },
    "name": "ui_tree"
  }
}

Input Schema

NameRequiredDescriptionDefault
rectsNoadd @x,y wxh
titleYespart of the window title (case-insensitive; an exact title wins) or a handle from list_windows (0x...)
filterNoonly lines containing this text
max_nodesNodefault 300

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observedv0.1.0

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only declare readOnlyHint=true and openWorldHint=false; the description adds genuinely non-obvious behavior: output is truncated to 25 items on long lists, and filtering only narrows printed lines while refs still cover the whole window. It also discloses a failure mode (near-empty results on games/Qt/canvas apps). It does not say whether refs remain valid across calls or what happens when multiple windows match.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three tight sentences, each carrying distinct information: output format, display/filter/rects modifiers, and the fallback path. The most decision-relevant content (what it returns) is front-loaded, and there is no filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description carries the burden of describing return values, and it does so adequately — line-per-control text with e12-style refs. Combined with the truncation rule and the ui_ocr escape hatch, an agent has enough to call it correctly. Minor gaps remain around ref lifetime and multi-window title matching, which the schema only partially covers.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, and the description earns a bump by explaining filter semantics beyond the schema: matching lines print only, but refs still cover the full window — a subtlety the schema text does not convey. It also glosses rects as adding screen rectangles, though it leaves the relationship between max_nodes (default 300) and the stated 25-item display cap unexplained.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

"A window's controls as compact text, one line per control with a ref (e12)" names the resource (a window's UI control tree), the output shape (one line per control), and the key artifact (refs). It also explicitly differentiates from ui_ocr as the fallback, so an agent can separate the two without opening schemas. The only weakness is that the action verb is implied rather than stated.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives an explicit when-not condition and names the alternative: "If it shows almost nothing (games, Qt, canvas apps), use ui_ocr." It also clarifies when to reach for filter and rects. It stops short of saying when to prefer this over ui_act, window, or capture, so it is strong but not a full routing guide.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.