Skip to main content
Glama

pilot_observe

Inspect a desktop window using its accessibility tree and an optional screenshot. Use it to understand UI structure before performing actions or verifying expected results.

Instructions

Observe a desktop window using accessibility and optionally an image.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "app": "<app>",
      "vision": "<vision>"
    },
    "name": "pilot_observe"
  }
}

Input Schema

NameRequiredDescriptionDefault
appYes
visionYes
window_idNo

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observedv0.4.0

TDQS

C2.4/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations supplied, the description carries the full burden. It discloses the underlying mechanism (accessibility tree plus optional image) but says nothing about whether the call is read-only, what happens if the window is missing, how large the payload may be, or what a vision capture costs. That leaves an agent guessing on behavior for an inspection tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

It is a single short sentence with no wasted words, but the brevity comes at the cost of substance rather than from tight editing. Front-loading is fine for what little content exists.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No output schema, no annotations, and zero schema description coverage on three parameters means the description is the only source of information and it omits the essentials: what an observation returns, how app and window_id combine, and what changes when vision is enabled. Inadequate for the tool's ambiguity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0% across three parameters, so the description must compensate and largely does not. 'Optionally an image' loosely maps to the vision boolean, but the required 'app' parameter and the optional 'window_id' are never explained, nor is the relationship between them or whether app is a name, bundle id, or process.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose3/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a verb ('Observe') and a resource ('a desktop window') plus the mechanism ('accessibility and optionally an image'), which distinguishes it roughly from action tools like pilot_act or desktop_click. However, 'observe' is imprecise about what the agent actually receives (tree text? screenshot? both?), and it does not contrast with read-style siblings such as desktop_get or desktop_find.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no when-to-use guidance, no mention of prerequisites such as the target window being focused or existing, and no routing between this tool and alternatives like desktop_get, desktop_find, or pilot_act. The only hint is that vision is optional, which an agent must infer the tradeoff for.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.