Skip to main content
Glama

Interact With Node

interact_with_node

Interact with desktop UI widgets by node ID or label to perform clicks, typing, value changes, and selection via silent AT-SPI actions, falling back to coordinate input and returning UI tree deltas.

Instructions

Interacts with an interactive UI widget by 1D node ID (e.g. 'b1', 'e1') or label.

Executes Level 1 AT-SPI actions silently (without moving the cursor), falling back to Level 2 coordinate/keyboard input only when AT-SPI is unsupported, and returns the post-action UI tree delta (modified/appeared widgets). Optional role narrows name matches like Playwright's getByRole. Supports actions: click, type, set_value, clear, select_all, select_range, copy, paste, cut, drag_select, double_click, right_click, hover.

Tool call template

This tool declares no required parameters. Add optional parameters from the input schema as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {},
    "name": "interact_with_node"
  }
}

Input Schema

NameRequiredDescriptionDefault
pidNo
roleNoOptional accessible role filter (e.g. 'push button', 'entry').
textNo
valueNo
actionNoclick
targetNoPrimary target: compact element ID (e.g. 'b1', 'e1') or widget label.
node_idNoDeprecated alias for 'target'; prefer passing 'target'.
end_offsetNo
start_offsetNo
settle_timeout_msNo

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observedv0.2.0

TDQS

B3.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Adds real value beyond annotations: it discloses that AT-SPI actions run silently without moving the cursor, that it degrades to coordinate/keyboard input when AT-SPI is unsupported, and that it returns a post-action UI tree delta. It says nothing about what happens on target-not-found, how settle_timeout_ms affects the wait, or whether Level 2 fallback is disruptive. This is consistent with annotations (destructiveHint false, idempotentHint false) and not contradictory.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, front-loaded with the primary identification mechanism, followed by execution strategy and action list. Dense but mostly efficient; the action list is long but load-bearing since no enum exists in the schema.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 10-parameter, no-output-schema tool, it does well to describe the return value (UI tree delta) and safety profile. But between the low schema coverage and the absence of guidance on action-specific parameters, an agent cannot reliably construct calls for type, set_value, or select_range without guessing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is only 30% across 10 parameters, so the description carries most of the burden. It documents target, role, and enumerates action values, but leaves text, value, start_offset, end_offset, settle_timeout_ms, and pid entirely undocumented. Six-plus parameters have no meaning anywhere.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource: interact with a UI widget by compact element ID or label, with an explicit action list. It does not differentiate itself from siblings that perform the same actions individually (click, double_click, hover, type_text, drag), leaving overlap unaddressed.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides contextual guidance implicitly by explaining the Level 1 AT-SPI-first strategy and the optional role filter analogous to Playwright's getByRole. However, it never says when to choose this multi-action tool versus the granular siblings like click or type_text, nor when the fallback to Level 2 input matters.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.