Skip to main content
Glama

screen_act

Idempotent

Do one thing on the page or app, like a person would: the pointer travels to what it clicks, fields are clicked before typing, scrolling uses the wheel; then a fresh look (no screenshot unless asked). action: click | double_click | hover | type | press | scroll | navigate | back | wait | say | focus | unfocus | highlight | hide | unhide. Prefer ref from the latest look over x/y. You edit the take as you record it: the cut is finished the moment you stop, so decide everything here. While recording, use action 'say' with text before each step to narrate it (it becomes a caption and a timed line of the saved script, ready to be voiced). To show something off (a result, a chart, a setting), use action 'focus' with its ref, several refs framed together, or an exact area x, y, w, h (it fills the view); pad adds room, zoom 1.1-4 says how close outright, seconds how long: the take's camera zooms in and holds that long, or until the next focus or 'unfocus' (nothing on the page changes); without it the camera follows your clicks and typing. The result's shots gives each focus's zoom and view (the part of the page in frame, CSS pixels): check it and focus again if it is not what a viewer should see. Before recording, a focus only reports its shot: rehearse your framing. To point out or copy words, use action 'highlight' with the ref and the text in it, and color, seconds, lift, zoom_in if you like: the words are selected by a drag (press Control+C, Meta+C on a Mac, to copy) and the take marks them, lifted off the page (in an app the element is marked, nothing is selected). To keep something private (an e-mail, a key, a customer's name), use action 'hide' with its ref (and the text in it to hide only those words), or an area x, y, w, h: it is blurred in the take from then, following it as the page scrolls, until 'unhide' (with the ref, or none for everything); hidden before recording, it is hidden from the first frame. While recording every result says at: the take's clock in seconds. Pass steps (a list of these same objects, max 40) to run several in a row. In an app, if the user moves the mouse you are stopped: ask them, then pass resume=true.

response_detail (summary|ids|compact|full), fields (dotted paths), cursor (next page) — see the server instructions.

Tool call template

This tool declares no required parameters. Add optional parameters from the input schema as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {},
    "name": "screen_act"
  }
}

Input Schema

NameRequiredDescriptionDefault
hNo
wNo
xNo
yNo
keyNo
padNo
refNo
urlNo
whyNo
liftNo
refsNo
textNo
zoomNo
clearNo
colorNo
stepsNo
actionNo
amountNo
cursorNo
fieldsNo
resumeNo
submitNo
secondsNo
zoom_inNo
directionNo
screenshotNo
response_detailNo

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed8 schema fields changed
    • addedInput schema / properties / color
      Added value: +{
      +  "anyOf": [
      +    {
      +      "type": "string"
      +    },
      +    {
      +      "type": "null"
      +    }
      +  ],
      +  "default": null,
      +  "title": "Color"
      +}
    • addedInput schema / properties / h
      Added value: +{
      +  "anyOf": [
      +    {
      +      "type": "number"
      +    },
      +    {
      +      "type": "null"
      +    }
      +  ],
      +  "default": null,
      +  "title": "H"
      +}
    • addedInput schema / properties / lift
      Added value: +{
      +  "anyOf": [
      +    {
      +      "type": "boolean"
      +    },
      +    {
      +      "type": "null"
      +    }
      +  ],
      +  "default": null,
      +  "title": "Lift"
      +}
    • addedInput schema / properties / pad
      Added value: +{
      +  "anyOf": [
      +    {
      +      "type": "number"
      +    },
      +    {
      +      "type": "null"
      +    }
      +  ],
      +  "default": null,
      +  "title": "Pad"
      +}
    • addedInput schema / properties / refs
      Added value: +{
      +  "anyOf": [
      +    {
      +      "items": {},
      +      "type": "array"
      +    },
      +    {
      +      "type": "null"
      +    }
      +  ],
      +  "default": null,
      +  "title": "Refs"
      +}
    • addedInput schema / properties / w
      Added value: +{
      +  "anyOf": [
      +    {
      +      "type": "number"
      +    },
      +    {
      +      "type": "null"
      +    }
      +  ],
      +  "default": null,
      +  "title": "W"
      +}
    • addedInput schema / properties / why
      Added value: +{
      +  "anyOf": [
      +    {
      +      "type": "string"
      +    },
      +    {
      +      "type": "null"
      +    }
      +  ],
      +  "default": null,
      +  "title": "Why"
      +}
    • addedInput schema / properties / zoom_in
      Added value: +{
      +  "anyOf": [
      +    {
      +      "type": "boolean"
      +    },
      +    {
      +      "type": "null"
      +    }
      +  ],
      +  "default": null,
      +  "title": "Zoom In"
      +}
  2. Changed1 schema field changed
    • addedInput schema / properties / zoom
      Added value: +{
      +  "anyOf": [
      +    {
      +      "type": "number"
      +    },
      +    {
      +      "type": "null"
      +    }
      +  ],
      +  "default": null,
      +  "title": "Zoom"
      +}
  3. Added

TDQS

A4.3/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Well beyond the annotations, it discloses that the camera zooms and holds until the next focus/unfocus, that hide blurs content from that moment onward and tracks scrolling and is effective from frame one if set before recording, that highlight performs a real drag-selection (so Control+C/Meta+C will copy) except in apps where only the element is marked, and that results carry `at` (take clock) plus `shots` with zoom and `view` in CSS pixels. Minor tension: the annotations declare idempotentHint=true while clicking/typing/navigating are inherently repeat-sensitive, but the safety-critical hints (readOnlyHint=false, destructiveHint=false) are consistent.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Purpose and the ref-over-x/y rule are front-loaded, and nearly every clause carries real information, but the body is a single dense paragraph of semicolon-chained clauses with no grouping by action. The trailing 'response_detail ... fields ... cursor ... see the server instructions' is tacked on and reads like boilerplate from another tool, diluting an otherwise high-value block.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 27-parameter, no-output-schema, 0%-coverage tool this is unusually rich on behavior and recording semantics, covering focus/highlight/hide and the returned shots/at fields. It still leaves roughly ten parameters undefined and never states which parameter combinations are valid per action, so an agent cannot reliably construct a 'press' or 'navigate' call from the description alone.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 27 parameters at 0% schema description coverage, the description carries the full burden and does explain a good share: ref over x/y, exact area x, y, w, h, refs framed together, pad, zoom 1.1-4, seconds, color, lift, zoom_in, text, steps (max 40), resume, and screenshot ('no screenshot unless asked'). However key, url, amount, direction, submit, clear, why, fields, cursor, and response_detail — including the appended response_detail/fields/cursor tail — are never tied to an action, and conditional requirements (e.g., that type needs text plus ref, press needs key) are unstated.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The opening states a concrete action verb against a concrete resource ('Do one thing on the page or app, like a person would') and immediately enumerates the action vocabulary (click | double_click | hover | type | press | scroll | navigate | back | wait | say | focus | unfocus | highlight | hide | unhide). It is clearly separable from screen_look (a fresh look) and screen_record_start/stop, which are named implicitly through the recording and 'latest look' references.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicit routing rules are given for nearly every variant: use 'say' with text before each step while recording, use 'focus' to show off a result/chart/setting, use 'highlight' to point out or copy words, use 'hide' for private data, use 'steps' to chain up to 40, and pass resume=true when a user moves the mouse in an app. It even states a when-not condition: before recording, focus only reports its shot, so it is for rehearsing framing. Nothing is left to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.