Skip to main content
Glama

scout_criterion

Record a ticket acceptance criterion's pass, fail, or not-tested verdict with confidence, reason, and linked findings to track test coverage.

Instructions

Record whether one acceptance criterion of a ticket read with scout_tickets passed, failed or was not tested, with how sure you are. The link from a criterion to the findings that show it is YOUR judgement, stated with a confidence — never matched on words. A "fail" names the findings that show it (file them with scout_finding first); "not-tested" says why in untestedBecause. Recording the same criterion again from the same session replaces your earlier verdict. Touches no browser.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "ticket": "<ticket>",
      "criterion": "<criterion>",
      "verdict": "<verdict>",
      "confidence": "<confidence>",
      "reason": "<reason>"
    },
    "name": "scout_criterion"
  }
}

Input Schema

NameRequiredDescriptionDefault
reasonYesWhat you saw, or why it could not be tried, in a sentence
ticketYesThe ticket's id as scout_tickets gave it, e.g. "PROJ-12" or "T1"
sessionNoTarget this session directly instead of the active one — pass it explicitly when dispatching to MULTIPLE sessions in one turn (e.g. two scout_click calls with different `session`), which then run CONCURRENTLY rather than queueing. Omit for single-session sequential use.
verdictYes"pass", "fail" or "not-tested"
findingsNoIds of the findings that show this criterion failing (required for a fail; may be given for a pass, none for not-tested)
criterionYesThe criterion's id, e.g. "AC2" (or just "2")
confidenceYesHow sure you are of this verdict and of the findings linked to it, from 0 to 1. State it honestly
untestedBecauseNoOnly with verdict "not-tested": "no-access" (the role this run used could not reach it), "observe-blocked" (it needs a change sent and this session is in observe mode), "out-of-scope" (it is outside what this run could check, such as an email or another system)

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed1 schema field changedv3.31.0
    • removedInput schema / additionalProperties
      Removed value: -false
  2. Addedv3.17.0

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and does well: it discloses that re-recording replaces the prior verdict (mutation/idempotency semantics) and that the tool 'touches no browser' (side-effect boundary). It could say more about permissions or what happens across sessions, but the key behavioral traits are covered.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded with the core action and stays dense but purposeful; each sentence (verdict semantics, finding linkage, replacement behavior, no-browser note) earns its place. Slightly long, but no filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For an 8-parameter mutation tool with no annotations and no output schema, the description covers the essentials: what is recorded, how findings link, replacement behavior, and side-effect scope. Nothing critical appears missing for correct invocation, though more on cross-session or concurrency implications could round it out.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline would be 3. The description adds value beyond the schema by framing the criterion-to-finding link as the agent's judgement stated with confidence rather than word-matching, and by spelling out that a fail names findings while not-tested uses untestedBecause — reinforcing the conditional parameter relationships.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb (record) and resource (a verdict on one acceptance criterion of a ticket read via scout_tickets), plus the confidence dimension. It distinguishes itself from siblings by referencing scout_tickets for the source and scout_finding for the linked evidence, so an agent can tell what it does and what it does not.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives explicit conditions: file findings with scout_finding FIRST when the verdict is 'fail', and explain 'not-tested' via untestedBecause. It also states the replacement behavior (re-recording the same criterion from the same session overwrites the earlier verdict), leaving little to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.