The Forge Gauntlet

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare non-read-only, non-destructive, non-idempotent, closed-world. The description adds value beyond them: it discloses the progression semantics (job verdict and next job on completion, report card after the last job) and the hard constraint that 'only the sandbox is touched', which reinforces the openWorldHint=false claim. It could still say what happens on invalid tool names or exceeding the call limit.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Tight, front-loaded, and organized as Inputs / Returns / Limits, so an agent can scan the key facts in one pass. No filler sentences; every clause carries operational information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description correctly carries the return-value burden by enumerating the three outcome shapes (tool result, verdict plus next job, report card). Given a nested args object and only two sibling tools, this is nearly complete; the main gap is error/limit-exceeded behavior.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is only 33% (only gauntlet_id is described), so the description must compensate. It names all three inputs, and 'tool' gains meaning from 'call one sandbox tool', but the format of tool names and the shape of args are left unspecified. The reference to 'each job's tool-call limit (in limits)' points at a field that is not in the input schema, which adds ambiguity rather than semantics.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource: 'call one sandbox tool in the current job' during a Gauntlet run. It is clearly a step-advancing action tool, distinguishable from gauntlet_start and gauntlet_status by the 'during a run / one tool call' scope, though it never names those siblings explicitly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

'Use this during a Gauntlet run' gives the context for use, and the mention of gauntlet_start in the gauntlet_id description hints at the lifecycle. But there is no explicit when-not guidance or explicit routing versus gauntlet_status (e.g. 'use status to inspect without acting'), so the agent must infer the ordering.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the annotations, the description discloses concrete operational traits: per-address limits of 10 runs a day (20 for Sprints), a 2-hour wall clock per run (30 minutes for a Sprint), and that no account is required. It also describes what is returned (a gauntlet_id plus the first job's brief and its sandbox tools), which is the key behavioral fact for a non-idempotent, non-read-only operation that creates state. Nothing here contradicts the supplied hints.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The trigger condition is front-loaded and the remaining content is organized into labelled Inputs, Returns, and Limits segments, so nothing is buried. The telegraphic phrasing ('Sprints 20', 'a Sprint 30 minutes') is compact to the point of slight opacity, which costs a point.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema, and the description compensates by stating exactly what the caller receives, along with rate limits, time limits, and the fact that no account is needed. For a three-parameter, zero-required tool with full schema coverage, an agent has everything required to call it and to set user expectations about cost and duration.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description earns credit for adding intent-level meaning: card_from is framed as a re-run carrying forward its trap card, and challenge as measuring against a finished run or a named model's stored runs. That explains why an agent would choose each parameter, not just their formats.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'start' a gauntlet where 'the user wants their own agent tested on everyday jobs with planted traps, refereed from the end state.' That phrasing distinguishes it from the sibling tools (gauntlet_act and gauntlet_status), which by name imply acting within and polling a run rather than starting one. An agent can pick this tool without inspecting the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

'Use this when the user wants their own agent tested...' gives an explicit trigger condition, and the inputs section distinguishes the two legitimate alternatives: card_from for a 'measured re-run with its trap card' versus challenge for a 'finished gauntlet_id, or model:<id>.' It does not name the sibling tools or state when not to use this tool, so it stops short of full routing guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, destructiveHint=false and openWorldHint=false, so the safety profile is covered. The description adds value beyond that with a state-dependent return shape (current job with brief/tools/calls-left, or the report card once finished) and a concrete rate limit of 60 calls per minute per address, which is exactly the kind of operational context annotations cannot express.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Three short labeled clauses (purpose, Inputs, Returns, Limits) front-load the intent and carry no filler. Every sentence contributes information an agent would otherwise have to guess.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-parameter read tool with no output schema, the definition covers input, both possible return states, and a rate limit, which is close to sufficient. It stops short of telling the agent the terminal condition for polling (e.g., stop when the report card is returned) or what happens for an unknown or expired gauntlet_id.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and the single parameter carries a regex pattern plus a pointer to gauntlet_start as its origin. The description's 'Inputs: gauntlet_id' restates the parameter name without adding format, sourcing, or error semantics beyond what the schema already provides, so baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific action and resource: retrieve the standing of a Gauntlet run, with a named input and a described return. It implicitly distinguishes itself from gauntlet_start and gauntlet_act by framing itself as an observation of an existing run, though it never names those siblings.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied by 'see where a Gauntlet run stands', which suggests a polling/check-in role after a run exists. There is no explicit when-to-use versus when-to-use-alternatives guidance and no stated precondition that the run must have been created by gauntlet_start, even though the schema hints at it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.