run_probe

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description must disclose all behavioral traits. It mentions returns (prompt, hint) and the lack of autograding. Missing side effects, permissions, rate limits, or error conditions. Adequate but not comprehensive.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, no fluff. The critical information is front-loaded. Every sentence earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given an output schema exists, the description doesn't detail return format. It mentions prerequisites implicitly (probe must be registered) but doesn't state error handling or preconditions. Adequate but could be more complete for a tool with one parameter and no annotations.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description carries full burden. It only implies 'probe_id' is the identifier of a registered probe. No details on format, examples, or constraints. Bare minimum, adding little beyond the parameter name.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'Surface' and resource 'registered probe'. It specifies the return values (prompt and grader hint) and distinguishes from sibling tools like 'register_probe' and 'record_attempt'. Purpose is immediate and unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly tells the agent to call 'record_attempt' afterward and explains why autograding is absent. It implies when to use this tool (when attempting a probe). No explicit exclusions but the flow is clear.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.