execute_full_test

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Describes exact execution behavior: launches 3-5 allowlisted workers via execFile, captures output, no model-supplied run-log. Also states what happens when opt-in is missing (returns BLOCKED). No annotations provided, so description fully covers behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Description is a single dense paragraph but front-loads key information and every sentence adds value. Could benefit from structure but remains clear and concise.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Provides context on execution environment, opt-in, and alternatives, but lacks detail on return format and does not explain all parameters (runId, hypothesisId) thoroughly. Adequate given tool complexity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 60% with descriptions for prompt, routes, and timeoutMs. The description adds context for routes (3-5 frontier worker routes) but does not explain runId or hypothesisId beyond what's in the schema. Adequate but not enhanced.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it's a supervisor-executed full test that launches frontier workers, distinguishing it from sibling tools like test_hypothesis which record run-logs without execution.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly states it's off by default with opt-in (env variable), and specifies alternatives (artifact_record + test_hypothesis) when exec is disabled. Also clarifies that failed launches don't count toward retirement.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.