Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only declare readOnlyHint=false and destructiveHint=false; the description adds real behavioral context beyond that — the variant arm is handed to the SDK with a configuration requirement, promotion switches the agent's model, and deployment is verified as a model swap on fresh traffic. It does not cover failure modes or whether a started test can be abandoned, but for a non-destructive mutating tool this is meaningfully richer than the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, each carrying distinct information (what it does, that no optimization is needed, what happens on promotion), with the core action front-loaded. The SDK-configuration sentence is slightly dense for a description but earns its place as a precondition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a mutating tool with no output schema, the description covers the essential lifecycle: what starts, the SDK-side requirement, and what promotion does. Nothing critical to invoking it correctly is missing, though the outcome of the test decision (parity rule, scoring thresholds) is only hinted at via the parameters.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and each of the seven parameters is documented in-schema, so the heavy lifting is already done. The description adds only conceptual framing ('a second model', 'the cheaper one ships'), with no extra guidance on trafficAllocation, variantPercentage, or minScoredConversationsPerArm semantics, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource (start a production A/B test) and immediately narrows the scope to a model-only change ('changes only the model'), which is the exact distinction from the sibling start_production_test. An agent can pick between the two test-starting tools without opening either schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives clear usage context: 'No optimization is required', and explains the prerequisite that the SDK must be configured to serve the target model via a same-client override or resolveModel hook. It stops short of explicitly naming the alternative (start_production_test or an optimization-driven flow) as the choice for quality-improving changes, so the when-not is only implied.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.