verify_judge_bias

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, which establishes the safety profile. The description adds behavioral insight by specifying the exact binomial test and the classes of artifacts considered (longer/first/same-family), which is valuable beyond the annotations. No contradiction present.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is exceptionally concise: two sentences that immediately frame the problem and provide usage instructions. There is no wasted text, and the exact binomial method is stated in a compact phrase. It is front-loaded with the core question, making it easy for an agent to understand.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple statistical tool with full schema descriptions and an output schema, the description covers the input requirements and method. It does not explain edge cases or prerequisites, but the output schema handles return values. The only minor gap is not elaborating on the 'longer/first/same-family' artifacts, but this is not critical for usage.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema descriptions cover all 4 parameters with 100% coverage (wins, n, p0, api_key). The description rephrases wins as 'count of verdicts the tested side won' and n as 'total,' which adds context but no new constraints or format details. Baseline 3 is appropriate when schema documents parameters well.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly conveys the tool's purpose: to determine whether an LLM-judge/metric preference is real or an artifact (longer/first/same-family) using an exact binomial test. The verb 'verify' is implied via the tool name and title, and the resource is specifically judge bias, distinguishing it from sibling verify_* tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage context is implied: pass wins and total count when you want to test a preference against chance. However, it does not explicitly state when to use this tool versus alternatives among the many verify_* siblings, nor does it mention exclusions or prerequisites. The instructions are present but not contrasted with other tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.