Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and openWorldHint, so safety behavior is covered. The description adds meaningful context beyond annotations: it lists the returned agreement statistics and explains that min_raters overrides the ranking's configured value and that api_key handling differs by endpoint. It does not contradict the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact and front-loads the returned metrics, which is the most important information. The remaining sentences about min_raters and api_key are useful; the 'Layer 4' phrase is mildly contextual but not wasteful. Structure is functional despite a mid-sentence line break.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so detailed return-value explanations are not strictly necessary, yet the description still names the returned statistics. Annotations cover the safety profile, and the description adds auth and override behavior. The only notable gap is lack of explanation for the required ranking_id parameter.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It explains min_raters as an override and clarifies api_key behavior across endpoints, but it does not explain the required ranking_id parameter's expected format or role. Coverage is partial rather than complete.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific resource and outcome: inter-annotator agreement metrics for a ranking, including Cohen's κ, Fleiss' κ, Krippendorff's α, Scott's π, and a coverage report. It is clear what the tool returns and is distinguishable from sibling metric tools like get_ceiling or get_crowd_bt, though it does not explicitly name those alternatives.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage context is implied by the metric list and by notes about min_raters overriding the ranking configuration and api_key being optional on the public endpoint but required on /mcp/mcp. However, it does not say when to use this agreement tool versus other ranking-analysis siblings, nor does it provide explicit exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.