run criterion alignment

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations carry only bare hints (readOnlyHint:false, idempotentHint:false, openWorldHint:true), and the description goes far beyond them: billable spend with per-label economics ('one judge call per label; a new run is new spend'), auth restrictions, dual sync/async execution with a threshold (<=50 labels), every major error mode with its status code and message, the polling endpoint, and a 300s maxDuration. It also clarifies the openWorldHint by disclosing that background work continues after the response. No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Well front-loaded: purpose first, then endpoint/auth, then the capitalized money warning, then behavioral branches and errors. Nearly every sentence earns its place, and the verbose return schemas are justified by the absence of an output schema. Minor inefficiency: the money warning is stated twice ('SPENDS MONEY' and again in 'Notes: MONEY...').

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

This is a genuinely complex tool — dual sync/async execution, billing, permissions, timeout, and no output schema — and the description covers every branch: both return payloads with field details, polling instructions, the 30-label minimum, all key error codes with example messages, and the 403 auth constraint. Nothing an agent needs to invoke and interpret this tool correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% — the single 'id' parameter is documented as 'Criterion id.' — so the baseline of 3 applies. The description only incidentally confirms the parameter via the URL template POST /v1/criteria/{id}/align and adds no format, validation, or usage nuance beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource with rich detail: 'Calibrate a criterion by re-judging every in-scope human-labeled trace and measuring agreement (TPR/TNR with Wilson intervals, Cohen's kappa)'. The 'earns a judge the trust needed to gate on it' clause gives operational purpose, and the detail distinguishes this from siblings like get_criterion_alignment (fetch results) and auto_improve_criterion (improve the criterion).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides clear context for invocation: it is the step that earns gating trust, it explicitly warns OWNER/ADMIN only (403) and that it spends wallet money, and it instructs how to follow up for large sets (poll GET /v1/criteria/{id}/alignment). However, no sibling alternatives are named (e.g., get_criterion_alignment for checking existing results without spending) and there is no explicit 'when not to use' statement.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.