update criterion

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description goes well beyond the annotations by disclosing side effects: instrument changes void calibration, judge_prompt changes delete stored confusion rows, retiring removes the criterion from online scoring, and enabling online monitoring spends wallet funds capped by online_cap_usd. It also states the return shape. No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact and front-loaded: purpose first, then endpoint/auth, errors, return value, and high-impact consequences. Every sentence carries distinct information, and there is no filler or repetition of schema details.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Despite no output schema, the description specifies the 200 response shape, auth, expected error codes, side effects, and cost implications. Combined with the exhaustive input schema, an agent has everything needed to select and invoke this tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents every parameter including constraints like unit being create-only and judge_prompt voiding calibration. The description adds a helpful high-level grouping and money caveat, but does not substantially add parameter-level semantics beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource—'Update a criterion's prompt, judge model, population scope, online-monitoring settings or lifecycle status'—and names the exact fields and endpoint. This clearly distinguishes update_criterion from sibling tools like create_criterion, get_criterion, delete_criterion, and run_criterion_alignment.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description states the operation context (PATCH update), required API-key scope (evals:write), access restriction (OWNER/ADMIN), and error cases (403, 404, 400). It does not explicitly name sibling alternatives or say when not to use it, but the field list and lifecycle notes make the intended use clear.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.