get eval samples

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already carry the safety profile (readOnly, idempotent, non-destructive), so the description focuses on behaviors annotations cannot express: server-side 2000-char clipping with a '[clipped]' marker, outcome enum differences between comparison ('win'|'loss'|'tie'|'failed') and criterion ('pass'|'fail'|'unparsed') runs, the '__stored__' baseline fallback semantics, empty-array-before-sampling behavior, and late-appearing human_verdict fields. This is dense, valuable behavioral disclosure beyond the structured data.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The purpose is front-loaded in sentence one, and the subsequent length is earned: with no output schema, the description must document a nested return payload, and it does so in a logical progression (endpoint/auth → array shape → field semantics → enums → caveats → alternative endpoint). Each sentence carries distinct information; nothing is filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with a nested return structure, run-state-dependent edge cases, and no output schema, the description covers everything an agent needs: the full item shape (prompt, baseline_answer variants, candidates array), per-run-type outcome enums, clipping limits with the escape hatch, human_verdict timing, empty/partial states, and the workspace 404. No critical gap remains.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% — the lone 'id' parameter is already documented as 'The eval run id' — so the baseline is 3. The description adds meaning on top: the id appears in the endpoint path, is workspace-scoped (404 for runs outside the workspace), and is valid across any run status. This modestly exceeds what the schema alone provides, warranting a 4.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The opening sentence states a specific verb and resource: 'Inspect the test cases behind a run's score', then enumerates exactly what is included (sampled prompt, each arm's answer with reasoning traces stripped, per-sample verdict). It differentiates from the sibling get_eval (run metadata) and get_eval_evidence (full transcripts) by naming the evidence endpoint explicitly as the alternative for full content.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides explicit routing: use GET /v1/evals/{id}/evidence?with_content=true when clipped 2000-char texts are insufficient. Also gives clear availability conditions — works on any status, partial data while RUNNING, empty array before sampling, 404 when the run is not in the workspace — so an agent knows when the call is valid vs. when results will be incomplete.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.