run_evaluator

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are supplied, so the description carries the full burden and does: it discloses persistence (an Evaluation Run appears in list_evaluation_runs and Metric history), the monetary cost of model-judged Evaluators with an instruction to confirm first, and the meaning of outcome=ERRORED and lingering CREATED/STARTED status with the polling fallback.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the core action and effect, then alternatives, then cost, then result semantics, then param docs — a sensible order. The cost warning is effectively stated twice (the standalone 'costs money' line and the later per-turn model-call sentence), which is mild redundancy in an otherwise dense passage.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema and no annotations, the description still covers invocation prerequisites, cost/consent, return shape (metrics per address with rationales, error_detail), and the async/polling path. Nothing an agent needs to call it correctly or interpret the response is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, yet every one of the four parameters is explained: pipeline_name's relationship to the Session, evaluator_version_id as the exact version (with where to find it), session_id accepting either a search_session_id or a single-turn query_id, and wait_seconds as the timeout before returning an id to poll. This compensates fully for the empty schema descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

Opens with a precise verb+resource+effect: runs one Evaluator version on one Session and persists the result. It immediately distinguishes itself from try_evaluator (stores nothing) and start_experiment_run (many Sessions via Metric Group), so an agent can route correctly without opening schemas.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicit when-to-use ('judge one Session with an Evaluator that already exists'), explicit alternatives for adjacent cases ('to judge many Sessions with a Metric Group use start_experiment_run'), and explicit non-equivalence with try_evaluator. It even tells the agent how to obtain the required ids via get_evaluator and list_sessions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.