create_evaluator

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the whole burden and does it well: it warns that Judged Metrics cost money when run, that hub-api rejects code whose @metrics declaration cannot be parsed, that the name must be unused in the workspace, that the Evaluator scores nothing until a Metric Group runs it, and that new_metric_group_name creates an additional object as a side effect.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Behavioral warnings and prerequisites are front-loaded in the first sentences, which is good structure. It loses a point because the trailing :param/:returns block partially restates what the prose already explained (notably python_code and the Metric Group options), adding length without new information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a creation tool with a 4-parameter schema, no annotations, and no output schema, the description covers prerequisites, side effects, failure modes, cost implications, and even the return value ('The created Evaluator, or an error message'). Nothing an agent needs in order to call it correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0% (bare titles only), yet the description documents all four parameters with meaning beyond the schema: name must not already exist, python_code is the Evaluation Function whose @metrics declaration is mandatory, metric_group_ids are existing groups with insertion at the end of each set, and new_metric_group_name creates a group containing only this Evaluator. This fully compensates for the coverage gap.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

Opens with a specific verb+resource and scope: 'Creates an Evaluator from Python source and adds it to the workspace.' It then names the sibling tools it is not (try_evaluator for validation, list_evaluator_templates for scaffolding, update_evaluator for changes), so an agent can distinguish it without opening any schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives explicit when-to-use routing: validate first with try_evaluator, start from list_evaluator_templates rather than from scratch, and use update_evaluator for later changes. It also explains the two mutually available paths for grouping (metric_group_ids vs new_metric_group_name) and the later alternative (update_metric_group).

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.