platform_eval_evaluators

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It only states 'Manage' and lists actions, revealing nothing about side effects, permissions, rate limits, reversibility, or the meaning of the confirm parameter (which is necessary for delete). The description adds no behavioral depth beyond what the schema already enumerates, making it inadequate for an agent to anticipate consequences.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is short (two sentences), but the first sentence about Arch Evaluate is generic context not specific to this tool, adding noise. The key information (resource + endpoint + actions) is in the second sentence, which is front-loaded enough. It is concise but could be tighter by removing the introductory clause and focusing solely on evaluator management.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given 6 parameters, nested objects, and no output schema or annotations, the description is under-specified. It does not explain what each action returns, the meaning of 'templates', the requirement to set confirm=True for delete, or any error handling. An agent calling this tool for the first time would lack critical operational details, making the definition incomplete for correct invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 83%, so most parameters (action, projectId, evaluatorId, confirm, body, query) already have descriptions or enums. The description adds no additional parameter meaning—it merely repeats the action list already in the enum. Per the rubric, high schema coverage sets a baseline of 3, and the description does not enhance parameter understanding, so the score holds at 3.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool manages 'eval evaluators' and lists specific actions (list, get, create, update, delete, templates), making the resource and operations explicit. It is distinct from nearby eval siblings like personas, scenarios, sets, and runs, though it does not explicitly contrast itself with them. The leading Arch Evaluate context is slightly tangential but does not obscure the core purpose.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. It does not mention which sibling tools to prefer, when to choose list vs get vs templates, or any conditions or exclusions. The actions are self-evident from the enum, but the description offers no contextual decision-making help, leaving the agent to infer usage.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.