CompletionKit

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the transparency burden. It discloses the write nature via 'upsert' and the conditional corrected_score requirement, but does not cover authorization, idempotency, conflict resolution, or return value behavior. Adequate but not exhaustive.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences with no redundancy. The first sentence states the action and key, the second clarifies constraints. Every word earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 7-param tool with no output schema, the description provides the essential usage conditions and key fields. It omits return value and error behavior, but is otherwise sufficient given the tool's simplicity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, so the description compensates by naming the key tuple (run, response, metric, created_by) and explicitly defining verdict enum values and the corrected_score condition. This adds semantic value, but the 'note' parameter remains unexplained in both the schema and description.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the specific verb 'Upsert an agreement' and identifies the unique key tuple (run, response, metric, created_by). This distinguishes it from agreements_list and other resource-specific siblings.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The usage context is implied through the description of the operation and constraints, but there is no explicit guidance on when to use this tool versus agreements_list or alternative mutation tools. No alternatives or exclusions are mentioned.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are present, so the description carries the full burden. It states that the tool lists agreements and can filter by specific fields, which implies a read-only operation. However, it discloses no additional behavioral traits such as pagination, authorization requirements, or return format. It is minimally transparent but not misleading.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two short sentences, front-loaded with the core action. No unnecessary words or repetition. Every phrase adds value: the first sentence defines purpose, the second enumerates filter options.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple list tool with four optional filters and no output schema, the description covers the basic purpose and filter parameters. However, it omits important context such as whether no filters returns all agreements, pagination behavior, and the shape of the response. It is adequate for a basic understanding but lacks completeness.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has four parameters with no descriptions (0% coverage). The description adds meaning by explicitly identifying these parameters as filters, which is beyond the bare type information. It does not explain filter semantics (e.g., exact match, combination logic), but it compensates for the schema gap by labeling them as filtering criteria.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action 'List agreements' and identifies the resource. It distinguishes from sibling agreements_create by focusing on listing rather than creating. The filter fields are specified, making the purpose unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear context for when to use this tool: to list agreements with optional filters. It does not explicitly exclude alternatives or name sibling tools, but the listing vs. creating distinction is implicit. No when-not guidance is given, so it stops short of a perfect score.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full behavioral burden and does well by disclosing header-row handling, special treatment of expected_output/actual_output columns, per-run overrides, and variable availability to prompts. It does not discuss side effects like persistence or permissions, but that is partially implied by the 'create' verb, and the added column behavior is valuable beyond the name and schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences long, front-loaded with the main action, and each sentence adds meaningful detail. It covers the essential CSV structure, special column behaviors, and extensibility without any wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Considering there is no output schema and no annotations, the description provides a surprisingly complete picture of the tool's behavior, including edge cases like prompt-less runs and per-run overrides. The main gap is the undocumented tag_names parameter, but the core functionality is well specified.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, so the description must compensate. It thoroughly explains the csv_data parameter, including header semantics and the role of special columns, and clarifies behavior related to expected_column and output_column. However, it does not explicitly define the tag_names parameter, which remains ambiguous.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool creates a dataset from CSV data, which is a specific verb+resource combination. It implicitly distinguishes from datasets_create_from_url by specifying inline CSV data, but does not explicitly name the sibling alternative, so it falls short of a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description conveys that this tool is for creating datasets when CSV data is already available, and it details relevant CSV formatting rules. It does not explicitly exclude scenarios or mention alternatives, but the context is clear enough for an agent to infer appropriate use.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the transparency burden. It discloses important behavioral details: the URL is SSRF-checked, downloads are capped at 10MB, the first row is the header, and special columns are recognized. It does not describe error handling or response formats, but it covers key operational constraints well.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise and front-loaded. The first sentence states the core purpose, and each subsequent sentence adds essential operational detail without unnecessary fluff.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Despite no annotations or output schema, the description provides solid contextual coverage: when to use it, URL requirements, size limits, CSV header behavior, and special column handling. It could mention auth expectations or failure modes, but it is sufficient for an agent to correctly select and invoke the tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is only 33%, but the description adds significant semantics beyond the schema: it explains the URL must be public http(s), that the server fetches the CSV directly, and that expected_output/actual_output columns have special treatment. It does not elaborate on 'name' or 'tag_names', but those are largely inferable from context.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: 'Create a dataset by downloading CSV from a URL instead of inlining it.' This uses a specific verb and resource, and explicitly contrasts with the inline creation approach, effectively distinguishing it from sibling datasets_create.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives explicit guidance: 'Use this for large datasets' and contrasts with 'instead of inlining it,' which points toward the alternative tool. It clearly conveys the intended use case, though it does not explicitly name the sibling tool or state when *not* to use this tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries full responsibility for disclosing behavioral traits. It does not mention that deletion is permanent, whether it requires special permissions, or any side effects on associated data. This is a significant gap for a destructive operation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely short, but it is under-specified rather than concisely informative. It lacks even a basic phrase structure that provides context, and the single sentence does not earn its place beyond echoing the tool name.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's destructive nature and the absence of annotations, output schema, or parameter documentation, the description is wholly inadequate. It does not explain the effect, prerequisites, or even what the 'id' refers to, making the tool impossible to use safely without external knowledge.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema has one required integer parameter 'id' with no description, and the tool description provides no explanation of what this ID represents or how to obtain it. With 0% schema description coverage, the description fails to compensate or add any meaning beyond the raw parameter type.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Delete a dataset' is a direct tautology of the tool name 'datasets_delete', restating exactly what the name already implies. It provides no additional specificity about the resource or operation beyond the name.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance on when to use this tool versus alternatives like datasets_update or datasets_get. No prerequisites, scenarios, or exclusions are mentioned, leaving the agent without any decision-making support.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It only states 'Get a dataset by ID' and does not mention whether the operation is read-only, what errors can occur, whether authentication is required, or what the response contains. This is a significant gap for a tool with no structured safety hints.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is very short and free of filler, which is structurally clean. However, it is under-specified; the brevity does not add value because it omits essential context. It is minimally adequate but not impactful, scoring between 'efficient' and 'under-specified'.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool has no output schema, no annotations, and very few siblings, the description must explain the return format, potential errors, or the nature of the dataset. It does none of this, making it completely inadequate for an agent to understand what happens when the tool is invoked.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema has one parameter (id) with no description (0% coverage). The description does not mention the parameter at all, so it provides no additional meaning beyond the bare schema. For a tool with a single parameter, the description should at least confirm or elaborate on what the ID refers to, but it does not.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'Get' and the resource 'dataset' with a specific scope 'by ID'. It is unambiguous and distinguishes itself from dataset creation/list/update/delete tools, though it does not explicitly differentiate from other resource-specific 'get' tools (e.g., metric_groups_get).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance on when to use this tool versus alternatives like datasets_list or datasets_delete. No context is provided about typical use cases or prerequisites, leaving the agent to infer when to call this function.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the burden of behavioral disclosure. It only says 'List all datasets', which repeats the tool name and provides no details on safety, pagination, response format, or side effects.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence with no unnecessary words. It efficiently conveys the tool's purpose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple list-all tool, the description is minimally adequate, but it lacks information about return values, pagination, or filtering. Since there is no output schema, the description could be more complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool takes no parameters (0 params) and the schema is empty, so the baseline is 4. The description does not need to add parameter details since there are none.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses the specific verb 'List' and the resource 'datasets', with the qualifier 'all' distinguishing it from datasets_get and other dataset operations. It clearly states what the tool does.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance on when to use this tool versus siblings like datasets_get or datasets_create. The description merely states the function without providing context for choosing it over alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description must disclose behavioral traits, but it merely says 'Update a dataset' without explaining whether updates are partial or full, what happens to unspecified fields, or any side effects. This leaves critical behavioral ambiguity unaddressed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single short sentence, which is concise in length, but it is under-specified. It does not earn its place because it lacks the necessary detail to be useful, making this a case of under-specification rather than effective conciseness.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool has four parameters, is a mutation operation, and has no output schema or annotations. The description is far too minimal to be complete. It provides no context on return values, error conditions, or operational semantics, making it inadequate for effective use.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema has 0% description coverage, and the description adds no meaning to the parameters. The property names (id, name, csv_data, tag_names) are self-evident to some degree, but the description does not clarify their roles or interactions, so it fails to compensate for the schema gap.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool updates a dataset, with a specific verb and resource. It is distinct from sibling tools like datasets_create, datasets_delete, etc., but lacks any scope or detail on what aspects can be updated, so it falls short of a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance on when to use this tool versus alternatives. No mention of prerequisites, context, or exclusions. The description provides no actionable usage direction.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden for behavioral transparency. The word 'compare' implies a read-only operation, and the constraint about belonging to the same metric suggests validation behavior. However, the description does not explicitly state whether the tool has side effects, what it returns, or how errors are handled, leaving important behavioral aspects undisclosed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences with the main purpose stated in the first sentence and essential requirements/constraints in the second. Every word earns its place; there is no fluff or redundant information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple comparison tool, the description covers the primary action, key inputs, and an important limitation (check metrics). However, since there is no output schema, the description does not explain what 'agreement stats' are or what the side-by-side output looks like, and it omits error behavior. These gaps make it only partially complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has three required integer parameters with 0% schema description coverage, so the description must add semantic meaning. It names all three parameters and, crucially, adds the relationship constraint that both version IDs must belong to the given metric_id. This goes beyond the bare schema and helps the agent understand the domain logic, although it doesn't detail value formats or enums.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Compare two versions of one metric's agreement stats side by side' clearly identifies the specific verb (compare), the resource (metric's agreement stats), and the scope (two versions of one metric). This distinguishes it from sibling tools like metrics_get or metric_versions_list, which do not directly offer side-by-side version comparison.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly states the required input parameters and a key constraint ('both versions must belong to that metric'), which clarifies when the tool is applicable. It also provides an exclusion ('Unavailable for check metrics'). However, it does not name any alternative tools for cases where the criteria aren't met, so it earns a 4 rather than a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden. It does disclose a key behavioral trait: 'This only sets up the run; call runs_generate to actually re-judge.' This is valuable. Yet it omits other behavioral details such as side effects, return values, or prerequisites (e.g., whether the judge must already exist). The transparency is adequate but not thorough.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is composed of two tight sentences. The first sentence delivers the core purpose and wrapper details; the second clarifies the workflow by referencing runs_generate. No unnecessary words or redundant content.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool has 5 parameters, no annotations, and no output schema. The description outlines the setup-vs-generate flow and the intended comparison scenario, but it lacks details on what the run creation returns, how the judge is selected, or any error/where it fits in the broader workflow. It is serviceable but incomplete for full contextual understanding.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is only 20% (only output_column has a schema description). The description additionally explains that output_column is supplied and prompt_id is omitted, which adds some meaning. However, it does not clarify the semantics of name, metric_id, dataset_id, or judge_model, which are all required. Given the low coverage, the description should compensate more but does not.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's function: 'Create a scoring run for the current judge over a dataset's existing outputs.' It also specifies that it wraps runs_create with a particular configuration, distinguishing it from the general runs_create sibling. This gives a specific verb, resource, and scope.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly instructs to 'call runs_generate to actually re-judge the outputs,' which guides the user on the next step and differentiates when to use this tool versus runs_generate. It also implies the use case of comparing against human verdicts. However, it does not explicitly exclude or contrast with other sibling tools like runs_regrade, so it's not a perfect 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations available, the description carries the full burden of behavioral disclosure, but it only states 'create' without noting side effects, required permissions, idempotency, or what the response contains. This leaves the agent with no understanding of the operation's behavior beyond the verb itself.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is appropriately brief, but its brevity crosses into under-specification. A single sentence that merely restates the tool's name is not sufficient to guide correct invocation. There is no front-loaded information beyond the action.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with four parameters and no annotations or output schema, the description is severely incomplete. It doesn't explain what constitutes a valid metric group, how metric_ids or tag_names are used, or what the expected behavior is on success/failure. This makes it difficult for an agent to invoke the tool without additional context.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

All four parameters (name, tag_names, metric_ids, description) have no documentation in the schema, and the description does not mention any of them. With 0% schema description coverage, the description fails to compensate by explaining parameter meanings, types, or relationships.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Create a metric group' clearly identifies the verb and resource, making it distinct from read/update/delete sibling tools. However, it does not elaborate on what a metric group is or how this tool differs from other create tools, so it is clear but minimal.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. There is no mention of prerequisites, when to prefer this over metric_groups_update, or any contextual hints. It simply states the action.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are available, so the description carries the full burden. 'Delete' implies destructive action, but it does not disclose whether deletion is permanent, what happens to associated metric versions, or if any confirmation is required. This is a significant transparency gap.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is incredibly short and front-loaded, but it is under-specified. It is not verbose, yet it omits key details about the id parameter and behavioral implications, making it less useful than a slightly longer, more informative description.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple delete tool with one parameter, the description covers the basic action, but given the absence of annotations and output schema, more context is needed. It does not mention side effects, what the id represents, or how to verify success, leaving the tool under-specified for reliable invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema has one parameter (id) with 0% schema_description_coverage, and the description does not mention the id parameter at all. It fails to explain what the id refers to (e.g., the metric group's unique identifier), leaving the agent to infer from the schema alone.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Delete a metric group' uses a specific verb and resource, clearly indicating the action. It distinguishes from siblings like metric_groups_create/update/get, though it could be confused with metrics_delete if not parsed carefully.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

No usage guidance is provided. There is no mention of when to use this tool versus alternatives, no preconditions, and no exclusions. The description gives no context for decision-making.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden for behavioral disclosure. It only states the basic action without mentioning return format, error behavior (e.g., 404 if not found), permissions, or whether it returns the full resource. This is a gap for a read operation with no other context.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, focused sentence with no redundant words. It front-loads the action and resource, making it highly efficient and easy to parse.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple get-by-ID tool with no output schema and no annotations, the description covers the core operation but lacks details about the return value, error cases, or any special behaviors. It is minimally viable but leaves the agent to infer standard REST semantics.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema has one integer parameter named 'id' with zero description coverage. The description's "by ID" clarifies that the parameter is the identifier of the metric group, adding modest semantic meaning beyond the raw schema. While it doesn't deeply compensate, the tool is simple enough that this is adequate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description "Get a metric group by ID" uses a specific verb (get) and resource (metric group) and explicitly scopes by ID. It clearly distinguishes from sibling tools like list, create, update, and delete.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied by the phrase "by ID" — the agent knows it should use this when it has a metric group ID. However, there is no explicit guidance on when to use this versus metric_groups_list or other alternatives, so it falls short of a clear directive.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden of disclosing behavior. It only says 'List all metric groups,' offering no details on read-only nature, pagination, return format, or authentication. This is minimal and leaves behavioral traits largely undisclosed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single short sentence with no wasted words. It is front-loaded and concise, effectively communicating the purpose without unnecessary detail.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool is a simple zero-parameter list operation, but the description lacks contextual completeness. No output schema is provided, and the description does not mention return format, pagination, or any other operational details, leaving the agent to infer these.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters, so there are no parameter semantics to explain. The description is not required to add meaning beyond the empty schema, and the baseline for zero-parameter tools is 4.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool lists all metric groups, with a specific verb and resource. It distinguishes from siblings like metric_groups_get (which fetches a single group), though it does not explicitly call out alternatives.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus alternatives. The description only states what it does, without context for when listing all metric groups is appropriate or how it differs from other list tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

There are no annotations, so the description must disclose behavioral traits. It merely states 'Update', implying mutation but declining to mention any effects, permissions, or return values. This is a serious gap for a mutation tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

At three words, the description is succinct but severely under-specified. It lacks essential context, so this is not successful conciseness; it is minimalism that fails to inform.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With 5 parameters, no output schema, and no annotations, the tool requires meaningful description. This one-sentence description is wholly insufficient for an agent to select and invoke the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, and the description adds no parameter-level information. The schema lists parameter names and types but no explanations; the description does not compensate, leaving the agent to guess what 'name', 'tag_names', 'metric_ids', etc. mean.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses the specific verb 'Update' with the resource 'metric group', clearly distinguishing it from sibling operations like metric_groups_create/delete/get/list. However, it lacks any scope detail (e.g., which fields can be updated) that would make it more precise.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. It does not mention prerequisites, conditions, or why one would choose update over create/delete/get, leaving the agent without context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the burden of behavioral disclosure. It explains the behavior of different check kinds (contains, regex, equals, etc.), how compare_to works, and when target_path is required. It also clarifies how expected values are graded. This is substantial added context, though it doesn't cover every possible side effect (e.g., return values, permissions), which would push it higher.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single dense paragraph but front-loads the purpose before diving into configuration details. It is information-dense without being redundant, though bullet points or clear section breaks would improve readability. Every sentence adds value and doesn't waste words, so it earns a 4 rather than a 5.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (nested check_config, enum metric_type, no annotations, no output schema), the description is incomplete. It fully details the 'check' metric type but gives no guidance for the 'llm_judge' type, leaving rubric_bands and instruction parameters unexplained. It also omits any mention of return values or errors. For a create tool with this much complexity, a 2 is appropriate because major areas are unaddressed.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 0% description coverage, so the description must compensate. It thoroughly explains check_config parameters like value, pattern, min/max, json_path, compare_to, and expected_path, and distinguishes between constant and expected comparisons. However, it does not explain simpler parameters like name, tag_names, instruction, or rubric_bands, leaving some gaps, but the complex parameters are well covered.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'Create a metric with evaluation criteria' with specific verb and resource. It distinguishes from sibling tools like metrics_update by focusing on creation, and provides a detailed breakdown of configurable check kinds and their required keys, making its purpose unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives clear guidance on when to use the tool (for creating metrics) and provides detailed configuration instructions for deterministic checks, including required keys per check kind and how to compare against expected values. However, it does not explicitly compare against alternative tools like metrics_update or mention scenarios where creation is inappropriate; the guidance is tool-internal rather than cross-tool, so it's not a full 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden of behavioral disclosure. It only states 'Delete a metric' without specifying whether the deletion is permanent, whether it cascades to related data, or if any special permissions are required. This falls short of disclosing the consequences of the operation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence that front-loads the action and contains no unnecessary words. It is optimally concise and easy to parse.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description is insufficiently complete for a destructive operation. There is no output schema, no annotations, and no explanation of success/failure behavior, reversibility, or error conditions. While the tool is simple, the lack of context leaves the agent without a full understanding of the operation's implications.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema lists a single required 'id' parameter with no description, and schema description coverage is 0%. The description implies that the id is the metric's identifier, but it does not explicitly document the parameter's role or format. The simplicity of having a single parameter mitigates the lack of detail, but the description still adds minimal semantic value.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Delete a metric' clearly states the verb (delete) and the resource (metric), making its purpose unambiguous. It distinguishes itself from sibling tools like metrics_create, metrics_get, and metrics_update by specifying exactly the delete operation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

No usage guidance is provided. The description does not explain when to use this tool instead of alternatives, nor does it mention any prerequisites or side effects, leaving the agent without context for making a selection.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description must fully disclose behavioral traits. It only states 'Get a metric by ID' and does not mention what happens if the ID is not found, whether the full metric object is returned, authentication requirements, or any side effects. Even for a read operation, this lack of context leaves the agent uncertain.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, short sentence that is front-loaded with the action and resource. Every word is useful, with zero redundancy, making it highly scannable and fit for quick understanding.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema and no annotations, the description is the sole source of behavioral context. It fails to specify the return value (e.g., the metric object, a 404 error, or a different shape), leaving a significant gap for a tool that is expected to return data.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema shows a single required integer 'id' with no description (0% coverage). The description's 'by ID' helps clarify that this parameter is the metric's identifier, partially compensating for the absent schema documentation. However, it provides no detail on ID constraints or how to obtain the ID.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb 'get' with the resource 'metric' and the qualifier 'by ID', clearly indicating this tool fetches a single metric. This distinguishes it from siblings like metrics_list (which lists all metrics) and metrics_create/update/delete (which modify metrics).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'by ID' implies the intended use case is retrieving a specific metric when an ID is available, but it does not explicitly contrast with alternatives or provide when-not-to-use guidance. Sibling names like metrics_list are not mentioned, so usage context is only implied.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It does not mention that this is a read-only operation, return format, pagination, or any side effects. It essentially restates the tool name ('metrics_list' vs 'List all metrics'), adding no behavioral context beyond the literal meaning.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, clear sentence with no filler words. It gets straight to the point and is appropriately sized for a tool with no parameters.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a zero-parameter list tool, 'List all metrics' is functional but minimal. It does not specify what fields each metric includes, whether results are paginated, or if there are any scoping caveats. Given the lack of output schema, slightly more detail on the return structure would improve completeness.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters and an empty schema, so there is no parameter information to clarify. The baseline for 0 parameters is 4, and no additional semantic detail is needed.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'List all metrics' clearly states the action (list) and the resource (metrics), distinguishing it from sibling tools like metrics_get, metrics_create, and metrics_update. The word 'all' clarifies scope.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies use when a full list of metrics is needed, but does not explicitly mention when not to use it or point to alternatives like metrics_get for individual metrics. Usage is inferred from the verb and resource rather than actively guided.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden of behavioral disclosure. It discloses that each variant is saved as a draft MetricVersion with source='suggestion', that the drafts are returned, and that Stripe-metering hooks fire via ActiveSupport::Notifications. This gives the agent a clear picture of side effects (persistence, metering) beyond the input schema, though it does not mention potential costs or permission requirements.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is three sentences long, each serving a distinct purpose: stating the core action, explaining the persistence behavior and return value, and noting the metering side effect. It is front-loaded with the primary verb and resource, contains no fluff, and is appropriately sized for the tool's complexity.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given no annotations and no output schema, the description provides sufficient context: the tool's purpose, its side effects (saving drafts), return value (persisted drafts), and an important notification mechanism. It does not explain error conditions or permission requirements, but for a relatively simple tool with three parameters, the description is adequately complete to guide selection and invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema already provides strong descriptions for 'count' and 'model', leaving 'metric_id' without a description. The tool description adds context by explaining that the metric's judge instruction is rewritten, which clarifies the role of 'metric_id'. However, the description does not add significant semantic value beyond what the schema already covers, and with 67% schema coverage, it partially compensates but does not fully enrich the parameter meanings.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's function: it asks the model to rewrite a metric's judge instruction in N variants. It specifies the resource (metric's judge instruction), the action (rewrite/generate variants), and the output (persisted drafts). This distinguishes it from sibling tools like metrics_update or prompts_suggest_improvement, which target different resources or actions.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage—'targeted at the recent disagreements' suggests it is for improving a metric after disagreements. However, it does not explicitly state when to use this tool versus alternatives, such as manually editing the metric or using other metric_versions tools. No explicit exclusions or alternative tool references are provided.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It does a good job disclosing behavioral constraints for check_config (required keys, conditional compare_to, target_path requirement). However, it does not disclose whether the update is a merge or replacement of existing fields, any potential destructive effects, or authorization requirements. This leaves the mutation semantics incomplete.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is dense but well-structured, front-loading the purpose ('Update a metric') and then using a compact list of per-kind requirements. It minimizes waste and organizes complex information effectively, though it could benefit from bullet points or clearer separation of the general update behavior from check_config specifics.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (7 parameters, nested objects, no output schema), the description is incomplete. It provides deep coverage of check_config but omits guidance on other metric fields (name, rubric_bands) and the llm_judge metric type. It also does not describe the return value or side effects, leaving the agent with partial understanding for a tool that can update more than just check_config.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must add meaning. It thoroughly explains the check_config object, including per-kind required keys, compare_to behavior, and target_path/expected_path semantics, which goes well beyond the schema. However, it does not cover top-level parameters like name, instruction, or rubric_bands, leaving them without semantic elaboration.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'Update a metric' which is a specific verb+resource that distinguishes it from create/get/list/delete siblings. However, it does not explicitly mention the full scope of updatable fields (e.g., name, instruction, rubric_bands), instead focusing almost entirely on check_config, which slightly narrows the apparent purpose.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides conditional guidance for using different check kinds ('For a deterministic check set metric_type...') and how to use compare_to with expected. However, it does not explicitly state when to use this tool over alternatives like metrics_create, nor does it provide exclusions. The usage context is implied but not fully articulated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are present, so the description carries the full burden. It discloses the destructive nature ('Destroy'), the draft-only restriction, the refusal of published versions, and a workaround. It does not mention irreversibility or error semantics, but 'Destroy' strongly implies permanence.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences, immediately states the core action, and every clause adds value (scope, sources, refusal, alternative). There is no fluff or repetition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-parameter destructive tool with no output schema, the description covers the action, constraints, and alternative workflow effectively. It does not cover potential response details or the exact meaning of 'draft,' but these are either implied or domain-specific and not critical for selection and invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, and the single parameter has no description. The tool description implies that metric_version_id references the draft version to destroy, and it clarifies which sources are valid. However, it does not explicitly define the parameter's role or format beyond the schema's integer type, so it partially compensates but not fully.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action 'Destroy a draft MetricVersion' with specific verb and resource. It also defines the scope (draft only), mentions both source types ('edit' or 'suggestion'), and sets it apart from sibling tools like metric_versions_publish and metric_versions_list by focusing on destruction.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

It explicitly states when not to use the tool ('Published versions are refused') and provides a concrete alternative ('publish a different one as current instead'). This gives clear when-to-use and when-not-to-use guidance, with a suggested workflow.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden. It discloses return fields (version_number, state, source, current flag, timestamps), inclusion of drafts and published versions, and ordering by newest first. This gives a clear behavioral profile for a read-only list operation, though it does not mention pagination or potential large result sets.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences, front-loaded with the action, and every word adds value. It efficiently conveys purpose, scope, and expected output without redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple list tool with one parameter and no output schema, the description covers the essential aspects: what is listed, for which metric, ordering, and the fields in each row. It is not missing critical information, though it could mention whether pagination is used or if the result is a full list.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate for the lone metric_id parameter. It only says 'for a metric,' adding minimal meaning beyond the name and schema. It does not clarify that metric_id is the unique identifier of the parent metric, expected format, or that it's required.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action (List every MetricVersion), the resource (MetricVersion), the scope (for a metric), and its composition (drafts + published) with ordering (newest first). This distinctly separates it from sibling tools like metric_versions_publish or metrics_list.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives clear context by specifying the tool's purpose and output fields, which implies when to use it (e.g., when needing all versions of a metric). However, it does not explicitly mention alternatives or exclusions, such as 'use metrics_get for current version only' or 'use metrics_list to see all metrics'.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and does well: it reveals transactional semantics, peer demotion, and writes back to the metric. This goes beyond surface-level; missing only details like permission requirements or failure modes.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences with no filler: first states purpose, second details side effects. Efficiently front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-parameter tool with no annotations or output schema, the description covers core functionality, side effects, and usage context. It could mention return value or error conditions, but the essential behavior is fully explained.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The single parameter metric_version_id is not explicitly parsed in the description, but the description's explanation of what a version is and how it gets published makes the parameter's role clear. Since schema coverage is 0%, this is a strong compensatory explanation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

Clearly identifies the verb (Publish), the resource (MetricVersion), and the intended effect (make it the live version of its metric). Distinguishes from sibling tools like metric_versions_dismiss and metrics_update by explaining both draft publication and revert flows.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides concrete usage context: it works for draft→published and reverting to an older published version, which tells an agent when to invoke it. Does not explicitly exclude alternatives or mention when not to use it, so not a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden. It discloses the mapping rules (llm-rubric/g-eval to judge metrics, deterministic asserts to check metrics), the return of a summary, and the guarantee that 'nothing is dropped silently.' This is substantial behavioral disclosure, though it could add details on error handling or idempotency.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is three sentences, front-loaded with the core action, and each sentence adds essential information: what it does, how it maps, and what it returns. No wasted words or redundant phrases.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description covers the input (yaml contents), the mapping logic, and the output (summary of mapped/skipped). It also states the non-silent-drop policy. Given the simplicity of the input schema and no output schema, this is nearly complete. It could mention error behavior for malformed YAML, but that is not critical for understanding the tool's purpose.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema already provides 100% coverage for the single 'config' parameter, describing it as 'The full promptfooconfig.yaml contents.' The description reinforces this by mentioning the filename format, but does not add deeper parameter semantics beyond the schema. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's function: 'Import a promptfooconfig.yaml' and explicitly describes the resulting artifacts (prompt, dataset, metrics). This distinguishes it from sibling tools that individually create prompts, datasets, or metrics, making the purpose unambiguous and specific.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear context: this tool is used when you have a promptfooconfig.yaml to import, and it explains what will be created. However, it does not explicitly mention alternatives or when not to use this tool, so it does not fully meet the 'explicit when/when-not' criteria. The implied usage is strong enough to warrant a 4.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description provides zero behavioral disclosure. It does not mention side effects, required permissions, return values, or any special behavior, leaving the agent without necessary safety or operational knowledge for a mutation tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single short sentence that is not tautological but is under-specified. It is concise but lacks any structured detail or explanation, offering minimal value beyond the name.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with 5 parameters, no annotations, and no output schema, a single descriptive phrase is grossly insufficient to guide correct invocation. The agent has no information about input semantics, required fields, or expected behavior.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The description does not mention any of the five parameters (name, template, llm_model, tag_names, description), and since the schema provides no descriptions, the agent has no semantic understanding of what these fields represent or which are required.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Create a prompt' uses a clear verb and resource, and the sibling tool list reinforces the create/read/update/delete pattern, distinguishing it from prompts_update or prompts_get. However, it provides no additional scope or clarification about what a prompt entails.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance on when to use this tool versus alternatives. It doesn't mention that this is for creating a new prompt as opposed to updating an existing one, nor any prerequisites or context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of disclosing behavioral traits. It only states the basic action without explaining whether deletion is permanent, if it has cascading effects, or if it requires special permissions. For a mutating operation, this is a significant gap.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two words: 'Delete a prompt'. It is front-loaded with the verb and resource, contains zero waste, and is perfectly concise.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool is simple (one integer parameter, no output schema), so the description is adequate for the core intent. However, it lacks behavioral context such as return value, error conditions, or side effects. For a delete operation, this is a minimum-viable description but leaves gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, and the description does not elaborate on the 'id' parameter. The meaning of 'id' as a prompt identifier is inferred from context, but the description does not explicitly confirm this or explain any constraints beyond the integer type in the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Delete a prompt' clearly states the action (delete) on a specific resource (prompt). It unambiguously distinguishes itself from sibling tools like prompts_create, prompts_get, and prompts_update, which perform different operations.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. There is no mention of prerequisites, exclusions, or context such as 'use this instead of unpublishing' or 'this permanently removes the prompt'.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description must disclose behavioral traits, but it adds nothing beyond the tool's name. It does not state whether the operation is read-only, what happens if the ID is not found, response format, or authorization requirements. The word 'get' implies retrieval, but no concrete behavior is disclosed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is one short, front-loaded sentence with zero redundant words. It is concise and easy to parse.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool is simple (one parameter, no output schema), but the description lacks any mention of return values or error conditions. Without an output schema, the description should clarify what the response contains (e.g., the full prompt object) or how missing IDs are handled. It is adequate but incomplete for an agent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already describes the only parameter 'id' as an integer 'Prompt ID' with 100% coverage. The description adds no additional meaning beyond 'by ID', so baseline of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Get a prompt by ID' clearly states the specific action (get), resource (prompt), and the required identifier (by ID). This differentiates it from sibling tools like prompts_list (which retrieves multiple prompts) and prompts_create/update/delete (which mutate data).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives no indication of when to use this tool versus alternatives. It is implied that you need the ID, but there is no guidance on how to obtain the ID (e.g., via prompts_list) or any exclusions. For a tool with many siblings, this is insufficient.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It states 'List all prompts' but does not disclose behavioral traits such as pagination, sort order, whether archived prompts are included, or authentication requirements. This is a minimal statement with no additional context.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely concise and front-loaded, using only two words to convey the essential action. There is no wasted text, though it might be seen as slightly terse. It earns a 4 for efficiency without being a tautology.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema and no annotations, the description does not explain what the list returns (e.g., array of prompt objects) or any relevant details like filtering or pagination. For a simple list tool, it provides the minimum but leaves the response format and behavior undefined, making the agent guess.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters, so the baseline is 4. The description needs to add nothing about parameter meaning since the schema already shows an empty object, and no parameters exist to document.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'List all prompts' clearly identifies the action (list) and the resource (prompts), and the word 'all' implies a collection-level operation, distinguishing it from prompts_get which would fetch a single prompt. However, it does not explicitly mention any alternative tools or edge cases, so it stops short of full sibling differentiation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance on when to use this tool versus alternatives like prompts_get or prompts_list variants. The description only states what it does, with no context about prerequisites, exclusions, or preferred scenarios.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description must disclose behavioral traits, but it only states the core action. It does not mention side effects, reversibility, permissions, or what happens to the previous current version, which is a significant gap for a mutation tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence with no wasted words. It conveys the essential action and result efficiently, making it highly concise and well-structured.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description is too minimal for the tool's context: no annotations, no output schema, and no parameter details. It lacks essential information about when to use it, what the id refers to, and the consequences of publishing, leaving the agent under-informed.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, and the description does not explain the 'id' parameter. It only vaguely refers to 'a prompt version', leaving the agent to infer that 'id' corresponds to a version ID without explicit confirmation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states a specific action ('Publish') and a specific resource ('a prompt version'), with a concrete outcome ('making it the current version'). This distinguishes it from sibling tools like prompts_create or prompts_update, though it does not explicitly contrast with them.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. It does not mention prerequisites, eligibility of versions, or relationships to other prompt tools, leaving the agent without usage context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description discloses important behaviors: it analyzes run data, returns reasoning and rewritten template, preserves {{variables}}, and persists a Suggestion. This gives a clear picture of the tool's operation and side effects, though it does not cover all edge cases.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is three sentences, front-loaded with the core purpose, and every sentence adds meaningful detail without redundancy. It is efficient and well-structured.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description covers what the tool does, how it does it, what it returns (reasoning + rewritten template), and a key prerequisite. Given the simplicity (one parameter, no output schema), it is thorough and complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The single parameter run_id is well-described in the schema (100% coverage), and the description adds crucial context that the run must have a prompt, enriching the parameter semantics beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's function: suggesting an improved prompt based on run results and judge feedback. It specifies the resource (prompt), the action (improve/suggest), and differentiates it from simpler prompt tools by emphasizing grounding in test results and persistence as a Suggestion.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides a clear use case (improve prompt using run feedback) and a prerequisite (run must have a prompt). However, it does not explicitly mention alternatives or when not to use this tool, so it falls short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries full responsibility for behavioral disclosure. It clearly states that updates to prompts with runs do not go live immediately, creating a draft with current=false, and that publishing is a separate step. This is a significant, non-obvious behavior fully disclosed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences long, front-loaded with the primary action, and every clause serves a purpose. It efficiently communicates core functionality and the key versioning nuance without wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description covers the essential behavior for an update tool: the versioning logic and the distinction between updating in place vs creating a draft. It lacks mention of return values or error conditions, but these are minor for this type of tool, and the core contextual information is present.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0% and the description does not elaborate on any of the six parameters. While parameter names like 'name' and 'template' are self-explanatory, the description adds no additional meaning or context about how they are used, especially the required 'id' parameter.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description starts with 'Update a prompt,' clearly stating the verb and resource. It also distinguishes the tool from siblings like prompts_publish and prompts_create by explaining the versioning behavior, which is unique and specific.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly explains when the tool creates a draft version (if runs exist) versus editing in place (if no runs). It also tells the user to use prompts_publish to promote a draft, providing clear guidance on workflow and alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations and no additional information in the description, there is zero disclosure of behavioral traits such as side effects, required permissions, or reversibility. The description merely restates the action implied by the name.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise in word count but severely under-specified. It is not appropriately sized for a tool with four parameters and no annotations; it omits essential context, making it more under-specified than concise.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (4 parameters, required fields, no output schema, no annotations), the description is wholly incomplete. It doesn't explain what a provider credential is, how to obtain the api_key, supported provider configurations, or any return behavior.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, and the description provides no meaning for any of the four parameters (api_key, provider, api_version, api_endpoint). The agent must rely solely on parameter names and types, which is insufficient.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Create') and the resource ('a provider credential'), which distinguishes it from sibling tools like provider_credentials_delete, get, list, and update. The verb-resource combination is specific and unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus alternatives such as provider_credentials_update or other create tools. The description simply states the action with no context or exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden of behavioral disclosure. It only states 'Delete' without mentioning permanence, cascading effects, or potential errors if the credential is in use. This is a destructive operation, but no warnings or additional behavior are disclosed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely terse (5 words) but this is under-specification, not genuine conciseness. It lacks necessary details about the operation, making it less useful than a more complete but still compact description.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Although the tool is simple (one parameter, no output schema), the description is incomplete. It does not confirm that the deletion is permanent or explain what 'id' refers to. Given the lack of annotations and schema descriptions, the tool is under-documented for reliable use.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema describes a single integer parameter 'id' with zero description coverage. The tool description does not mention 'id' at all, failing to compensate for the schema's lack of explanation. The agent is left without any semantic meaning for the parameter beyond its name.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Delete a provider credential' clearly states the action (delete) and the resource (provider credential), which distinguishes it from sibling tools like provider_credentials_update or datasets_delete. It is specific and unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives, such as when a credential should be deleted, whether it is reversible, or when to prefer update instead. No context or exclusions are given.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the burden of behavioral disclosure. It explicitly states 'API key is not exposed', which is a valuable safety-related behavior beyond what the schema shows. The verb 'Get' implies read-only, and the caveat helps agents avoid expecting sensitive data in the response.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single well-structured sentence with no unnecessary words. The main purpose is front-loaded, and the parenthetical adds important security context without redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple one-parameter get operation with no output schema, the description covers the key aspects: what it retrieves and a critical limitation. It lacks explicit return format or error behavior, but given the simplicity, the provided context is sufficient for an agent to select and invoke the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It mentions 'by ID', which connects the single required `id` parameter to the credential's identifier, but adds no further detail beyond the schema's integer type. The semantics are clear but minimally elaborated.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Get') and resource ('provider credential') with a clear scope ('by ID'). It distinguishes from sibling tools like provider_credentials_list by focusing on a single credential retrieval. The parenthetical about API key not exposed adds a unique distinguishing detail.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'by ID' clearly indicates this tool is for retrieving a single specific credential when its ID is known. It provides clear context for usage, though it does not explicitly mention alternatives like provider_credentials_list. No exclusions are stated, but the purpose is unambiguous.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden of behavioral disclosure. The parenthetical 'API keys are not exposed' is a significant behavior note, informing users that the output excludes sensitive key material. However, it does not mention pagination, ordering, or required permissions.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, concise sentence that states the core functionality and a relevant caveat. It is well-structured and front-loaded, with no extraneous words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a zero-parameter list tool with no output schema, the description is largely complete. It tells the user the scope ('all') and a key limitation (API keys hidden). While it could mention return format or pagination, these are not essential for such a simple tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema is empty with zero parameters, so the description does not need to explain parameter semantics. According to the rubric, zero-parameter tools receive a baseline score of 4, as there is nothing to clarify.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'List' and the resource 'provider credentials', with 'all' indicating the scope. This distinguishes it from sibling tools like provider_credentials_get, which likely retrieves a single credential.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided about when to use this tool versus alternatives such as provider_credentials_get or provider_credentials_create. The description simply states what it does without contextualizing its specific use case.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of disclosure, but it only restates the operation without any behavioral details. It does not mention whether this is a partial update or full replacement, whether it overwrites existing fields, what permissions are required, side effects, or reversibility. This is a significant gap for a mutating tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is short with no wasted words, but it is under-specified. For a tool with 5 parameters and no parameter descriptions, a single generic sentence is not appropriately sized; it fails to serve as a useful guide, falling into the category of under-specification rather than concise clarity.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (5 parameters, no annotations, no output schema), the description is extremely incomplete. It does not explain return behavior, field update semantics, or usage context. The tool is not self-explanatory, and the description fails to fill any of the gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema has 5 parameters with 0% description coverage, and the description does not compensate at all. It neither lists the updatable fields nor explains the meaning of the parameters such as id, api_key, provider, api_version, or api_endpoint. The description provides no additional semantic value beyond the raw schema property names.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action (update) and the resource (a provider credential), which is specific enough to differentiate from the create, delete, get, and list siblings. However, it adds no detail about what aspects can be updated, making it slightly less informative than a description that lists updatable fields.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus alternatives. The description is a bare statement with no mention of prerequisites, exclusions, or context in which updating a credential is appropriate. It relies entirely on the user inferring usage from the tool name.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. It only repeats the tool's purpose ('get a specific response') without revealing return format, whether it returns the full response object, authentication requirements, or any side effects. This is essentially a tautology of the tool name.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, concise sentence that is easy to read and front-loaded with the key action. However, it is under-specified, so while it is appropriately brief, it lacks the substantive details needed for a complete tool definition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema and no annotations, the description should explain what the tool returns and how the parameters relate to a 'response'. It does neither, making the tool incomplete for an agent to invoke correctly without additional external knowledge.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has two required parameters (run_id and id) with no descriptions, and schema description coverage is 0%. The description does not compensate by explaining the relationship between run_id and id or how they are used to identify a response. Both parameters remain undefined in the description.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Get a specific response' clearly states the action (get) and resource (response), and the word 'specific' distinguishes it from the sibling tool responses_list, which lists responses. However, it does not explicitly differentiate from other getter tools like runs_get or explain what a 'response' represents in this context.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives such as responses_list or runs_get. There is no mention of prerequisites, typical use cases, or when not to use it. The only implied usage is that you need a specific response ID, but this is not stated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It reveals the return envelope, explains the rationale behind the 50-row default (large payloads), and suggests filter/sort combinations for failure-mode analysis. It does not mention pagination caveats or rate limits, but for a read-only list tool the disclosed behavior is substantial and practical.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is three sentences long, front-loaded with the core purpose, followed by practical usage tips, and ends with an alternative tool reference. Every sentence contributes meaningful information with no redundancy or fluff.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool has 8 parameters and no output schema, but the description covers the return envelope, defaults, parameter combination strategies, and an alternative for aggregated metrics. The schema covers parameter-level details (enum values, limits, filter semantics) with high coverage, making the overall description complete for a list operation with practical usage context.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is high (88%), so the schema already documents most parameters. The description adds value by explaining why the default limit is 50, how 'fields' relates to payload size, and how min_score/max_score and score_asc should be combined. This goes beyond the schema's individual parameter descriptions, though run_id remains undocumented in both schema and description.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'List responses for a run, in row order,' using a specific verb and resource, and distinguishes itself from runs_get by noting it is for row-level listing, not per-metric averages. This differentiates it from sibling tools like responses_get and runs_get.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit guidance on when to use this tool and when not to: it explains the default 50-row behavior, suggests using 'fields' to drop bodies, 'min_score'/'max_score' to isolate low scorers, and 'sort: score_asc' for worst rows. It also explicitly directs users to runs_get for per-metric averages, naming an alternative tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It discloses the key behavioral distinction between generation and grading existing outputs, but it does not mention side effects, permissions, rate limits, or what happens when conflicting parameters are provided.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise, consisting of two sentences with no redundant wording. The second sentence is a bit dense but still clear, and the overall structure is effective for a tool with this complexity.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool has 12 parameters, no output schema, and no annotations, the description is insufficiently complete. It does not explain return values, typical combinations of other parameters (like metric_ids or judge_model), or the relationship between dataset_id and the columns. The description focuses on only the two main modes and leaves many operational details unexplained.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The description adds meaning to prompt_id and output_column by explaining their roles in the two modes, which is valuable. However, the schema already provides descriptions for about half of the 12 parameters, and the description does not compensate for the others (e.g., tag_names, metric_ids). The added value is significant for those two params but not comprehensive.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb and resource ('Create a run') and adds a key conditional that distinguishes the scoring mode from the generation mode. However, it does not explicitly differentiate from sibling tools like runs_generate or runs_rerun by name, so it lacks full sibling differentiation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives clear context on the two primary modes: generate new outputs (provide prompt_id) or score existing outputs (omit prompt_id and provide output_column). This is actionable guidance, though it does not mention when to prefer this tool over alternatives like runs_generate.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. It merely states 'Delete a run' without addressing irreversibility, cascading effects, required permissions, or any other consequences of the deletion.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise but under-specified, offering no value beyond the tool name. A minimal, useful description would at least reference the target id or clarify deletion semantics.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a destructive tool with no annotations and no output schema, this description is inadequate. It fails to convey the impact of deletion, any prerequisites, or what happens after deletion, leaving the agent without critical context.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 0% schema description coverage, the description must compensate for the undocumented 'id' parameter, but it does not mention it at all. The parameter's meaning (the run to be deleted) is left entirely to inference.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Delete a run' simply restates the tool name 'runs_delete' without adding any additional detail or scope. It is essentially a tautology rather than an informative definition.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool instead of sibling operations, nor does it mention prerequisites, alternatives, or exclusions. The only usage cue is the verb 'delete' in the name, which is not elaborated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden of behavioral disclosure. It does reveal key behavior (generates responses or copies/grades a column), but it omits details like return value, whether the run blocks or executes asynchronously, and any side effects. This is a meaningful but incomplete disclosure.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences, front-loaded with the core action ('Start a run'), and every clause adds value. It packs the two usage modes and a key requirement into a concise, well-structured format without redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Despite its simplicity, the description leaves gaps: no clarification of the id parameter, no return value or execution model, and no guidance on how it relates to sibling tools like runs_create. It explains the core behavior but is not fully complete for a tool with no annotations and no output schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema has 0% description coverage and the description never explains what the required integer 'id' refers to. It is not clear whether this is a run ID, dataset ID, or something else. The description adds no meaning beyond the schema's bare type definition.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses the specific verb 'Start a run' and clearly identifies the resource. It distinguishes this tool from siblings by explaining the two modes: generating responses with a prompt, or copying the graded dataset column and grading it for score-only runs, which makes its purpose unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear context by stating it is 'Required for every run, including score-only runs', which tells when to use this tool. However, it does not explicitly name alternatives or exclusions relative to sibling tools like runs_rerun or runs_regrade, so it stops short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the behavioral burden and discloses that the response includes a 'metric_averages' field with a precise structure (average score, rows graded, low-scored count). It does not mention error handling or authentication, but for a simple GET operation, this is adequate.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences long, front-loaded with the core action, and every phrase earns its place. It avoids redundancy and is highly efficient.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity (one parameter, no output schema), the description adequately covers the main return feature (metric_averages) and the primary use case. It does not list all return fields, but that is not necessary for a get-by-ID tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The description echoes the schema's single 'id' parameter by saying 'by ID', but adds no extra meaning about what the ID is or its constraints. Since schema description coverage is 0%, the description provides minimal compensation, though the parameter is straightforward.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's action ('Get a run by ID') and resource ('run'), and distinguishes it from sibling tools like runs_list by highlighting the unique 'metric_averages' breakdown. It uses a specific verb and resource, making the purpose unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides an explicit use case ('Use this to find the metric dragging a prompt down without listing responses'), which tells the agent when to use this tool. It implies an alternative (listing responses) but does not name a specific sibling tool, so it lacks explicit exclusion guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. 'List' implies a read-only operation, but the description does not explicitly confirm safety, mention return format, or disclose any behavioral limitations. It adds minimal context beyond the literal action.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, concise phrase 'List all runs' with no redundant words. It is front-loaded and every word contributes to the meaning, making it highly efficient.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a zero-parameter tool with no output schema, the description is nearly complete. It tells the agent exactly what the tool does, but could provide more context about what a 'run' is or confirm that no filters exist. Otherwise, it is adequate for the tool's simplicity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters and schema coverage is 100%, so the description does not need to explain parameters. The baseline for 0-parameter tools is 4, and the description adequately conveys that no arguments are needed.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'List all runs' uses a specific verb (list), resource (runs), and scope (all), clearly distinguishing it from siblings like runs_get, runs_create, and other list tools. It is unambiguous and action-oriented.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance about when to use this tool versus alternatives such as runs_get for a single run or other list tools. The description does not mention any exclusions, prerequisites, or alternative tools, leaving the agent to infer usage solely from the name.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description highlights key behavior: it re-grades without regenerating. With no annotations, this gives the agent essential understanding of the operation's scope and side effects.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely concise with two sentences, no superfluous words, and front-loads the key action and condition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple single-parameter tool with no output schema, the description covers the core purpose and usage context adequately. It could mention return value or limitations, but overall it is fairly complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The only parameter 'id' (integer) is not described in the schema (0% coverage). The description implicitly suggests it's the run ID ('a run's existing responses'), but does not explicitly state its meaning or format.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 're-grade' and the resource 'run's existing responses' with current metrics, distinguishing it from similar tools like runs_generate or runs_rerun.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

It explicitly advises using this tool after attaching or editing metrics, providing clear context. It does not explicitly state when not to use or compare to alternatives, but the guidance is sufficient.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description must cover behavioral traits. It indicates a mutation (creates a new run) and lists replicated elements, but does not disclose potential side effects, permissions required, or whether the original run is affected. Adequate but lacking detail.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is only two sentences, front-loaded with the action, and each sentence adds value. There is no redundancy or unnecessary information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple tool with one parameter and no output schema, the description covers the core functionality and use case. It could clarify whether the new run is automatically started or just created, but overall it is sufficiently complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must add meaning for the single parameter 'id'. The description implies id is the run to copy, but does not explicitly define it or explain where to obtain it. This adds some meaning but not comprehensive guidance.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb (Create and start), resource (a fresh copy of a run), and scope (same prompt, dataset, metrics, and settings). It distinguishes this tool from siblings like runs_update or runs_regrade by emphasizing a clean copy rather than mixing versions.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly tells when to use this tool: 'Use when the judge changed and you want a clean run instead of mixing versions.' It provides clear context but does not name specific alternative tools, leaving some guidance implicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries full burden. It only describes the basic action but lacks behavioral details about run status changes, output, auth requirements, or side effects. Minimal transparency.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Single sentence with no fluff, directly states the core functionality. Information is front-loaded and every word earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Adequate for a simple retry tool with no output schema, but lacks details on result (e.g., returns updated run, modifies in place) and prerequisites. Could be more complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so description must compensate. It explains the 'only' parameter as optionally limiting to specific response IDs, but does not explicitly describe the 'id' parameter (the run ID). Partial added value.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool re-runs only failed responses of a run, which is a specific verb and resource. It distinguishes from sibling 'runs_rerun' which likely reruns all responses.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage for partial retry of failed responses via the 'only' parameter, but does not explicitly compare with alternatives like runs_rerun or runs_regrade. No when-to-use or when-not-to-use guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden of behavioral disclosure, but 'Update a run' reveals nothing about side effects, permissions, reversibility, or what happens to the run's data. Although some parameter descriptions in the schema mention behaviors like temperature_ignored flag, the main description itself is silent.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely short ('Update a run'), which avoids verbosity but is under-specified. It simply restates the tool name without adding useful detail, so it is not effectively concise—it is just minimal.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool has 12 parameters, no output schema, and no annotations, the description is grossly inadequate. It fails to explain typical use cases, the effect of updating a run, or any operational context, making it nearly useless for an agent deciding whether and how to invoke the tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The description adds no meaning for any of the 12 parameters. Schema coverage is only 33%, so the description was expected to compensate for the undocumented parameters, but it provides zero parameter-related information.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Update a run' clearly identifies the verb (update) and resource (run), but provides no scope or specifics about what 'update' entails. It does not distinguish from sibling tools like runs_rerun or runs_regrade, which also modify runs, leaving the agent unsure of the exact purpose.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives no guidance on when to use this tool versus alternatives such as runs_regrade or runs_rerun. No context, prerequisites, or exclusions are provided, leaving the agent without direction for tool selection.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the burden. It adds one behavioral detail—'Color is auto-assigned'—which is helpful context beyond the schema. However, it does not disclose other traits like permissions, uniqueness constraints, return values, or idempotency.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two short sentences with no redundancy. Every word adds value: the first states the purpose, the second adds a behavioral note. It is front-loaded and appropriately sized for a simple tool.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with one parameter and no output schema, the description is reasonably complete but lacks context about the return value, any constraints on the name, or post-creation behavior. The auto-color note is useful, but the agent is left guessing about what response to expect.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0% and the description does not mention the 'name' parameter at all. It adds no meaning beyond the schema's minimal type definition. The agent gets no guidance on name format, uniqueness, or requiredness beyond the schema's 'required' flag.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: 'Create a tag.' This clearly distinguishes it from sibling tools like tags_update, tags_delete, and tags_get. The action is unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool vs alternatives. It does not mention prerequisites, exclusions, or when not to use it, despite the existence of sibling tags_* tools. Only the verb 'Create' implies usage.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the burden of behavioral disclosure. It explicitly reveals a key side effect: 'Removes the tag from every linked metric, prompt, run, and dataset.' This goes beyond a simple 'Delete a tag' and alerts users to the cascading nature of the operation. It does not mention irreversibility or permissions, but the noted side effect is substantial.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is exactly two sentences, front-loaded with the primary action ('Delete a tag.') followed by the critical side effect. There is no unnecessary verbosity or repetition, and every word adds value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple single-parameter delete operation with no output schema, the description is complete. It states the primary action and the full scope of impact, including the removal from linked resources. It does not need to explain return values or error handling, as these are not critical for understanding the tool's purpose.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema has only one integer parameter 'id' with no description, and schema description coverage is 0%. The tool description does not explicitly define what 'id' refers to, but the phrase 'Delete a tag' and the tool name imply it is the tag's ID. This is adequate but provides no additional meaning beyond the schema field itself.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action with a specific verb and resource: 'Delete a tag.' It also details the scope of the deletion—removing the tag from every linked metric, prompt, run, and dataset—which distinguishes it from sibling tools like tags_update or tags_list.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear context on when to use this tool: when you need to delete a tag entirely and unlink it from associated resources. It does not explicitly mention alternatives or exclusions, but the action is unambiguous given the tool name and sibling list.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It only restates the operation without disclosing behavioral details like return format, error handling for missing IDs, authentication requirements, or read-only safety. The absence of any such context leaves the agent guessing about important behaviors.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is one short, front-loaded sentence with no wasted words. For a simple get-by-ID operation, this level of conciseness is appropriate and every word earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool is a simple one-parameter retrieval, so the description is minimally viable. However, with no output schema and no annotations, it omits return value details and error behavior, leaving some gaps. For such a simple operation, it is barely adequate but lacks completeness.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema contains a single integer parameter 'id' with 0% description coverage. The description merely repeats 'by ID' without adding meaning such as what the ID represents, where it comes from, or any validation rules. This is nearly tautological and does not compensate for the schema's lack of details.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Get a tag by ID' clearly specifies the action (get), the resource (tag), and the identifying parameter (ID). It distinguishes from sibling tools like tags_list, which would list all tags, by implying a targeted lookup.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no explicit guidance on when to use this tool vs alternatives, such as tags_list for listing all tags or tags_update/delete for mutations. The phrase 'by ID' hints at a scenario, but does not state exclusions or alternative tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden of behavioral disclosure, but it only states the action. It does not mention pagination, ordering, authorization requirements, or the exact response format, leaving significant behavioral traits undisclosed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence with no wasted words. It front-loads the action and resource efficiently, making it highly concise and well-structured for its simplicity.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the simplicity (no parameters, no output schema), the description is minimally adequate but incomplete. It does not clarify what the response contains (e.g., tag objects or names) or any limits, leaving room for ambiguity despite the straightforward nature.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters, so per rubric this gets a baseline of 4. The description does not need to explain any parameters since none exist, and it adds no parameter-related meaning beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'List all tags' uses a specific verb (list) and resource (tags) with a scope ('all'), clearly distinguishing it from sibling tools like tags_get which retrieves a single tag. It is unambiguous and immediately conveys the tool's function.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided about when to use this tool versus alternatives. The description does not mention exclusions, prerequisites, or point to other tools such as tags_get for specific tag retrieval. The agent is left to infer usage solely from the verb 'list'.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. It identifies a mutation ('rename') but does not disclose side effects, idempotency, required permissions, error behavior, or what happens if the tag does not exist. This is a significant transparency gap for a mutation tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single concise sentence that front-loads the core purpose. It is efficient with no wasted words, though it is terse and lacks supporting detail. For such a simple operation, the brevity is appropriate, but it does not add extra value beyond the core statement.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with no output schema and no annotations, the description is incomplete. It does not specify the effect of the operation, error handling, or whether renaming is the only update supported. Given the simple input schema, it should at least note that 'name' must be provided to rename and describe the expected outcome.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema has 0% description coverage, and the description only says 'Rename a tag,' which implies the 'name' parameter is the new name and 'id' targets the tag. However, it does not clarify whether 'name' is required for the operation, especially since the schema only marks 'id' as required. This ambiguity leaves the optionality of 'name' unexplained.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Rename a tag' is a specific verb+resource statement that clearly distinguishes this tool from siblings like tags_create, tags_delete, tags_get, and tags_list. It precisely conveys the action of changing an existing tag's name.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. It does not mention that tags_create is for new tags, tags_delete for removal, or that this is the tool for modifying existing tag names. The sibling list implies context, but the description itself gives no exclusions or alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It discloses that the tool is a read-only usage query, and details the hard-blocking behavior upon limit exhaustion. While comprehensive, it does not mention any authentication or deprecation details, but those may be implicit.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Two highly efficient sentences. First sentence states purpose and output data. Second sentence gives usage guidance and behavioral note. Front-loaded with key information, no fluff.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Despite having no output schema, the description fully explains what the tool returns and its behavioral context. For a simple zero-parameter tool, this is complete and leaves no ambiguity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters with 100% schema coverage. The description adds value by explaining what data is returned (runs and prompt fetches used, limits, remaining, reset time), which goes beyond the empty schema. Baseline for 0 params is 4, and description meets it.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'Get this organization's plan usage and limits for the current billing period' with specific resources (runs, prompt fetches) and details (used, limits, remaining, reset). It distinguishes from sibling tools like runs_create or runs_get by focusing on quota/usage rather than run operations.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly advises 'Call this to pre-check quota before starting runs' and explains the behavior when limit is reached (hard-blocked with grace band, runs fail with run_limit_reached). This provides clear when-to-use and consequence information.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.