oasdiff

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries full burden for behavioral disclosure. It states the tool returns 'breaking and warning-level changes', and notes the limitation about unresolved external $refs. However, it does not describe error behavior for invalid specs or the exact output format. Slightly more detail on the response structure would improve transparency.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise and well-structured. It starts with the core purpose and return type, then provides usage guidance and alternatives, and ends with constraints. Every sentence adds value and there is no redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity of the tool (comparing two OpenAPI specs for breaking changes) and the absence of an output schema, the description covers purpose, return type, usage, sibling distinctions, and input constraints. It does not detail the output format or error handling, but overall it is sufficiently complete for an agent to select and invoke the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, with both parameters described as strings. The description adds meaning by specifying they can be YAML or JSON and that they must be valid, self-contained OpenAPI (no external $ref resolution). This significantly adds context beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: detecting breaking API changes between two OpenAPI specifications. It specifies the verb ('detect'), resource ('breaking API changes'), and scope ('between two OpenAPI specifications'). The description also distinguishes from siblings by naming alternative tools and their use cases.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicit usage guidance is provided: 'Use this when the question is whether an API change breaks existing clients.' The description also lists when to use alternatives (oasdiff_changelog, oasdiff_diff, oasdiff_validate). Additionally, it states prerequisites: both specs must be valid, self-contained OpenAPI with no external $ref resolution.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations provided, so description carries full burden. It correctly indicates the tool is read-only (produces a changelog) and warns about $ref limitations. Could mention potential performance or rate limits, but is otherwise clear about its non-destructive, analytical nature.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Two efficient sentences: first states purpose and scope, second gives usage guidance and constraints. No wasted words, front-loaded with key information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (two input params, no output schema) and sibling tools explained, the description is largely complete. It covers purpose, when to use, and constraints. Missing a description of output format (e.g., 'returns a Markdown changelog'), but overall adequate.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with descriptions for both params. Description adds the constraint that they must be valid, self-contained OpenAPI (YAML or JSON), which goes beyond the schema. A score of 4 reflects that it adds meaningful context beyond the schema's basic type/description.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool produces a full changelog of every change between two OpenAPI specs, down to informational changes. It specifies the verb 'produce', the resource 'changelog', and distinguishes from siblings oasdiff_breaking_changes and oasdiff_diff.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly says when to use this tool ('when you want the complete set of changes at every severity') and when to use alternatives ('oasdiff_breaking_changes instead when you only care whether the change is breaking', 'oasdiff_diff for the raw structural diff'). Also provides prerequisites: both specs must be valid, self-contained OpenAPI.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations present, so description bears full responsibility. It discloses constraints: both specs must be valid, self-contained OpenAPI, no external $ref resolution. It does not explicitly state the tool is read-only, but that is implied. Could mention output format but acceptable.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Three concise sentences that efficiently convey purpose, usage, constraints, and alternatives. No unnecessary words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Covers all essential aspects: purpose, usage, constraints, and sibling differentiation. Lacks output description, but given the tool's simplicity and no output schema, it is mostly complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, baseline 3. Description adds format clarification ('as YAML or JSON') and constraints (no external $refs) that augment the schema's property descriptions, justifying a higher score.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool shows 'structural diff between two OpenAPI specifications' and specifies 'what was added, removed, or modified'. It distinguishes from siblings by explicitly naming and contrasting with oasdiff_breaking_changes and oasdiff_changelog.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides explicit guidance: 'Use this when you want the raw structural changes rather than a backward-compatibility verdict' and directs to sibling tools for other purposes. No ambiguity.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are present, so the description carries the burden. It discloses the important constraint that the spec must be self-contained (no external $ref resolution). Returns findings (empty when valid). While it does not detail the exact format of findings or error handling, for a validation tool this is reasonably transparent.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is very concise: four sentences, each earning its place. It starts with the main purpose, then usage guidelines, then a constraint. There is no wasted text. Structurally excellent.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool has one parameter, no output schema, and no annotations, the description covers the key aspects: what it does, when to use it, input format (YAML/JSON), a constraint (self-contained), and expected output (findings, empty if valid). It does not explain the structure of findings or error messages, but for a validation tool this is adequate.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The single parameter 'spec' is fully described in the schema (100% coverage). The description adds 'Pass the spec as YAML or JSON' which is redundant with the schema. It also mentions the self-contained constraint, which is about the spec content rather than parameter semantics. Hence, the description adds little beyond the schema, so baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

Purpose is clearly stated: 'Validate a single OpenAPI specification against the OpenAPI and JSON Schema rules.' It uses a specific verb (validate) and resource (single OpenAPI spec). It also distinguishes itself from siblings by explicitly mentioning that for comparing two specs, one should use other tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicit usage guidance is provided: 'Use this when you have one spec and want to check it is well-formed.' It also gives alternatives for different needs: 'For comparing two specs, use oasdiff_breaking_changes, oasdiff_changelog, or oasdiff_diff instead.' This is excellent.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.