Server Quality Checklist

Tool Scores

• spec_listA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and idempotentHint=true, so safety is covered. The description adds that the tool takes no arguments and returns 'a list of specifications with their IDs and domains,' providing return-format context beyond the annotations. It doesn't contradict annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with sections (When to use, Parameters, Returns) and front-loaded purpose. It is slightly repetitive (no arguments stated twice) but every sentence contributes to understanding, so minor redundancy prevents a 5.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple 0-parameter list tool with annotations present, the description is complete: it explains what the tool does, when to use it, and what it returns. No output schema exists, but the Returns section provides sufficient information about the response contents.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With zero parameters, the baseline is 4. The description explicitly reinforces 'This tool takes no arguments' and 'This tool has no parameters,' making the parameter semantics clear even though the schema already shows an empty object.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Lists all available API specifications (OpenAPI/Swagger) in the workspace' with a specific verb and resource. It distinguishes itself from siblings like spec_by_id by positioning itself as the starting point for discovery, making its purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The 'When to use' section explicitly lists scenarios (e.g., discovering APIs, answering 'What APIs do you support?', seeing scope before exploring). It doesn't name alternatives but implies them with 'before exploring further,' providing clear context without explicit exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• spec_by_idA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and idempotentHint=true, covering safety. The description adds value by specifying exactly what is returned (domain, collections with IDs, titles, tag counts, method counts) and clarifying scope ('all associated collections'). It also implicitly states it does not drill into tags. This goes beyond the annotated safety profile.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (overview, when to use, parameters, returns). Every sentence serves a purpose, and the use of bullets and headers enhances readability. It is appropriately sized for the tool's simplicity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  There is no output schema, so the description carries the full responsibility for explaining return values, and it does so thoroughly (domain and collections with specific fields). It also covers parameters and usage context. For a tool with one parameter and clear annotations, this is complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already provides full coverage (100%) for the sole parameter 'id', including its MD5 format and pattern. The description's parameter section restates this information without adding new semantic meaning. Since schema coverage is high, baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Retrieves detailed information about a specific API specification by its ID, including its domain and all associated collections.' This distinguishes it from siblings like spec_list (which lists specs) and collection_by_spec (which likely focuses on collections only). The mention of 'after discovering it via spec_list' further clarifies its role.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  A dedicated 'When to use' section provides explicit scenarios and contrasts with spec_list and drilling into tags. It clearly states when this tool is appropriate and implies alternatives by pointing to spec_list and tag-related tools. This is model-tier usage guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• endpoint_by_idA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnly and idempotent hints. The description adds that it returns a summary and lists the exact fields, clarifying the limited scope. No contradictions with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-organized with headings and bullet points, making it easy to scan. While somewhat long, each section adds practical value and there is no redundant fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter tool with no output schema, the description fully covers the return value, usage context, and parameter format. It is complete for an agent to decide when to invoke and what to expect.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers the parameter but only with 'Unique identifier'. The description adds a valuable detail: the ID is a 32-character MD5 hash, which helps the agent understand what value to provide. This goes beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it returns a quick summary of a single endpoint with specific fields (method, path, summary, deprecation status). It distinguishes itself from the `inspect` sibling tool by contrasting the summary level vs. full OpenAPI operation object.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit 'When to use' and 'Do NOT use' sections provide clear guidance, including using `inspect` for technical details. This gives the agent concrete criteria for selection versus alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• response_outlineA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the readOnlyHint and idempotentHint annotations, the description reveals key behavioral traits: it does not return the actual data, it serves as a gatekeeper for subsequent exploration, and it enforces a policy that only response_* tools may read saved files. These insights are not present in the annotations and add significant context for the agent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is longer than minimal, but it is well-structured with clear headers (When to use, When NOT to use, Parameters, Returns, Example workflow). Each section serves a distinct purpose, especially the explicit forbidden commands list. While verbose, it front-loads the core purpose and remains organized, earning a 4 rather than a 5.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, the description compensates by providing a detailed 'Returns' section listing the outline fields (type, size, lineCount, depth, structure, schemaHint, compressionHints, navigationHints). It also includes an example workflow showing how it chains with response_compress and response_slice, making the tool's role fully understandable.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with each parameter described inline. The description's Parameters section restates the same information (path from fileRef.path, maxDepth default 3, maxArrayItems default 5) without adding new semantics. Since the schema already carries the meaning, the baseline of 3 is appropriate; the description adds no extra clarity beyond the structured data.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a clear, specific statement: 'Returns a high-level structural summary of a large JSON response file... It does not return the actual data — only the shape, keys, array lengths, and hints...' This identifies the exact verb ('Returns'), resource ('JSON response file'), and distinguishes the tool from siblings like response_compress and response_slice, which manipulate data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The 'When to use' section explicitly instructs to use this tool 'immediately after invoke returns a fileRef' and labels it the 'first and mandatory step.' The 'When NOT to use' section names specific forbidden alternatives (bash, cat, head, tail, etc.) and states that only response_* tools may read saved files. This provides clear direction on when to use versus alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• tag_by_idA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already mark this as readOnly and idempotent. The description adds useful behavioral context by specifying exactly what is returned (ID, title, method count) and clarifying that it does not return the list of endpoints, which exceeds the annotation-only information.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with headings, a summary up front, and bullet points for usage guidance. It is slightly longer than necessary, but each section adds value and it remains highly scannable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Since there is no output schema, the description compensates by explaining the return fields (ID, title, method count). It also provides clear usage context and sibling differentiation, making it complete for a simple read-only lookup tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description adds valuable detail by specifying the id format as a '32-character MD5 hash ID', which is not present in the schema description, and reinforces that it is required.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool 'returns information about a single tag: its ID, title, and how many methods it contains.' It explicitly distinguishes itself from the sibling tool endpoint_by_tag, which lists endpoints within a tag.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description includes a dedicated 'When to use' section with specific scenarios and a 'Do NOT use' instruction that names the alternative (endpoint_by_tag). This directly guides the agent on tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• endpoint_by_collectionA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and idempotentHint=true. The description adds useful context about the 'regardless of their tag' behavior and details the return fields (IDs, HTTP methods, paths, summaries, deprecation status). It does not contradict annotations and provides additional insight, though it does not cover potential edge cases like pagination or missing IDs.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (overview, when to use, parameters, returns). It is concise, wastes no words, and front-loads the primary purpose. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one required parameter, no output schema), the description is complete. It explains what the tool does, when to use it, alternative tools, parameter format, and return contents. No further context seems necessary.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema only describes collectionId as 'required,' while the description specifies it is 'The 32-character MD5 hash ID of the collection,' adding format/type context beyond the schema. This adds meaningful value for correct parameter usage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states 'Lists all endpoints within a specific collection, regardless of their tag.' This is a specific verb+resource pair with a clarifying qualifier, clearly distinguishing it from the sibling tool endpoint_by_tag which filters by tag.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  A dedicated 'When to use' section lists concrete scenarios, and it explicitly directs to the alternative: 'For a filtered view by tag within a collection, use `endpoint_by_tag` instead.' This is explicit when/when-not guidance with a named alternative.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• endpoint_by_specA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint and idempotentHint, so safety is covered. The description adds value by detailing the return payload (IDs, methods, paths, summaries, deprecation status) and the comprehensive nature of the result. It doesn't mention potential pagination or error cases, but for a list-all tool this is reasonably transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (Purpose, When to use, Parameters, Returns). Every sentence adds useful information, and it is front-loaded with a concise summary. No filler content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that this is a simple one-parameter tool with annotations and no output schema, the description covers all necessary context: purpose, usage criteria, parameter format, and return value. It is fully complete for an agent to select and invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, but the schema's description is terse ('required,'). The tool description adds important semantic detail by specifying that specId is a 32-character MD5 hash ID, which helps the agent understand the expected format. This goes beyond the schema baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Lists') with a clear resource ('all endpoints across an entire API specification') and explicitly distinguishes itself from sibling tools by noting its comprehensive scope. The phrase 'spanning all collections and tags' plus the contrast with `endpoint_by_collection` and `endpoint_by_tag` makes the purpose unmistakable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description has a dedicated 'When to use' section that lists concrete scenarios (e.g., 'show me all endpoints in API X') and explicitly names alternative tools for narrower scopes. This provides clear guidance on when to choose this tool over siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• endpoint_by_tagA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint and idempotentHint, and the description complements this by specifying the return payload: 'A list of endpoints with their IDs, HTTP methods, paths, summaries, and deprecation status.' No side effects are claimed. It doesn't discuss error cases, but that's not expected for a read-only lookup.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Structured with headings, no filler. Every section adds practical information—summary, use cases, parameter format, and return values.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter read-only tool, the description covers purpose, usage timing, alternatives, parameter format, and return shape. No output schema exists, but the return list description compensates.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema only states tagId is required, but the description enriches it: 'The 32-character MD5 hash ID of the tag.' This gives the agent the exact format needed, exceeding schema baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a precise statement: 'Lists all endpoints grouped under a specific tag within a collection.' It also contrasts with tag_by_id, clarifying the tool's unique role among siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The 'When to use' section explicitly lists scenarios (having a tag ID, user queries) and gives an alternative: 'To see the tag's metadata (title, method count) without listing endpoints, use tag_by_id instead.' This is model behavior.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• infoA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnly and idempotent hints. The description adds valuable context beyond those: it notes the tool fetches 'latest_version (from GitHub)', exposes rate_limiting fields, and details the exact categories of returned data. It also gives presentation guidance for humans, showing the agent understands the output format. No contradictions with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear headers (# info, When to use, Parameters, Returns, Presentation). It front-loads the core purpose and uses bullet points for readability. Every section earns its place; the presentation guidance is practical and not redundant.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that there is no output schema, the description compensates by listing all returned categories and providing formatting advice for human-readable output. It also covers the critical note about rate limiting for agents running loops. This makes the tool fully self-contained for an agent to understand and use effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has no parameters (0 params), so baseline is 4 as per rubric. The description explicitly states 'This tool has no parameters' and 'takes no arguments', reinforcing the schema. There is no extra parameter semantics to add since there are none.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The first line clearly states the tool returns 'a comprehensive summary of the swag2mcp runtime' and enumerates specific content (version, configuration, active specs, HTTP client settings, MCP transport, auth methods, mock mode). This is a specific, resource-defined purpose that distinguishes it from sibling tools focused on specs or endpoints.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  A dedicated 'When to use' section provides explicit triggers ('user asks what's the current configuration?', 'before calling invoke in a loop', etc.). It gives clear context but does not mention exclusions or alternatives to this tool, which would warrant a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• tag_by_collectionA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint and idempotentHint, covering safety. The description adds context about the hierarchical navigation pattern and discloses the return format (list of tags with IDs, titles, method counts), providing value beyond structured data.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-organized with clear headings (overview, when to use, parameters, returns), is appropriately sized for a simple tool, and contains no redundant or filler language.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with one parameter and no output schema, this description fully covers purpose, usage, parameter semantics, and return shape. The sibling-tool context and annotations further round out the operational picture, leaving no critical gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  While schema coverage is 100%, the schema's description is merely 'required,' which provides no real meaning. The tool description compensates by explaining that collectionId is a '32-character MD5 hash ID of the collection,' offering actionable semantic detail beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Lists all tags within a specific collection,' which is a specific verb+resource+scope statement. It explicitly distinguishes from the sibling tool tag_by_spec by noting the collection-level scope versus spec-wide scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description includes a dedicated 'When to use' section with concrete scenarios (having a collection ID, hierarchical navigation, user queries) and explicitly directs users to tag_by_spec for the alternative case, satisfying both when and when-not guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• tag_by_specA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The annotations already declare readOnlyHint=true and idempotentHint=true, so the description only needs to add context beyond that. It adds a clear 'Returns' section describing the output structure (tags with IDs, titles, and method counts), which goes beyond what annotations provide. It doesn't cover pagination or errors, but for a simple read-only listing tool this is sufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description uses a clean structure with a short summary, 'When to use' bullets, a parameter list, and a returns section. Every section earns its place, with no fluff or repetition. It is concise yet complete, using clear headings and bullets for scannability.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with a single parameter, no output schema, and read-only/idempotent annotations, the description provides everything an agent needs: what it does, when to use it, the parameter format, and the return value shape. There are no significant gaps. The complexity is low, and the description fully covers the context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description for specId is minimal ('required,'), but the description enriches it by specifying 'The 32-character MD5 hash ID of the specification.' This gives concrete format details beyond the schema, adding real value. Schema coverage is 100%, and the description supplements it well, warranting a score above the baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool 'Lists all tags across an entire API specification, spanning all collections.' This specifies a concrete verb (lists), a resource (tags), and a scope (entire API spec), and it distinguishes itself from the sibling tool tag_by_collection by emphasizing the global scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides an explicit 'When to use' section with bullet points, including situations like wanting every tag or needing a global view. It also explicitly directs users to use tag_by_collection instead for single-collection queries, giving clear when-to-use and when-not-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• collection_by_idA
  4.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate `readOnlyHint: true` and `idempotentHint: true`, correctly marking it as a safe read. The description adds useful context by detailing the return payload and noting that the ID is a 32-character MD5 hash, which helps set expectations without repeating the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-organized with clear headers and succinct bullets. Every sentence adds value: it explains the purpose, provides use cases, defines the parameter, and summarizes the return value without fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple single-parameter tool with no output schema, the description fully covers what it does, when to use it, what the parameter means, and what the response contains. No critical information is missing for an agent to correctly invoke it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Although the schema covers 100% of the parameter with a description, the tool description enriches it further by specifying the exact format ('32-character MD5 hash ID') and reinforcing its purpose. This goes beyond the schema's generic 'Unique identifier' text.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Retrieves detailed information') and resource ('specific collection'), and lists the exact contents (tags, spec metadata, method statistics). It also distinguishes from the sibling tool `collection_by_spec` by specifying that this tool is for a single collection ID, not listing all collections.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The 'When to use' section provides three concrete scenarios and explicitly points to `collection_by_spec` as the alternative for listing all collections. This gives the agent clear decision guidance and prevents confusion among siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• collection_by_specA
  4.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint and idempotentHint, so the safety profile is covered. The description adds useful behavioral context beyond annotations, such as the scope ('within a specific API specification') and the return shape (list of collections with IDs, titles, and statistics). This is consistent with annotations and enriches the agent's understanding.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections: a one-sentence purpose, a 'When to use' list, a parameter definition, and a returns summary. Every sentence serves a purpose, and the most important information is front-loaded. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, no output schema), the description covers all necessary aspects: what it does, when to use it, parameter format, and return content. It also fits into the broader navigation flow with sibling tools, making it complete for an agent to select and invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema only says 'specId' with description 'required,' which is bare. The description compensates by specifying it is a 32-character MD5 hash ID, adding crucial semantic detail. Since there is only one parameter and it is fully explained, the description adds significant meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Lists') and resource ('all collections within a specific API specification'), clearly distinguishing it from siblings like collection_by_id (single collection) and tag_by_collection (tags). It also defines 'collections' as 'logical groups of endpoints.'

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The 'When to use' section explicitly states when to use this tool, including having a `specId` from `spec_list` or `spec_by_id`, user intent examples, and navigation flow. It also names alternatives (`collection_by_id`, `endpoint_by_collection`) for next steps, providing clear context and exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• inspectA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already mark this as read-only and idempotent, and the description adds valuable context: the note about auth-injected parameters being automatically handled and not to be passed. This explains critical behavior beyond the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with clear headings, bullet lists, and a concise note. Each section adds essential information without redundancy or unnecessary filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite lacking an output schema, the description fully explains the return value (full OpenAPI operation object) and includes the critical auth-injection warning. For a simple single-parameter tool with good annotations, this is complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema already covers the single parameter with 100% coverage, but the description reinforces that `endpointId` is a 32-character MD5 hash and adds usage nuance about auth parameters. The baseline is 3; the extra clarification warrants a 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Retrieves the full OpenAPI operation object for an endpoint' — a specific verb and resource. It distinguishes from sibling tool `endpoint_by_id` by noting the difference between full spec and quick summary.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit 'When to use' section lists concrete scenarios, and directly names the alternative `endpoint_by_id` for quick summaries. This gives clear guidance on choosing this tool over others.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• response_compressA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the annotations (readOnlyHint, idempotentHint), the description discloses that the tool may return either an inline body or a new fileRef if the result is still too large, and provides a hint. It also explains compression modes and default behavior, adding significant context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with headers, bullet points, and a workflow example. It is slightly verbose with duplicate parameter details, but every section serves a purpose and the front-loaded summary is clear.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Without an output schema, the description fully explains return values (body, fileRef, hint) and provides a concrete workflow example. It covers all parameter semantics and mode nuances, making it self-contained for effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The parameter section goes far beyond the schema by explaining each mode's effect, providing defaults for arrayHead (3), arrayTail (2), stringLen (80), and jsonPath (root), and detailing how selectKeys works. This is critical meaning not present in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a clear verb and resource: 'Reduces a JSON value inside a saved response file so it fits within the response size limit and can be returned to the LLM inline.' This distinguishes it from sibling tools like response_slice and response_filter by focusing on size reduction and inline return.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Contains explicit 'When to use' and 'When NOT to use' sections. It names the predecessor (response_outline) and successor (response_slice), and explicitly forbids manual file reading with external commands, directing users to this tool or response_slice instead.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• invokeA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description fully discloses behavioral traits: automatic auth, auth-injected parameters, large-response disk saving, one-at-a-time invocation, bounded retries, and rate-limit handling. This is extensive and far exceeds the minimum burden.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear headers, bullets, and front-loaded purpose. Every sentence serves a concrete purpose, with no redundancy or repetition of schema definitions.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (dynamic endpoint invocation, auth, rate limits, large responses), the description covers all necessary aspects: when to use, prerequisites, parameter roles, auth, error handling, and rate limits. Without an output schema, it still explains return values and disk-saved responses adequately.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, but the description adds critical semantics: clarifies `parameters` includes path/query/header only, `requestBody` for POST/PUT/PATCH, and explicitly warns against passing auth-related params. This prevents common misuse and goes well beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The first sentence clearly states the tool executes a real API call and returns response data, status code, and headers. This distinguishes it from sibling tools like `inspect` and `info`, which are read/analysis tools, while `invoke` performs the actual action.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The 'When to use' section explicitly limits usage to explicit user action requests, provides example phrases, and mandates `inspect` first. It also excludes destructive operations without confirmation and points to `info` for rate-limit checks, offering clear alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• response_filterA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already mark readOnlyHint and idempotentHint, and the description adds substantial behavioral context: processing strategy (memory vs streaming), return fields, filter operator syntax, gjson path limitations (recursive descent not supported), and case-insensitive substring search. No contradictions with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear headers (When to use, When NOT to use, Parameters, Returns, Examples) and a front-loaded one-sentence summary. The length is justified by the tool's complexity, and every sentence adds value—no fluff or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite no output schema, the description includes a full 'Returns' section explaining each field (page, pageSize, total, totalPages, items, strategy). It also covers gjson syntax caveats, filter operators, and warnings about not reading files manually, making the description complete for a tool of this complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Although schema coverage is 100%, the description enriches every parameter: it documents defaults (page=1, pageSize=10), max pageSize (50), gjson syntax examples, supported filter operators, and clarifies that jsonPath can be empty for root arrays. This goes well beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb+resource: 'Filters, searches, and paginates through arrays in saved JSON response files.' This clearly distinguishes it from siblings like response_slice, response_outline, and response_compress by covering three distinct operations on saved response arrays.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides an explicit 'When to use' list (find specific items, filter by condition, paginate, response too large for response_slice) and a 'When NOT to use' section that forbids alternative tools like bash/cat/head. It even names response_slice as the alternative for manual inspection, giving clear usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• response_sliceA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the readOnly/idempotent annotations, it documents return fields (slice.lines, slice.value, slice.fragment, slice.context, slice.isComplete, slice.nextPath/prevPath) and edge behaviors such as root array returning first element, default around=20, and fileRef only when size limit is exceeded.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a clear purpose, usage criteria, parameter explanations, return values, and a concrete example. It front-loads the main purpose and keeps every section purposeful.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no output schema, the description fully explains inputs, defaults, return fields, and navigation. It also covers when not to use the tool and provides a working example, making it complete for an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, yet the description adds gjson dotted syntax, @this sentinel, root-array behavior, line/range/around defaults, and example path expressions. This significantly exceeds the schema's brief property descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Extracts a specific fragment of a saved JSON response file by logical jsonPath or by line range,' providing a specific verb, resource, and method. It clearly distinguishes itself from response_outline/response_filter by focusing on direct extraction of a known fragment.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Contains explicit 'When to use' and 'When NOT to use' sections. It instructs agents to use it when a specific object/array/field is already identified, prefer jsonPath over line numbers, and prohibits using shell commands to read the file.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• searchA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Even with readOnlyHint and idempotentHint annotations, the description adds rich behavioral context: query syntax (field filters, boolean operators, fuzzy, wildcards, phrases), path prefix matching nuance (e.g., 'path:/api/v3/*' matches only prefixes), the AND behavior for queries with '/', and the return format (list with IDs, methods, paths, summaries).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is long but impeccably structured with headers, bullet lists, a table of examples, and clear sections. Every sentence contributes to understanding the tool, and the formatting makes it scannable without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple 2-parameter tool with no output schema, the description fully covers what the tool does, how to use it, what parameters mean, and what it returns (list of endpoints with IDs, methods, paths, summaries). No critical information is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Although schema coverage is 100%, the description vastly expands parameter semantics. It details query syntax (method:, tag:, path:, boolean +/-, fuzzy ~, wildcards *, phrases), provides explicit examples for each filter, and adds min/max constraints for the limit parameter not present in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a definitive statement: 'The ONLY tool for finding endpoints when you don't have the endpoint ID.' It clearly specifies the action (searches endpoints) and resource (across all specifications), and distinguishes itself from sibling tools like endpoint_by_tag and endpoint_by_collection by stating those are for navigation after you already know what you're looking for.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The 'When to use' section provides explicit triggers, and the 'DO NOT' section lists alternatives and warns against manual traversal. The user intent-to-search-query examples further clarify when and how to apply the tool in practical scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.