Server Quality Checklist

Tool Scores

• docs_searchA
  3.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and destructiveHint=false. The description adds context that it is a search operation and provides instructions for usage, which is consistent with annotations. No contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is somewhat verbose, starting with an instruction block before the actual purpose. While it is clear, it could be more concise by front-loading the purpose and removing the meta-instruction format.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite annotations and high schema coverage, the description lacks details about what the tool returns (e.g., page links, titles, summaries). This is a significant gap for a search tool with no output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so both parameters are well documented in the schema. The description minimally adds value by repeating the 'site' parameter's purpose and giving an example, but this does not significantly enhance understanding beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it searches for specific topics within a documentation site. It mentions knowing the domain first, which implies a contrast to exploration tools like docs_discover, but does not explicitly differentiate from siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit when-to-use context: 'when you ALREADY know the documentation domain and need to find a specific page.' It also advises providing the 'site' parameter and gives an example. However, it does not clearly state when not to use or list alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• docs_mapA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description includes '[READ-ONLY]' which aligns with annotations (readOnlyHint: true). It adds that it returns a 'complete table of contents with every documentation page organized by section', providing behavioral context beyond annotations. However, it does not detail potential limitations or error conditions, which would be needed for a higher score.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded with the key instruction and purpose. It uses two clear sentences without unnecessary words. The structure is logical: instructional note, purpose, example, and guidance. It could be slightly more streamlined but is very efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, no output schema), the description explains the output type (table of contents organized by section) and provides a use case. It is complete enough for an agent to understand what to expect. Minor gaps like output format specifics are acceptable without an output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with a clear description of the 'domain' parameter including examples. The description adds no additional semantic value beyond what is already in the schema, so the score is at the baseline of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool maps the complete documentation structure of any domain, and explicitly distinguishes its role as the first step when exploring a new API. The phrase 'Map the complete documentation structure' combined with the example usage provides a specific verb and resource, setting it apart from siblings like docs_read and docs_search.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly instructs to use this tool as the FIRST step when exploring a new API, and advises subsequent use of docs.read on specific pages. This provides clear when-to-use guidance and mentions the follow-up tool. It could be improved by explicitly stating when not to use it (e.g., if the user already knows the structure), but it is still strong.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• docs_discoverA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and destructiveHint=false. The description adds behavioral context: it is read-only and should not be called in rapid loops, providing valuable guidance beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is reasonably concise and front-loaded with the key instruction. Each sentence adds value, though it could be slightly streamlined without losing important guidance.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description explains the tool's input well but does not describe the output format (e.g., what a documentation source looks like). Given the simple parameter set and annotations, the missing return description leaves some ambiguity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% coverage with a description for the query parameter. The description adds rich semantic guidance: use specific, descriptive queries, combine intents, and provides an example, significantly enhancing understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Find documentation sources for any service, library, or API'. It specifies it should be the first step, distinguishing it from sibling tools like docs_search which might be for more specific searches.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage instructions: 'Use this as the FIRST step', examples of good queries, and a warning against rapid loops. It lacks explicit alternatives but the context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• docs_extractA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and destructiveHint=false. The description adds context by noting it is 'READ-ONLY' and explains the dual behavior (summary without tag, filtered endpoints with tag). This goes beyond the safety profile to clarify the tool's operational characteristics.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise (two sentences) and well-structured. It front-loads the usage condition and then explains behavior. The '[INSTRUCTIONS]' prefix is slightly redundant but not detrimental. No unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description covers the main use cases (with/without tag, filtering by method/search). It could mention the output format but is adequate for an agent to understand the tool's purpose and behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% so parameters are well-documented. The description adds value by explaining that omitting 'tag' returns a summary of all tags with endpoint counts, which is not in the schema. This clarifies default behavior and enhances interpretability.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool extracts structured endpoint information from OpenAPI, Swagger, or Postman specs. It specifies the action (extract), resource (endpoint info from specs), and scope (filtering by tag). This distinguishes it from sibling tools like docs_read (which likely reads raw content) and docs_search (which searches).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly instructs to use 'ONLY when docs.read has identified an OpenAPI/Swagger spec URL or Postman Collection.' It also explains behavior when tag is provided vs. not, giving clear guidance on when to use and what to expect.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• docs_readA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations declare readOnlyHint=true and destructiveHint=false, and the description adds specific behaviors: handles multiple formats (HTML, JSON, YAML, etc.), PDFs under 5MB, and returns related documentation links. Fully consistent with annotations and provides extra detail.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two paragraphs: one explaining functionality and one giving instructions. It is mostly concise but could be slightly more streamlined. The key information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a read tool with no output schema, the description fully explains the output format (Markdown) and includes handling of multiple input types and extraction of related links. Also provides actionable instructions for the agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema has 100% coverage with descriptions for both parameters. The description adds value by explaining the tool returns Markdown and handles various input formats, which is not in the schema. Slight redundancy but acceptable.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool reads documentation pages and returns clean Markdown, listing supported formats. It distinguishes from sibling tools like docs_discover and docs_search by focusing on reading a specific page.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly instructs to 'ALWAYS check these links for authentication and getting-started pages before generating code,' providing clear guidance on when and how to use the tool's output. Implicitly suggests using this tool for reading docs rather than discovering or extracting.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.