Server Quality Checklist

Tool Scores

• read-documentationA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It clearly describes what the tool does (retrieves complete documentation content) and provides usage context, but doesn't address potential behavioral aspects like error conditions, rate limits, authentication requirements (beyond the recommendation to read auth docs), or what happens with invalid references. It adds value beyond the minimal schema but doesn't fully compensate for the lack of annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (purpose, usage guidance, recommendations, common use cases) and front-loads the core functionality. While comprehensive, some sentences could be more concise (e.g., the second paragraph repeats similar information). Overall, it's appropriately sized for a tool with important usage context, though slightly verbose in places.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's single parameter with full schema coverage and no output schema, the description provides substantial contextual information. It explains the relationship with the sibling tool, gives specific usage examples, and recommends starting points. The main gap is the lack of output format description (what 'complete documentation content' actually returns), but otherwise it's quite complete for a read-only documentation retrieval tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 100% schema description coverage for the single parameter, the baseline would be 3. However, the description adds significant value by explaining that 'documentReference parameter is typically obtained from search-documentation results' and providing concrete examples of valid references like 'Guides and FAQs/amazon-business-apis-getting-started-guide.md'. This gives practical context beyond the schema's generic description of 'Reference to the document file from search results'.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose as 'Retrieve complete documentation content using a document reference' and 'fetches the full content of documentation files identified by documentReference'. It distinguishes from its sibling tool 'search-documentation' by explaining that this tool retrieves full content while search-documentation provides references. The verb+resource combination is specific and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use this tool versus alternatives. It states that 'documentReference parameter is typically obtained from search-documentation results' and gives specific examples of when to use it (when a document contains references to other docs). It also provides 'HIGHLY RECOMMENDED' starting points and lists common use cases, giving clear context for appropriate usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search-documentationA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes the tool's behavior: semantic search returning relevant documentation chunks with relevance scores and document references, explains how to handle referenced documents, and provides important implementation guidance (e.g., OAuth prerequisites, recommended starting documents). It doesn't mention rate limits or authentication requirements beyond OAuth, but covers most behavioral aspects well.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is comprehensive but verbose, with multiple sections (DOCUMENTATION PATH SELECTION, Helpful Hints, Common use cases, Note) that could be more streamlined. While all content is relevant, it could be more front-loaded and concise. Some redundancy exists in explaining the relationship with read-documentation tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (semantic search across documentation corpus with path filtering) and no output schema, the description provides substantial context: explains result format (snippets with relevance scores and references), relationship to sibling tool, implementation guidance, and extensive usage rules. It lacks explicit information about output structure details, but covers most aspects needed for effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds significant value beyond the schema by explaining the semantics of documentationPath parameter through the detailed documentation path tree and helpful hints that guide parameter selection. It provides context about how to choose appropriate paths based on query intent, which goes beyond the schema's basic description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Search through documentation using a natural language query' and 'provides semantic search capabilities across the entire API documentation corpus.' It specifically distinguishes from its sibling tool 'read-documentation' by explaining that search-documentation returns snippets while read-documentation fetches complete documents.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides extensive guidance on when to use this tool vs alternatives, including explicit rules for documentation path selection (e.g., 'If you are searching for code generation/building applications → API Swagger Model'), when to use the sibling read-documentation tool, and common use cases. It also specifies when NOT to provide a documentation path ('If nothing is relevant, search across all the documentation paths. DO NOT provide any documentation path').

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.