Server Quality Checklist

Tool Scores

• read_documentationA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes key behaviors: URL domain and format requirements, handling of long documents via truncation and chunking, and output format details (markdown with preserved structure). It could improve by mentioning rate limits or authentication needs, but covers core operational behavior well.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (Usage, URL Requirements, Example URLs, Output Format, Handling Long Documents) and front-loads the core purpose. It's appropriately sized for the tool's complexity, though the 'Handling Long Documents' section is somewhat verbose. Every sentence serves a purpose, with no wasted text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (3 parameters, no annotations, but with output schema), the description provides complete context. It covers purpose, usage constraints, URL requirements, examples, output format, and handling of edge cases (long documents). With an output schema present, it doesn't need to detail return values, and it adequately compensates for the lack of annotations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description adds minimal value beyond the schema: it mentions start_index for chunking and max_length for truncation in the 'Handling Long Documents' section, but doesn't provide additional semantic context. This meets the baseline expectation when schema coverage is complete.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Fetch and convert an AWS documentation page to markdown format.' It specifies the exact action (fetch and convert), resource (AWS documentation page), and output format (markdown). It also distinguishes from sibling tools like 'search_documentation' by focusing on retrieving specific pages rather than searching.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use this tool: for AWS documentation pages from docs.aws.amazon.com ending with .html. It also offers alternatives for handling long documents (continue reading with start_index or stop early) and implicitly contrasts with 'search_documentation' by focusing on fetching specific URLs rather than searching.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• recommendA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes what the tool does (returns recommendations in four categories), provides context about recommendation types, and explains how to interpret results. It doesn't mention rate limits, authentication needs, or error handling, but covers the core behavior well for a read-only recommendation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (Usage, Recommendation Types, When to Use, Finding New Features, Result Interpretation) and uses bullet points effectively. It's appropriately sized for the tool's complexity, though some sections could be more concise. Every sentence adds value, with no redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity, no annotations, but with an output schema (implied by 'Returns' section), the description is complete. It explains what the tool does, when to use it, how recommendations are categorized, how to interpret results, and includes practical examples. The output schema handles return value documentation, so the description appropriately focuses on usage context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents the single 'url' parameter. The description adds minimal value beyond the schema, only mentioning the parameter in the 'Args' section without additional context. The baseline score of 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get content recommendations for an AWS documentation page' and 'provides recommendations for related AWS documentation pages based on a given URL.' It distinguishes itself from sibling tools (read_documentation, search_documentation) by focusing on recommendations rather than direct content retrieval or search.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use this tool with a dedicated 'When to Use' section listing five specific scenarios, including 'After reading a documentation page to find related content' and 'To find newly released information.' It also includes a 'Finding New Features' subsection with step-by-step instructions, clearly differentiating use cases from alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_documentationA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses that the tool uses the 'official AWS Documentation Search API' and provides detailed behavioral context in the 'Search Tips' and 'Result Interpretation' sections, including how to phrase queries and what results contain. However, it doesn't mention rate limits, authentication needs, or error handling.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (Usage, Search Tips, Result Interpretation, Args, Returns) and front-loaded purpose. However, it includes some redundancy (e.g., repeating parameter info in Args that's in the schema) and could be more concise by integrating tips into the main flow.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity, 100% schema coverage, and presence of an output schema, the description is complete. It covers purpose, usage, behavioral tips, result format, and parameters, providing sufficient context for an AI agent to use the tool effectively without needing to explain return values explicitly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters fully. The description adds minimal value beyond the schema, only briefly mentioning 'search_phrase' and 'limit' in the Args section without additional semantics. Baseline 3 is appropriate when schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool 'searches across all AWS documentation for pages matching your search phrase' using the 'official AWS Documentation Search API.' It distinguishes from sibling tools by specifying this is for searching when you don't have a specific URL, unlike 'read_documentation' which likely reads specific pages, and 'recommend' which suggests content.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states 'Use it to find relevant documentation when you don't have a specific URL,' providing clear when-to-use guidance. It also distinguishes from alternatives by implying this is the tool for broad searches, while siblings like 'read_documentation' handle specific URLs.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.