Server Quality Checklist

Tool Scores

• list_sourcesA
  4.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses the tool's scope and even its deliberate omissions, but it does not explicitly state side-effect-freedom or other behavioral details; the 'List' framing implies a read operation without being emphatic.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences front-load the purpose and immediately give the usage context. All every sentence earns its place; there is no repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has no arguments and an output schema exists, so the description only needs to say what the tool is for and when to call it, which it does completely.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so there are no parameter ambiguities for the description to resolve.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: 'List the sources this server covers, their limits, and what it deliberately omits.' This clearly distinguishes the tool from the search/read sibling tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says 'Call this when unsure whether a lookup belongs here or with the official AWS Knowledge MCP Server,' giving a direct when-to-use condition and pointing to the appropriate alternative.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_aws_contentA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations available, the description carries the full burden of behavioral disclosure and does so thoroughly. It explains host-dependent behavior, which sources return raw markdown versus HTML conversion, that builder.aws.com bodies are excluded, and that long pages are truncated with a start_index marker for resumption.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact, front-loads the core action, and uses a scannable bullet list for the host-specific dispatch behavior. Every sentence contributes useful information and none are wasted on filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with three parameters, a clear output schema, and no annotations, the description is complete enough for an agent to invoke correctly. It covers all important call-time behaviors: supported host patterns, content transformation, truncation, and continuation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the parameters are already well documented. The description adds meaningful context by relating max_length and start_index to the truncation behavior and the resume marker, going slightly beyond the schema without restating it.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: "Fetch a page from a supported source and return it as markdown." It then confirms differentiation with concrete host-specific behaviors, so an agent can distinguish it from search_aws_knowledge and list_sources without guessing.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use the tool—when you already have a URL from a supported source—and the schema documents that the URL comes from search_aws_knowledge. However, it never explicitly states the workflow relationship to its siblings or says when not to use it, leaving some routing to inference.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_aws_knowledgeA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations present, the description fully carries the behavioral burden. It discloses that sources are queried in parallel, that a slow or broken source degrades to an entry in errors rather than failing the call, and that snippets are intentionally shorter. It also sets expectations by directing the agent to read_aws_content for full content. This is strong behavioral context beyond what any schema could imply.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Despite being longer than average, every sentence earns its place. The scope statement is front-loaded, then the when-to-use and when-not-to-use are clearly separated, and the parallel-query behavior and handoff to read_aws_content are provided without fluff. There is no redundant restatement of the input schema or output format.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a search tool with three parameters, one required, an output schema, and two sibling tools, the description covers everything needed to invoke it correctly: the exact sources, the scoping boundary, the failure behavior, and the follow-up step. The only thing not described in detail is return formatting, which the output schema already documents, so the description is sufficiently complete without pretending to replace structured schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3, but the description adds meaningful extra semantics. It explains what each source ('repost', 'kiro', 'builder') means, makes clear that 'plain keywords' are intended, and ties sources to the use-case list. It doesn't merely repeat the schema descriptions; it gives conceptual grounding and selection guidance.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a crisp scoping statement: 'Search AWS knowledge sources that the official AWS Knowledge MCP Server does not cover.' It names the specific sources (repost, kiro, builder) and explicitly differentiates from the sibling tool read_aws_content by saying to pass promising URLs to that tool. An agent can immediately grasp what this tool searches and what it does not.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit when-to-use and when-not-to-use guidance, including examples: troubleshooting threads for the same error (repost), questions about Kiro itself (kiro), and recent community write-ups (builder). It explicitly says NOT to use this tool for official AWS documentation, API references, blogs, What's New, Well-Architected guidance, or re:Post Knowledge Center articles because the official AWS Knowledge MCP Server indexes those and is better. This is exactly the kind of selective routing guidance an agent needs.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.