Server Quality Checklist

Tool Scores

• find_apiA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided; description does not disclose read-only nature, auth needs, or rate limits. It hints at not fetching specs directly but lacks full behavioral disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with purpose, no fluff. Could potentially be slightly more concise but overall well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 6 parameters and no annotations, description does not explain return value (likely summary of candidates) or how queries interact with other parameters. Output schema exists but content unknown.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Description lists search dimensions (path, keyword, tag, module, method) but does not explain 'query' parameter (likely full-text search) or 'limit' semantics. No enum hints. With 0% schema coverage, description adds some value but leaves gaps.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool finds candidate APIs by various dimensions (path, keyword, etc.) and distinguishes from get_api_detail which retrieves full documentation. It uses specific verb and resource.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use (to find candidate APIs) and provides a follow-up action (use get_api_detail). Implicitly excludes fetching Swagger URLs directly.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_specsA
  4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, and description does not disclose any behavioral traits beyond listing modules and status (e.g., read-only nature, auth needs, rate limits). For a list tool, it's likely benign, but lacks explicit transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with core purpose and usage instruction. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no parameters, presence of output schema, and no annotations, the description is largely complete for a simple list tool. Lacks details on permissions or output structure but output schema covers that.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters in input schema; schema coverage is 100%. Baseline is 4, and description adds no parameter info as none exist.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states verb 'List' and resource 'aggregated Knife4j / Swagger modules and their loading status', differentiating from siblings by specifying not for reading API details or fetching spec URLs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says use for module discovery and diagnostics, and not for reading API details or fetching spec URLs, providing clear context. Does not name alternative tools but implies boundaries.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• refresh_specsA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. Discloses that partial failures are isolated, which adds transparency, but does not specify if the operation is safe to call repeatedly or if it requires special permissions. For a mutation (reload) tool, more detail would help.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, each providing distinct value: first states the action and a key behavior (failure isolation), second provides usage guidance. No redundant or unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Output schema exists, so return values are covered. Description explains when to use and a behavioral detail. Could mention prerequisites or side effects, but for a zero-param tool with a simple purpose, it's reasonably complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Tool has 0 parameters, and schema coverage is 100%. Description doesn't need to add parameter info. Baseline 4 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the tool forces a reload of swagger-resources and all module specs, distinguishing it from sibling tools that focus on finding, listing, or getting details of APIs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly tells when to use: when upstream docs changed or stale results. Does not mention when not to use, but context from sibling tools makes it clear that this is for reloading, not for regular browsing.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_api_detailA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description itself clarifies it is a read operation that returns expanded schemas. It does not mention side effects or permissions, which would be expected for a read tool; however, the warning about not fetching spec URLs indicates safe behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is composed of two brief, front-loaded sentences that convey all necessary information without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema, the description sufficiently covers the tool's purpose and usage. It could be enhanced by listing example parameters, but overall it is complete for its complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must explain parameters. It says 'module + exact path + HTTP method' but does not define what constitutes a valid module, path format, or HTTP method. No examples are given, leaving the agent to infer from context.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states it is the 'Preferred tool for full API documentation' to 'read the server-indexed API detail, including recursively expanded request and response schemas'. This clearly specifies the verb (read), resource (API detail), and distinguishes it from siblings like find_api.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly says to 'use module + exact path + HTTP method' and warns 'Do not fetch /v2/api-docs, /v3/api-docs, or spec URLs directly', providing clear when and when-not usage guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.