Server Quality Checklist

Tool Scores

• swagger_get_api_detailA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, covering the safety profile. The description adds meaningful behavioral details: what the response includes (basic info, parameter definitions, response examples, mock field table) and explains the 'Required' column semantics ('是' = required, '否' = optional, '?' = ambiguous). This goes beyond annotations without contradicting them.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a clear first sentence and bullet points for parameters and response sections. While it is longer than necessary, every bullet adds meaningful detail (e.g., required column semantics, response components). It is front-loaded with the primary purpose and avoids fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the output schema exists (so return values are formally specified) and annotations declare safety traits, the description covers the essential operational context: how to specify the target API (source, method, path) and what the response contains. It does not cover error scenarios or rate limits, but these are not critical for a read-only detail operation. The description is sufficiently complete for a well-annotated tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage for all three parameters, with clear explanations (e.g., method enum, path example, source from sibling tools). The description adds little beyond what the schema already provides—the source field is explained more explicitly in the schema, and the path/method examples are duplicated. Baseline 3 is appropriate when schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Get full details of a single API interface, including all parameter definitions and response examples.' This is a specific verb+resource combination that distinguishes it from sibling tools like swagger_list_sources (listing sources) and swagger_search_api (searching APIs).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly tells the agent that the 'source' parameter comes from the output of swagger_list_sources or swagger_search_api, providing context on when to use this tool. However, it does not explicitly state when not to use it or compare with alternatives, so it falls short of a perfect 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• swagger_refresh_cacheA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already cover idempotence, non-destructiveness, and non-read-only. The description adds context about force-refetch semantics (ignoring cache), that it updates the cache, and the return shape (success/failure and interface counts). This goes beyond annotations, though it doesn't disclose every possible side effect.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and well-structured: it starts with the core action, then provides a usage hint, parameter explanation, and return summary. Every sentence contributes value without redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simple signature (one optional param), full schema coverage, rich annotations, and existing output schema, the description provides sufficient context for an agent to invoke it correctly. It covers purpose, usage timing, parameter behavior, and return contents.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema has 100% description coverage for the single 'source' parameter, and the description repeats the schema text almost exactly. No additional meaning is added; the baseline of 3 for full schema coverage applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Force-refetch') and resource ('Swagger documentation data'), and it distinguishes itself from sibling tools (list, search, get detail) by focusing on cache refresh. The action is unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use the tool ('after the documentation has been updated...') but does not mention when not to use it or explicitly reference alternatives. This clear context earns a 4, lacking only the exclusion/alternative guidance for a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• swagger_search_apiA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate readOnlyHint=true and destructiveHint=false, and the description does not contradict them. It adds valuable behavioral context: search scope, default for include_deprecated, default/max limit, and response fields, giving a clear picture beyond the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with sections for parameters, response, and examples. It is lengthy but every sentence earns its place, and the front-loaded summary makes intent immediately clear. No filler or redundant content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (5 params, filters, response fields), the description is complete: it covers search behavior, filters, defaults, response content, and examples. With an output schema present and good annotations, there are no significant gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% (all parameters described in schema). The description still adds meaning beyond schema by providing example keywords, explaining the source origin from swagger_list_sources, and clarifying enum usage (HTTP method). This compensates with practical guidance.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Search API interfaces in the internal Swagger documentation' and specifies the searchable fields (name, description, path, module). This distinguishes it from siblings like swagger_list_sources (lists sources) and swagger_get_api_detail (fetches details).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides concrete usage context through examples and notes that the 'source' parameter comes from swagger_list_sources results. However, it does not explicitly state when to use this tool versus alternatives or mention exclusions, which prevents a perfect score.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• swagger_list_sourcesA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds valuable behavioral nuance beyond the annotations: it explains that fetchedAt may be null (meaning not yet loaded) and that totalInterfaces is only shown when loaded. This clarifies caching behavior and return semantics, which the annotations do not cover. No contradictions with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured: a clear opening sentence, a bulleted list of return fields, and a usage tip. Every sentence serves a purpose, and it is easy to scan.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a parameterless read-only list operation with an output schema, the description is fully complete. It explains what is returned, including edge cases, and how it fits into the broader workflow with sibling tools.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the baseline is 4. The description focuses on return fields rather than parameters, which is appropriate. It doesn't need to add parameter details because none exist.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's specific function: 'List all configured Swagger documentation sources and their cache status.' It uses a strong verb ('List') and a specific resource ('Swagger documentation sources'), and effectively distinguishes itself from siblings by noting it should be used before swagger_search_api.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit usage guidance is provided: 'Use this before calling swagger_search_api to discover which services are available.' This tells the agent exactly when to employ this tool and mentions a sibling tool as the subsequent step.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.