Server Quality Checklist

Tool Scores

• gateway_statusB
  3.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations available, the description carries the burden of behavioral disclosure. It indicates a read-only operation ('view') and lists the data shown, which is helpful. However, it omits any mention of permissions, rate limits, or potential side effects, leaving some ambiguity for a status tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence that states the purpose directly without any redundant words. It is front-loaded with the action and target, making it easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  While the tool appears simple and an output schema exists, the complete absence of parameter semantics and usage guidance leaves gaps. The description covers the core purpose but not the operational context needed for an agent to invoke it correctly with optional parameters.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description mentions none of the three parameters (server, profile, verbose). The agent gets no explanation of their meaning or effect, so the description fails to compensate for the missing parameter details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool views the health status, prefix, and tool count of configured gateway sub-services. It uses a specific verb ('view') and identifies a distinct resource, differentiating it from sibling tools that search, describe, or call specific tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like search_gateway_tools or describe_gateway_tool. There are no exclusions or contextual hints about appropriate scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_gateway_toolsA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must carry the burden. It adds behavioral context about output format (compact results, filtering to 'actually callable' tools). Yet it omits behaviors like whether force_refresh bypasses caching, how server/profile scope the search, or any auth/error conditions. It provides some value but not complete transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence, action-first, and includes a specific list of return fields. Every word earns its place with no fluff or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 4 parameters and no annotations, the description is insufficient. It explains purpose and return format but fails to cover parameter usage and when to use this vs siblings. The output schema exists, but the input side is under-specified, making it hard to use correctly without external knowledge.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It doesn't name or explain any input parameters. Only the query parameter is inferable from the tool name. server, profile, and force_refresh are completely unexplained, leaving the agent to guess their effect.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action '搜索' (search) and the resource 'gateway 下真实可调用的工具' (actually callable gateway tools), distinguishing it from siblings like describe_gateway_tool or call_gateway_tool. It also specifies the return scope (compact results with tool name, description, required params).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage: use when you need to discover callable tools under the gateway. However, it offers no explicit exclusions or alternatives, such as 'use describe_gateway_tool for details of a specific tool'. The context is clear but not fully guided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• call_gateway_toolA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full responsibility for behavioral disclosure. It does not mention that calling an arbitrary gateway tool may have side effects, require authorization, or produce errors. The description is minimal and only explains the source of the name parameter, leaving the agent unaware of the potentially operationally impactful behavior of invoking an arbitrary tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence with no filler. It front-loads the core action and immediately provides the critical constraint on the 'name' parameter. Every word adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is a generic gateway invoker, and the description covers the main idea, names the discovery tools, and clarifies the arguments parameter. However, it omits any mention of 'max_chars' and does not warn that the underlying tool's behavior may be unknown or potentially destructive. Given the output schema exists, return value explanation is unnecessary, but the description still leaves important contextual gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 0%, so the description must compensate. It adds meaning for 'name' (must come from search/describe) and 'arguments' (real parameter dictionary of the tool), but does not explain 'max_chars' at all. The partial coverage of 2 out of 3 parameters brings the score above baseline but leaves a significant gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: to call a real tool under the gateway. It uses a specific verb ('call') and resource ('a real tool under gateway'), and distinguishes itself from sibling tools like search_gateway_tools and describe_gateway_tool by indicating it performs the actual invocation rather than discovery.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides usage guidance by stating that the 'name' parameter comes from search_gateway_tools or describe_gateway_tool, implying the correct workflow: first discover, then invoke. It does not explicitly state 'when not to use' but the reference to sibling tools makes the alternative path clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• describe_gateway_toolA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It correctly implies the tool is read-only ('查看') and focuses on schema inspection rather than execution, which is useful. But it does not disclose the effect of the include_output_schema flag, error behavior, or return format details beyond the schema, leaving gaps for a no-annotation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single two-sentence Chinese sentence that front-loads the core purpose ('按工具名查看单个 gateway 工具的详细 schema') and then adds a concise workflow. Every word earns its place, with no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has only 2 parameters and an output schema, and the description provides a clear workflow. However, the lack of any mention of the optional include_output_schema parameter and no guidance on expected output beyond the schema leaves it incomplete for a user trying to fully leverage the tool's options.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It explains the 'name' parameter via '按工具名', but it completely omits the 'include_output_schema' parameter, which is a boolean with a default. Thus, only half the parameters are addressed, and the description adds limited value over the raw schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: view the detailed schema of a single gateway tool by name. It distinguishes itself from siblings like search_gateway_tools (searching) and call_gateway_tool (invoking), making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It gives explicit workflow guidance: '先用 search_gateway_tools 搜，再按需 describe' (first search with search_gateway_tools, then describe as needed), which tells the user when to use this tool relative to the search tool. However, it does not explicitly exclude call_gateway_tool or mention when not to use it, so it slightly misses full when-not coverage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.