Server Quality Checklist

Tool Scores

• execute_toolA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses permission validation and forwarding, but omits details like error handling, side effects, rate limits, or what constitutes a valid permission. It is adequate but not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise at three sentences, front-loaded with the core action, and contains no filler or redundant information. Each sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that output schema exists and parameters are well-described in schema, the description completes the picture by explaining the gateway validation and forwarding flow. It is nearly complete, though it could mention that args must match the downstream tool's inputSchema (already in schema) and that errors may come from the downstream server.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds context about the gateway role ('validates permissions then forwards'), but the parameter descriptions in the schema already explain each field sufficiently. No additional meaning beyond schema is provided.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action: 'Execute a tool on a downstream MCP server'. It specifies the resource (downstream MCP server) and distinguishes from sibling tools (list_servers, get_server_tools, get_gateway_status) which are about discovery or status, not execution.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies that this tool should be used after listing servers and tools, but it does not explicitly state prerequisites or when not to use it. There is no guidance on alternatives or context, leaving the agent to infer usage from parameter names.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_server_toolsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It adds one behavioral trait: returns only permitted tools (filtered by policy). It does not cover other aspects like error handling, prerequisites (e.g., server must exist), or whether results are cached.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise, with two sentences that front-load the purpose and immediately connect to the next step. Each sentence adds value without fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description explains the tool's output (tool definitions) and how to use them with 'execute_tool'. It is missing a mention of the prerequisite relationship with 'list_servers' (to get server names). However, for a discovery tool with an output schema, it is reasonably complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. The description adds no additional meaning to the parameters beyond what the schema already provides. It does not elaborate on 'agent_id', 'names', 'pattern', or 'max_schema_tokens'.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: discover tools available on a downstream MCP server. It specifies the resource ('tools') and the action ('discover'), and distinguishes it from siblings like 'execute_tool' (which executes) and 'list_servers' (which lists servers).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description indicates when to use the tool (when needing to know available tools) and connects it to the sibling 'execute_tool' for the next step. However, it does not explicitly state when not to use it or mention alternatives (e.g., if tools are already known).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_serversA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It mentions that access is policy-driven, implying read-only behavior, but does not disclose side effects, authentication requirements, rate limits, or data modification risks. The description is minimal in behavioral detail, leading to a mid-range score.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise: two sentences plus a numbered workflow that front-loads the core purpose. Every sentence adds value, with no redundancy or filler. It is efficiently structured for quick understanding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (2 optional parameters, output schema exists), the description provides sufficient context: it explains the tool's role in a multi-step workflow, mentions policy-based access, and outlines the next steps. It is complete enough for an agent to use this tool correctly in the intended sequence.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already provides descriptions for both parameters (agent_id, include_metadata) with 100% coverage. The description adds no additional meaning beyond naming and workflow context, so the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states the purpose: to discover downstream MCP servers available through the gateway. It uses specific verbs ('Discover') and resource ('MCP servers'), and the workflow clearly distinguishes this tool from its siblings (execute_tool, get_server_tools, get_gateway_status).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a clear workflow (1) list_servers, 2) get_server_tools, 3) execute_tool) implying when to use it as the first step. It also notes that access is determined by gateway policy rules, giving context. However, it does not explicitly state when not to use it or mention alternatives, slightly reducing the score from 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_gateway_statusA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries burden. It implies a read operation and adds the debug mode constraint, but lacks details on side effects or permissions beyond that.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no wasted words. Efficiently communicates purpose and a key usage condition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Has output schema, so return values are handled. Description covers the three aspects (status, config, diagnostics) and the debug mode condition, sufficient for a read tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so description does not need to add parameter details. The description provides no extra meaning beyond what the schema already specifies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Get gateway status, configuration state, and hot reload diagnostics' with a specific verb and resource. It distinguishes itself from sibling tools like 'execute_tool' and 'get_server_tools'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Includes a note 'Only available when debug mode is enabled,' providing explicit context for when to use the tool. Does not mention alternatives, but siblings are distinct.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.