Server Quality Checklist

Tool Scores

• scanB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It states it tests connections but does not disclose whether the tool is read-only, modifies anything, has rate limits, or handles errors. The behavioral traits are insufficiently explained.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence, front-loaded with the main action, and contains no filler. Every word earns its place, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has no output schema, yet the description does not explain what the tool returns (e.g., list of configs, success/failure status). For a no-parameter tool, the description should at least hint at the output format. This omission makes it incomplete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has no parameters, and the schema is empty (100% coverage). Per guidelines, 0 parameters yields a baseline score of 4. The description adds no parameter information, but none is needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool discovers MCP server configs and tests connections via JSON-RPC handshake. The verb 'Discover' is specific, and the scope is defined as 'all MCP server configs', which distinguishes it from sibling tools like bench, doctor, and security.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not provide any guidance on when to use this tool versus alternatives. There is no mention of prerequisites, context, or exclusions, leaving the agent to infer usage without explicit direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• benchA
  3.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description carries full behavioral burden. It only states the action but does not disclose if it is read-only, sends requests, or impacts server performance.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One efficient sentence, front-loaded with the verb 'benchmark', and no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a parameterless tool, the description is mostly complete. It clearly states what it does and its scope, though it could mention output format or side effects.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters and 100% schema coverage, so the description does not need to add parameter info. Baseline 4 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool benchmarks response latency for all configured MCP servers. It distinguishes from siblings like 'doctor', 'scan', 'security' by focusing on latency measurement.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for latency benchmarking but provides no guidance on when to use this tool vs alternatives, nor does it mention any prerequisites or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• doctorA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It states it performs scans, audits, and benchmarks, but does not disclose side effects, permissions, or whether it is read-only. Adequate but lacks depth beyond the actions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, front-loaded with purpose, and no extraneous words. Every part is necessary and clear.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no parameters and no output schema, the description adequately explains what it does and its scope. Could be slightly improved by mentioning result format or prerequisites, but is mostly complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema has 0 parameters, so baseline is 4. Description correctly implies no parameters are needed, adding no further value but also no deficit.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses specific verbs and resources: 'scan connections, audit security, and benchmark latency' for 'all configured MCP servers'. It clearly distinguishes from sibling tools like 'bench', 'scan', and 'security' by stating it runs all checks at once.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Implied usage is to run a comprehensive check, but no explicit when-to-use or when-not-to-use compared to individual sibling tools (bench, scan, security). No alternatives or exclusions are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• securityA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses what issues the tool checks for (hardcoded secrets, tokens, shell injection), which is helpful. However, with no annotations, it does not explicitly state whether the tool is read-only or has side effects, nor does it mention authorization requirements or output format.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence of 16 words that front-loads the purpose with a strong verb ('Audit') and provides concrete examples. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a parameterless audit tool, the description is fairly complete: it states the target (MCP server configs), the action (audit), and examples of issues. However, it does not describe the output format or how results are presented, which would be beneficial but not critical given the tool's simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has zero parameters and 100% coverage, so the schema fully documents the lack of parameters. The description adds no param info, which is acceptable; baseline for 0 params is 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: auditing MCP server configs for security issues like hardcoded secrets, tokens in args, and shell injection patterns. This is specific and distinguishes it from sibling tools (bench, doctor, scan) which likely cover performance, health, and other scans.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for security auditing but provides no explicit guidance on when to use this tool versus alternatives, nor any conditions or prerequisites. The context is clear but lacks exclusions or comparison.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.