Server Quality Checklist

Tool Scores

• probe_connectA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations indicate this tool is not read-only (readOnlyHint=false), and the description states it 'opens a connection' implying persistence. However, it does not elaborate on side effects, resource management, or behavioral differences between transport types beyond parameter descriptions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence that front-loads the key action and output, with no unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (5 parameters, no output schema), the description covers the main output but lacks details on error handling, timeouts, or cleanup. Still, it provides sufficient context for basic use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so parameter details are fully documented in the schema. The description adds no further semantic information beyond referencing the transport types.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: opening a connection to an MCP target server and returning specific information (connectionId, server name, version, etc.). It distinguishes well from siblings like probe_disconnect, probe_lint, etc.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when needing to connect to an MCP server, but does not explicitly state when to use or avoid this tool compared to alternatives. No prerequisites or limitations are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• probe_reportA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations provide readOnlyHint=false and openWorldHint=true, indicating possible side effects. The description adds context about running optional fuzz and defaults, but does not elaborate on behavioral traits like destruction, auth needs, or rate limits beyond what annotations and schema already convey.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One sentence efficiently lists all actions and output components with no filler. Every part earns its place; it is front-loaded with the main verb and resource.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given there is no output schema, the description fully explains the return value (Markdown report with overall score, letter grade, per-dimension breakdown, findings, fuzz table). It also covers optional fuzz behavior and defaults, making the tool's behavior complete for selection and invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. The description adds minor context (e.g., fuzz triggers behavioral fuzzer) but does not significantly enhance parameter meaning beyond the schema's own descriptions and defaults.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('run introspect + lint + fuzz') and the resource ('the target'), specifies the output ('Markdown report with score, grade, breakdown, findings, fuzz table'), and distinguishes from siblings by combining multiple operations into one.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies that fuzz is optional via 'when requested' and gives a default for fuzz. However, it does not explicitly say when to use this tool versus individual sibling tools (e.g., probe_lint, probe_fuzz) or provide any exclusions or alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• probe_lintA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations provide readOnlyHint, idempotentHint, openWorldHint. Description adds that findings have stable codes, severities, locations, and fix hints, clarifying the output characteristics beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, front-loaded with action, no fluff. Efficiently communicates purpose and output for a simple tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Describes return value (list of findings with codes, severities, locations, fix hints) despite no output schema. Covers essential behavioral aspects for a tool with one optional parameter.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and the schema already describes the parameter (connectionId) fully. Description adds no additional semantic information beyond what the schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states action 'Run lint rules over tool schemas' and output 'findings with stable codes, severities, locations, and fix hints'. Differentiates from siblings (connect, fuzz, report, list, disconnect) through naming and description.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Implies usage after connecting (via connectionId) but does not explicitly state when to use vs alternatives, prerequisites, or when not to use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• probe_listA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint and idempotentHint. The description adds value by detailing the output (name, description, input schema) and the connection parameter behavior (default vs specific). No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence of 15 words, front-loaded with the action and resource. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  While there is no output schema, the description adequately conveys the return information (name, description, input schema). It covers the main usage scenario but omits edge cases like missing connections.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description's mention of 'default or a specific connection' reinforces the schema's parameter description but adds no new meaning beyond it.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the verb 'Enumerate' with resource 'tools' and specifies the returned structure (name, description, input schema). It clearly distinguishes from sibling tools like probe_connect and probe_lint.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implicitly indicates use after a connection, but provides no explicit when-to-use or when-not-to-use guidance. No alternatives are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• probe_disconnectA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations indicate write operation (readOnlyHint=false) and idempotency. The description adds critical detail: behavior changes based on id presence (single vs all). No contradictions, but could mention potential errors (e.g., invalid ID) or safety concerns with closing all.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence that front-loads the action and covers both cases. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one parameter and no output schema, the description provides essential behavior. It could be enhanced with error handling or return value hints, but is largely complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema already covers the single parameter with full description (connection ID and omission behavior). The tool description adds no extra meaning beyond the schema. High schema coverage justifies baseline 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool closes connections, specifying the two modes: single connection by ID or all if omitted. It effectively distinguishes from siblings like probe_connect (opening) and probe_list (listing).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context (closing connections) and shows two scenarios. It doesn't explicitly state when not to use it or list alternatives, but the sibling names provide differentiation. Adequate for a simple tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• probe_fuzzA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond annotations, the description discloses detailed behavior: generating one valid and several malformed inputs, recording outcomes, latency, and silent acceptance. It explicitly notes destructive tools skipped by default and the safety guard, adding significant context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is information-dense in a single sentence, which is efficient but slightly run-on. It front-loads key operations and outcomes, earning its content without significant waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (fuzzing, safety, multiple outputs), the description covers input generation, calling, outcome recording, default skip behavior, and return value (coverage summary). Minor ambiguity about whether detailed outcomes are returned, but overall sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 100% schema coverage, the description adds value by explaining the safety role of fuzzDestructive and functional role of maxTools and connectionId. It provides context not present in the schema, justifying a score above baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: generate valid and malformed inputs per target tool, call each, and record outcomes. It specifies verb+resource with precise scope, and distinguishes well from sibling tools like probe_connect and probe_lint.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains default behavior (skipping destructive tools) and provides guidance on when to use fuzzDestructive. It implies safety context but does not explicitly state when not to use or describe alternatives beyond the sibling list.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.