Server Quality Checklist

Tool Scores

• probe_lintA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate readOnly and idempotent behavior, so the description does not need to restate those. It adds useful details about the output structure (stable codes, severities, locations, fix hints) but does not disclose any other behavioral traits such as prerequisites, failure modes, or performance characteristics. The added context is helpful but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, focused sentence that front-loads the main action and outcome. Every phrase adds value: it specifies the target, the operation, and the return format. There is no waste or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with one optional parameter and no output schema, the description covers the necessary context: what it does and what the return value contains. Annotations handle safety and idempotency. The tool's simple nature means no additional details are needed for competent use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides 100% coverage for the only parameter (connectionId) with a clear description. The tool description adds no additional parameter information, so it does not exceed the baseline. The schema is sufficient; the description contributes nothing extra here.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: run lint rules over tool schemas and return findings. It names the specific action, resource, and output components (codes, severities, locations, fix hints), making it distinct from sibling tools like probe_fuzz or probe_list.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not provide any guidance on when to use this tool versus alternatives among the sibling tools. It neither mentions exclusions nor suggests scenarios where probe_list or probe_report would be more appropriate. Usage context is only implied by the tool's function.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• probe_fuzzA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already mark the tool as destructive and open-world, and the description adds concrete behavioral details: it records ok/toolError/protocolCrash, detects silently accepted malformed inputs, measures latency, and returns a coverage summary. The safety default of skipping destructive target tools unless fuzzDestructive is set is also disclosed, going beyond the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences and front-loads the core action, making the tool's function immediately clear. The first sentence is somewhat long, but every detail earns its place and the safety sentence is essential.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Even without an output schema, the description states it returns a coverage summary of fuzzed vs skipped tools, which covers the primary return value. Combined with full schema coverage and annotations, this is complete for the tool's complexity; the probe_connect prerequisite is handled by the schema's connectionId description.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and all three parameters (maxTools, connectionId, fuzzDestructive) are already documented in the input schema. The description only restates fuzzDestructive's behavior and does not add new parameter-level meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific action—generate one valid and several malformed inputs per target tool, call each, and record outcomes—which clearly defines the tool's purpose. It distinguishes probe_fuzz from siblings like probe_connect and probe_lint by emphasizing malformed-input generation and the resulting coverage summary.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies this tool is for fuzzing target tools and explicitly explains the fuzzDestructive flag behavior. However, it does not name alternative tools or explicitly state when to choose this over a sibling, so usage guidance remains implicit rather than explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• probe_reportA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=false and openWorldHint=true, which covers the possibility of side effects. The description offers additional context about the report format and that fuzz runs only when requested, which implies a safety guard. However, it does not explicitly warn about the potential destructive effects when fuzzDestructive=true, though the schema parameter description does. This is an adequate but not thorough disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, information-dense sentence. It front-loads the main verb and quickly enumerates all key output components. While it is a bit long, it contains no filler words and every clause adds useful information. The structure is acceptable for a tool that performs a multi-step process.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given moderate complexity (4 optional parameters, no output schema), the description adequately explains the tool's purpose and the content of its returned report. It covers the key aspects of what the report includes and mentions the optional fuzz component. However, it could be more explicit about preconditions like needing a connection (connectionId) and about the four dimensions being scored, but these are partially covered by schema and context. Overall, it is reasonably complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema covers 100% of parameters with descriptions, so the baseline is 3. The description does not add any parameter-specific semantics beyond what the schema already provides, such as the meaning of fuzz, maxTools, connectionId, or fuzzDestructive. It only indirectly references fuzz via 'when requested', but this adds no new meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's composite function: running introspect + lint (and optionally fuzz), scoring results on four dimensions, and returning a Markdown report with specific components (overall score, letter grade, breakdown, findings, fuzz table). The verb 'Run... score... return' is specific, and the resource is the target. This distinguishes it from siblings like probe_lint and probe_fuzz, which are individual operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use this tool: when a comprehensive scored report is needed rather than just lint or fuzz. However, it does not explicitly say 'use this instead of probe_lint when you need scoring' or mention any exclusions. The guidance is clear from the context but lacks explicit alternative differentiation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• probe_connectA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=false, so the agent knows this is not read-only. The description adds that it returns connectionId and server info, which is helpful. However, it does not disclose that the connection remains open and requires a subsequent probe_disconnect call, which is a significant behavioral detail beyond what annotations provide.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that front-loads the main action and return information. Every word earns its place; there is no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 5 parameters and no output schema, the description does well by stating what it returns (connectionId, name, version, capabilities, counts). It misses the caveat that the connection persists and should be closed via probe_disconnect, which would make it more complete for an agent managing resources.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema covers 100% of the parameters, providing descriptions for transport, command, url, args, and env. The description adds no parameter-level detail beyond what the schema already gives, so it meets the baseline for high schema coverage without needing compensation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Open a connection'), names the resource ('MCP target server'), and specifies the transports (stdio or HTTP). It also states the return value (connectionId and server info), which clearly distinguishes it from siblings like probe_disconnect and probe_list.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly implies when this tool is used—when you need to establish a connection to an MCP server. It doesn't explicitly mention alternatives or exclusions, but the sibling names and the action of 'probe_connect' make the context obvious, so it earns a 4 for clear context without explicit guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• probe_disconnectA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate idempotency and non-read-only behavior. The description adds useful context by stating the operation can target a single connection or all connections, and that the id comes from probe_connect. It does not go deeper into failure modes, but that is acceptable for this simple tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, tightly packed sentence that conveys both modes of operation without extra words. It is immediately readable and front-loaded with the core action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a one-parameter tool with no output schema and no nested objects, the description fully covers the essential usage semantics. It competently supports the agent in selecting and invoking the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%: the id parameter's meaning and omission behavior are already documented in the schema. The tool description adds no new parameter-level insight beyond what the schema provides, so baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Close') and resource ('connection'), and clearly distinguishes the tool from siblings by explaining it closes connections opened by probe_connect. The optional-id behavior is explicit, making the tool's purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear conditional usage: include an id to close one connection, omit it to close all. It does not explicitly name alternatives, but the sibling context (probe_connect) makes the intended pairing obvious.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• probe_listA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and idempotentHint=true, and the description adds useful context by specifying the default connection behavior and the exact data returned (name, description, input schema). This goes beyond the structured annotations without contradicting them.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that conveys the core purpose, output contents, and connection handling without any filler. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  This is a low-complexity tool with one optional parameter, clear annotations, and no output schema. The description adequately covers what the tool returns and how the connection is selected, making it complete for an agent to use correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With one parameter and 100% schema description coverage, the schema already documents connectionId well, including its source and default behavior. The description adds only a brief restatement of 'default or a specific connection,' which is helpful but not significantly beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Enumerate' with a specific resource ('the target's tools') and explicitly lists the returned contents (name, description, input schema). It clearly distinguishes this from sibling tools like probe_connect, probe_lint, and probe_report by indicating that its purpose is listing tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context: it should be used to enumerate tools after establishing a connection, with the connection optional and defaulting to the most recent one. It does not explicitly name alternatives or exclusions, but the usage context is sufficiently implied.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.