Server Quality Checklist

Tool Scores

• mcp_doctor_diagnose_serverC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose side effects, permissions, and output, but it does not. 'Diagnosis' implies a read-only operation, but that is not confirmed; it also gives no indication of what the tool returns (report, status, data). The description adds little beyond the basic action.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one sentence, front-loads the verb, and contains no filler. It is appropriately sized for the information it conveys, even though it omits optional details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool performing deep diagnosis with no output schema and no annotations, the description is under-specified. It does not explain what a successful diagnosis looks like, how results are represented, or how it relates to the other two sibling tools. Agents cannot reasonably predict call outcomes or next steps from this description.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so all four parameters are already described with reasonable detail. The description's reference to 'command, arguments, environment' aligns with those parameters but adds little beyond reinforcing that they form a 'server definition.' This matches the baseline for a fully-described schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the primary action ('diagnosis') and the target resource ('specific MCP server definition'), naming the relevant aspects (command, arguments, environment). The phrase 'in-depth' helps differentiate from a generic scan, though it doesn't explicitly contrast with mcp_doctor_scan or mcp_doctor_repair_json.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives no indication of when to use this tool versus the siblings. There is no mention of prerequisites, alternative conditions, or exclusions. An agent is left to infer that single-server diagnosis is intended, but the description does not state it outright.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• mcp_doctor_repair_jsonA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  There are no annotations providing safety or side-effect information, and the description does not disclose what the tool returns after repairing (e.g., the repaired JSON, a success status, or an error). It also does not mention whether the repair is in-place or returns a new value, leaving important behavioral aspects ambiguous. The description focuses solely on the action, not the outcome or consequences.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with no fluff or redundancy. It efficiently lists the key repair operations in a parenthetical, making the tool's functionality clear without wasted words. The structure is compact and information-dense, earning a perfect score.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description adequately covers the tool's input and purpose but omits critical context: there is no output schema and the description does not state what the tool returns after repair. An agent using this tool would not know whether to expect the repaired JSON, a confirmation message, or an error. It also does not mention edge cases (e.g., non-JSON input or unresolvable malformations). This lack of output and error handling information makes the description incomplete for practical use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The single parameter `raw_json` has a schema description ('The broken or unformatted JSON text') that fully explains its meaning, so the schema coverage is high. The tool description itself does not add any additional explanation about the parameter beyond what is already in the schema, so it provides no extra value. Per the baseline for high schema coverage, a score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: auto-repair malformed MCP configuration JSON. It lists specific repair actions (fixing trailing commas, removing invalid comments, converting single quotes, ensuring a root mcpServers key), which is a specific and unambiguous verb+resource. The name and description also differentiate it from sibling tools like scan and diagnose, making its purpose distinct.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use the tool: when you have malformed or unformatted MCP JSON that needs fixing. It does not explicitly mention alternatives or state 'use this instead of scan/diagnose', but the action-oriented phrasing and the sibling tool names make the appropriate context clear. A slight deduction for not making the when-to-use-versus-alternatives guidance explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• mcp_doctor_scanA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The verbs 'scans' and 'diagnoses' imply a read-only, non-destructive operation. No side effects or permissions are mentioned, but the phrasing naturally suggests no modifications. Since there are no annotations to rely on, this transparency is sufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, compact sentence that conveys all necessary information without redundancy or unnecessary detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no parameters and no output schema, the description provides sufficient context about its function and scope. It does not describe output format, but that is not expected given the absence of an output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so there is no parameter information to describe. Per the baseline rule, a score of 4 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific action (scans) and a specific resource (MCP config files for three applications), and the diagnostic scope (server configurations, binary paths, env vars). It is distinct from sibling tools like diagnose_server (which likely targets a single server) and repair_json (which fixes issues), making its purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description tells when to use the tool: when you need to scan and diagnose across multiple local MCP config files. It does not explicitly mention when to avoid it or compare against siblings, but the scope is clear enough that an agent can infer the appropriate context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.