Server Quality Checklist

Tool Scores

• remove_serverC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool deletes server settings, implying a destructive mutation, but doesn't cover critical aspects like permissions required, whether deletion is reversible, error handling specifics, or rate limits. The return format is mentioned but without details on success/error structures, leaving gaps in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded, starting with the core purpose. The Args and Returns sections are structured but could be more integrated. It avoids unnecessary verbosity, though the return type explanation is somewhat technical without practical guidance.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (destructive mutation), lack of annotations, no output schema, and minimal parameter coverage, the description is incomplete. It doesn't address safety concerns, error scenarios, or integration with sibling tools, making it inadequate for confident agent use without additional context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds minimal semantics beyond the input schema. It explains that the 'name' parameter is '削除するサーバーの名前' (the name of the server to delete), which clarifies its purpose. However, with 0% schema description coverage and only one parameter, this adds some value but doesn't fully compensate for the lack of schema details (e.g., format constraints). Baseline is 3 due to low parameter count.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose as 'MCPサーバー設定を削除する' (deletes MCP server settings), which is a specific verb+resource combination. It distinguishes itself from siblings like 'add_server', 'update_server', and 'get_server' by focusing on deletion. However, it doesn't explicitly differentiate from 'list_servers' in terms of destructive action, which slightly reduces clarity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., server must exist), exclusions, or compare with siblings like 'update_server' for modifications or 'get_server' for checking before deletion. Usage is implied through the action but lacks explicit context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_serverB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states this is a read operation ('取得する' - get), implying it's non-destructive, but doesn't mention permissions, rate limits, error handling, or what '設定情報' (setting information) includes. For a tool with zero annotation coverage, this leaves significant gaps in understanding its behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and well-structured. It starts with a clear purpose statement, followed by concise sections for 'Args' and 'Returns'. Each sentence adds value without redundancy, making it easy to parse and understand quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (1 parameter, no output schema, no annotations), the description is minimally complete. It covers the basic purpose and parameter but lacks details on usage guidelines, behavioral traits, and output specifics. Without annotations or output schema, more context on what '設定情報' includes would improve completeness, but it meets the minimum viable threshold.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds minimal semantic context for the single parameter: 'name: 取得するサーバーの名前' (name: name of the server to get). With 0% schema description coverage, the schema provides no parameter descriptions. The tool has only one parameter, so the baseline is 4, but the description doesn't fully compensate by explaining format, constraints, or examples, keeping it at an adequate but basic level.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: '特定のMCPサーバーの設定を取得する' (Get settings of a specific MCP server). It uses a specific verb ('取得する' - get) and resource ('MCPサーバーの設定' - MCP server settings). However, it doesn't explicitly distinguish this from sibling tools like 'list_servers' or 'update_server', which would be needed for a score of 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention sibling tools like 'list_servers' (for listing all servers) or 'update_server' (for modifying settings), nor does it specify prerequisites or appropriate contexts for use. The agent must infer usage from the tool name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• update_serverA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It adds some context: it specifies that only provided fields are updated (partial updates) and explains the 'replace_env' parameter's effect. However, it lacks details on permissions, side effects, error handling, or rate limits, which are important for a mutation tool. The description doesn't contradict annotations (none exist).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and appropriately sized. It starts with the core purpose, adds a key behavioral note (partial updates), then details parameters and return values in labeled sections. Every sentence earns its place, though the return statement could be more concise. It's front-loaded with the main action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (mutation with 5 parameters), lack of annotations, and no output schema, the description is moderately complete. It covers parameters well and hints at behavior, but misses critical context like error conditions, idempotency, or response structure details. For a mutation tool, this leaves gaps an agent would need to infer.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate fully. It successfully does so by listing all 5 parameters with clear explanations: 'name' identifies the server, 'command' and 'args' are optional new values, 'env' adds/updates environment variables, and 'replace_env' controls full replacement vs. merge. This adds essential meaning beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: '既存のMCPサーバー設定を更新する' (update existing MCP server settings). It specifies the verb (update) and resource (MCP server settings), making the function unambiguous. However, it doesn't explicitly differentiate from sibling tools like 'add_server' or 'remove_server' beyond the 'update' action.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., server must exist), compare to 'add_server' for creation or 'remove_server' for deletion, or specify scenarios where updating is appropriate versus other operations. Usage is implied but not explicitly stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• export_configA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It describes what the tool does (exports configuration) and what it returns, but doesn't mention important behavioral aspects like whether this requires special permissions, if it's a read-only operation, potential rate limits, or how large configurations are handled. It provides basic context but lacks comprehensive behavioral details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly structured and concise: a clear purpose statement, a usage context sentence, and a detailed return value specification. Every sentence earns its place, with no redundant information. The Japanese text is efficiently translated in the response format.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no parameters, no annotations, and no output schema, the description does a reasonable job explaining what the tool returns. However, for a configuration export tool, it could benefit from mentioning whether this includes sensitive data, if the export is in a specific format version, or any limitations on what can be exported. The description is adequate but has clear gaps in contextual information.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the baseline score is 4. The description appropriately doesn't waste space discussing non-existent parameters, and the 100% schema description coverage for the empty schema is irrelevant since there are no parameters to document.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool exports the entire MCP configuration in JSON format, which is a specific verb (export) + resource (MCP configuration). However, it doesn't explicitly differentiate from sibling tools like 'get_server' or 'validate_config' which might also retrieve configuration data, so it doesn't reach the highest score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides implied usage context ('for display or copying to another location'), but doesn't explicitly state when to use this tool versus alternatives like 'get_server' for specific server configurations or 'validate_config' for validation purposes. No explicit exclusions or alternative tool recommendations are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• validate_configA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden. It discloses the tool's behavior by describing what it validates (file existence, JSON format, required fields) and the return structure. However, it doesn't mention error handling details, performance characteristics, or whether this is a read-only operation (though implied by validation).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly structured and concise. It starts with the core purpose, details the validation scope in bullet-like clarity, and explicitly documents the return format. Every sentence adds value with zero redundancy or wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter validation tool with no output schema, the description provides excellent context. It explains what gets validated and documents the return structure in detail. The only minor gap is lack of explicit mention about whether this is a read-only operation, though validation strongly implies it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters with 100% schema description coverage, so the baseline is 4. The description correctly indicates no parameters are needed by not mentioning any, which aligns with the empty input schema. No additional parameter information is needed or provided.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('検証する' - validate) and resources ('現在のMCP設定ファイル' - current MCP configuration file). It distinguishes from siblings by focusing on validation rather than CRUD operations on servers (add_server, remove_server, etc.) or configuration export/retrieval.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context through its purpose - validating configuration files. However, it doesn't explicitly state when to use this tool versus alternatives like checking individual servers with get_server or listing servers with list_servers. The context is clear but lacks explicit exclusions or comparison to sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• add_serverA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden. It discloses key behavioral traits: the tool creates new configurations, returns error on duplicate names, and provides success/error responses. However, it doesn't mention permission requirements, side effects, rate limits, or whether the change is persistent/immediate. For a creation tool with zero annotation coverage, this is adequate but lacks comprehensive behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly structured and front-loaded: purpose statement first, then key behavioral constraint, followed by organized parameter and return value sections. Every sentence earns its place - the duplicate name warning is crucial, and the parameter documentation is essential given the schema coverage gap. No wasted words or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a creation tool with 4 parameters, 0% schema coverage, no annotations, and no output schema, the description does remarkably well. It covers purpose, key constraint, all parameters, and return value format. The main gap is lack of output schema, but the description partially compensates by mentioning the return type. Given the complexity, it's nearly complete but could benefit from more behavioral context about the creation process.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description fully compensates by providing clear parameter documentation in the Args section. It explains all 4 parameters (name, command, args, env), indicates which are optional, and gives meaningful Japanese descriptions that add semantic understanding beyond the bare schema. This is excellent parameter documentation given the schema coverage deficiency.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('新しいMCPサーバー設定を追加する' - adds new MCP server configuration) and resource (server settings). It distinguishes from siblings by focusing on creation rather than retrieval (get_server, list_servers), modification (update_server), or deletion (remove_server). The purpose is unambiguous and well-defined.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context about when NOT to use this tool ('既に同名のサーバーが存在する場合はエラーを返します' - returns error if server with same name exists), which helps avoid duplicate creation attempts. However, it doesn't explicitly mention when to use alternatives like update_server for modifying existing servers or compare with other siblings, leaving some guidance gaps.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_serversA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It clearly indicates this is a read-only operation (listing/displaying) and provides valuable behavioral context about the return format (dictionary with servers list and config path). This goes beyond what the empty input schema provides, though it doesn't mention potential limitations like pagination or rate limits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly structured: a clear purpose statement followed by detailed return format documentation. Every sentence earns its place - the first explains what the tool does, the second explains what it returns. No wasted words or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter read operation with no output schema, the description provides excellent context: clear purpose, behavioral transparency about being a listing operation, and detailed return format. The only minor gap is the lack of explicit guidance on when to use versus sibling tools, but overall it's quite complete for this tool's complexity level.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters with 100% schema description coverage (empty schema). The description appropriately doesn't discuss parameters since none exist, which is correct for a zero-parameter tool. No additional parameter semantics are needed beyond the baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('リスト表示する' - list display) and resource ('設定されている全てのMCPサーバー' - all configured MCP servers). It distinguishes itself from siblings like get_server (which retrieves a specific server) and add_server/remove_server/update_server (which modify servers).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context by specifying '設定されている全ての' (all configured), suggesting this is for viewing all servers rather than filtering or modifying. However, it doesn't explicitly state when to use this versus alternatives like get_server for a specific server or validate_config for configuration validation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.