Server Quality Checklist

Tool Scores

• get_breaking_changesA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and openWorldHint=false. The description adds valuable behavioral context: it matches by major version against curated maps, reveals resolution details, and clarifies the meaning of found=false. It does not contradict annotations and provides transparency beyond the structured fields.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise—four sentences, no fluff. Each sentence earns its place: purpose, version input, resolution, and result handling. It is front-loaded with the core action and remains tightly focused, making it easy for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no output schema, the description covers key aspects: input format, resolution, and behavior when no data exists. It mentions source citations and last_verified dates, but does not enumerate all return fields. Given the tool's complexity and annotations, it is largely complete, though a note on pagination or errors would improve it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. The description adds meaning by explaining that version accepts SemVer ranges and that resolution is disclosed via match_type/resolved_via. This goes beyond the schema properties and helps an agent understand parameter usage and output expectations.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns breaking changes for a package version upgrade, with citations and last_verified dates. It specifies version input as concrete or SemVer ranges and explains resolution disclosure. However, it does not explicitly differentiate from sibling tools like check_compatibility or get_migration, missing a chance to clarify when this tool is the best choice.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives some usage guidance, such as not to guess when found=false, and explains version matching and resolution. However, it lacks explicit when-to-use or when-not-to-use comparisons with sibling tools. There are no exclusions or alternative recommendations, leaving the agent to infer usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_available_mapsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true. Description adds that the tool returns specific fields and supports filtering but does not discuss pagination or rate limits. Adequate but not extensive additional behavior beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two tightly written sentences: the first defines output fields, the second specifies filters and usage context. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a low-complexity tool (2 optional params, no output schema), the description covers the essential details: what is returned, optional filters, and its role relative to siblings. Could mention if results are paginated, but not critical for this case.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with both parameters fully described in the JSON schema. The description restates the filter options but does not add new semantics beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states it lists migration maps with specific fields (package, ecosystem, versions, status, last_verified) and optional filters. Distinguishes from siblings by positioning it as a discovery tool before calling get_migration.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Gives explicit usage context: 'Use this to discover coverage before calling get_migration.' Does not detail when not to use, but the guidance is clear for a simple listing tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_compatibilityA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond annotations (readOnlyHint, openWorldHint), the description discloses that the tool is not implemented, returns 'implemented=false' and no data, and warns against misinterpreting the response. This fully informs the agent of behavioral traits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the key status ('PLANNED'), followed by critical warnings and an alternative. Every sentence adds essential information with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a planned tool with no output schema, the description covers all essential aspects: status, behavior (returns implemented=false), misuse warning, and alternative. It is complete for the tool's current state.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is only 20%, so the description should compensate by explaining parameters. It does not: the four required parameters (package_a, version_a, package_b, version_b) have no additional meaning beyond their names. The ecosystem parameter is described in schema but not in description, missing an opportunity to clarify its default and allowed values.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool is a planned feature for compatibility analysis, explicitly marking it as 'PLANNED' and 'not yet implemented'. It differentiates from the sibling tool check_peer_compatibility by stating its scope (project-aware vs static SemVer check).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly advises when not to use the tool ('Do not infer compatibility...') and provides a direct alternative ('use check_peer_compatibility instead'), giving clear usage guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_peer_compatibilityA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Disclosure goes beyond annotations (readOnlyHint, openWorldHint) by clarifying no filesystem access, return types (true|false|'unknown'), and semantics of 'unknown'. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences with critical information front-loaded. Every sentence adds value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 5 parameters, no output schema, and sibling tools, the description fully explains purpose, behavior, parameter usage, and result interpretation. No gaps identified.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with clear descriptions for all 5 parameters. Description adds no further semantic detail for parameters but provides examples and return value context, maintaining baseline adequacy.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States explicit verb ('check') and resource ('peer compatibility between two package versions'), and distinguishes from sibling 'check_compatibility' by noting its different scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use (static SemVer check against curated data) and when not to (no filesystem access, contrast with unimplemented check_compatibility). Includes guidance on interpreting 'unknown' results.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_migrationA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations readOnlyHint=true and openWorldHint=false are reinforced and expanded by description: explains SemVer range resolution disclosures (match_type, resolved_via, match_note) and outcome when no data exists (found=false). No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences: first states purpose and return content, second explains parameter behavior and edge case. No redundant words; highly efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 4 parameters, no output schema, no nested objects, the description adequately covers return value details, parameter behavior, and failure case (found=false). No gaps for agent usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers 100% of parameters with descriptions. Description adds value by explaining how non-exact versions are resolved and what fields are returned (match_type etc.), going beyond schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states it returns the verified migration map for upgrading a package between two versions, listing specific contents (breaking changes, code, deprecations, URLs, date). This distinguishes it from siblings like get_breaking_changes or list_available_maps.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Description explains parameter flexibility (concrete versions or SemVer ranges) and resolution behavior for non-exact lookups. It also advises on found=false case ('do not guess'). No explicit comparison to siblings, but context provides sibling list.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.