Server Quality Checklist

Tool Scores

• get_changes_between_refsB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description must carry the full burden of behavioral disclosure. It only says the tool 'gets' commits and uses the GitHub compare API; it does not mention pagination, rate limits, auth requirements, potential response size, or any other operational behavior beyond the implied read-only nature.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences with the main operation front-loaded and no filler. The mention of the GitHub compare API is brief and serves as useful context without bloating the definition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with four required and fully documented parameters, the description plus schema covers the core invocation. It also states what the response contains semantically. However, with no output schema and no annotations, it leaves response shape, pagination, and error/limit behavior unspecified, so it is not fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline applies. The schema already documents owner, repo, base, and head, including that base is the older point and head is the newer point. The description's phrase 'between two git refs' adds little semantic value beyond what the schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the action: get all commits between two git refs, and specifies the enrichment (PR metadata, author information, linked issues). It does not explicitly contrast with sibling tools like get_pull_requests_in_range or get_release_summary, but the commit-focused wording makes the distinction mostly clear.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance about when to use this tool versus the sibling tools. It names the GitHub compare API but does not explain when an agent should prefer this over get_pull_requests_in_range or get_release_summary, nor does it state prerequisites or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_pull_requests_in_rangeA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of disclosing behavior. It specifies that only merged PRs are returned, lists included metadata (labels, linked issues, review counts, files changed), and explains label categorization. It does not mention auth, pagination, or error behavior, but for a retrieval tool the core behavior is transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no filler. The key action and scope are front-loaded, and the category list adds useful detail without bloating the text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a straightforward read/list tool with all parameters documented in the schema, the description adequately covers what the tool returns and how results are categorized. It does not describe return format or pagination, but it is sufficiently complete for an agent to call it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already fully documents all four parameters. The description adds no parameter-specific details beyond what the schema provides, so the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific verb 'Get' and the precise resource ('all merged pull requests between two refs'). It differentiates from the siblings by emphasizing PRs with labels, metadata, and categorization, which is distinct from get_changes_between_refs and get_release_summary.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use the tool: when you need merged PRs between two refs, with the additional distinction that only merged PRs are included. However, it does not explicitly name alternatives or state when not to use this tool, so it stops short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_release_summaryA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It transparently describes behavior by listing what data the tool combines and clarifying that the output is a structured object intended for synthesis. It does not discuss side effects, rate limits, or edge cases, but 'Generate' and the aggregation wording sufficiently signal a read-only, non-destructive operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with no wasted words. It front-loads the core purpose and then efficiently enumerates the combined data sources, giving the agent maximum signal per word.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  There is no output schema, but the description compensates by naming the output type and its major components. The four required parameters are fully covered by the schema. Some detail about output shape or edge cases is absent, but the description is complete enough for an agent to select and invoke the tool effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description adds only the 'range between two tags' framing, which reinforces from_tag and to_tag semantics but does not meaningfully expand on what the schema already documents for owner, repo, from_tag, or to_tag.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names a specific verb ('Generate'), a specific resource ('structured release context object'), and the scope ('range between two tags'). It also lists concrete contents (commit data, PR metadata, linked issues, contributors, aggregate statistics), which clearly distinguishes it from sibling tools focused on individual data types.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'ready for AI synthesis into release notes' gives a clear context for when to use the tool. It does not explicitly name sibling alternatives or state when not to use them, but the 'Combines...' clause implies that this is the aggregate choice when multiple data sources are needed.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.