Server Quality Checklist

Tool Scores

• merge_prC
  2.5/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Given that annotations are entirely absent, the description carries the full burden of behavioral disclosure. It only says 'Merge a pull request'—a minimal restatement of the tool's name—without mentioning side effects like branch deletion, permission requirements, or potential merge conflicts. This adds no transparency beyond the name itself.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence with zero waste. It states the essential purpose directly and is front-loaded. While it may be too brief for other dimensions, conciseness is a strength here.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 6 parameters, no annotations, and no output schema, this description is severely under-specified. It does not explain merge behavior, default method, possible outcomes, or any constraints. The single sentence is insufficient for an agent to understand when and how to invoke the tool correctly beyond what the name already suggests.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema covers 100% of the parameters with descriptions, so per baseline rules a score of 3 is appropriate. The description itself adds no parameter-specific semantics, but the schema already documents all six parameters, including optional commit_title, commit_message, and merge_method.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Merge a pull request' clearly states the action (merge) and the resource (pull request), which distinguishes it from sibling tools like create_pr, get_pr, and list_prs. However, it lacks any qualifying detail about scope or variant (e.g., merge into default branch), so it's clear but not maximally informative.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines1/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. It does not mention prerequisites like approvals or CI status, nor does it reference sibling tools or alternative merge strategies. The description is purely definitional and offers no usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_prB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full responsibility for behavioral disclosure. It merely says 'Get details about a pull request' without detailing what 'details' include, whether the operation is read-only, any potential side effects, or the response format. This is a significant gap for a tool with no output schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that gets straight to the point with the verb front-loaded. It wastes no words, though it is sparse. It earns a 4 for effectively communicating the core purpose without unnecessary verbiage.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple get tool with no output schema and no annotations, the description should provide at least some context about the returned data or usage context. The current description is a restatement of the tool name and does not explain what 'details' means, leaving the agent to guess about response structure and edge cases.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already provides 100% documentation for all three parameters (owner, repo, pr_number). The description adds no additional meaning beyond what the schema states, so it neither improves nor detracts from parameter clarity. Baseline of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly uses a specific verb ('Get') and resource ('details about a pull request'), making the tool's purpose immediately obvious. It also distinguishes from sibling tools like list_prs (which lists multiple PRs) and create_pr/comment_on_pr/merge_pr (which perform actions), by focusing on retrieval of a single PR's details.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided about when to use this tool versus alternatives such as list_prs. The description only states what the tool does, not the scenarios in which it should be preferred, nor any exclusions or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• comment_on_prB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose behavioral traits but only states the obvious action. It does not mention side effects, permissions, idempotency, or any side effects beyond adding a comment, which is insufficient for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence that effectively communicates the core purpose without redundancy. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite the simple action and complete parameter schema, the description lacks behavioral context, usage guidance, and return value information. With no annotations and no output schema, a more complete description is needed to guide the agent effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  All parameters are fully documented in the schema (100% coverage), so the description does not need to add parameter details. The description adds no extra meaning, but the schema already provides the necessary semantics, yielding the baseline score.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (add) and the resource (comment to a pull request), distinguishing it from sibling tools like get_pr and merge_pr. It is specific and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. There is no mention of prerequisites, context, or exclusions, despite the presence of sibling tools for PR operations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_prB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It only mentions 'create', implying a write operation, but discloses no side effects, prerequisites, or return behavior. This is minimal and mirrors the mid-tier example that scored 2.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with zero filler words. It is highly concise and clearly structured, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite a complete schema, the description lacks essential context such as what a pull request is, what inputs are needed, or expected outcomes. With no annotations and no output schema, this minimal description is insufficient for a tool with six parameters.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage for all six parameters, so the schema already provides parameter semantics. The description adds no extra meaning beyond the schema, matching the baseline of 3 for high coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Create a pull request' uses a specific verb and resource, clearly stating the tool's action and target. It distinguishes itself from sibling tools like get_pr, merge_pr, list_prs, and comment_on_pr by implying a creation operation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance on when to use this tool vs. alternatives. The description simply states the action without context, exclusions, or comparison to sibling tools, so the agent gets no explicit usage direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_prsB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It only says 'list', implying a read operation, but does not explicitly state that it is non-mutating, nor does it mention return format, pagination, ordering, or default state. The description adds little beyond the tool name and schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence, front-loaded with the key verb and resource. Every word earns its place with no unnecessary filler or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is minimal and does not explain return values, pagination, or any behavior beyond listing. While the schema covers parameters well, the absence of output schema and annotations leaves room for more context. However, for a simple list tool with well-defined parameters, this is adequate but not complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema provides descriptions for all three parameters (owner, repo, state) with 100% coverage. The description does not add any additional meaning to the parameters. Baseline 3 is appropriate since the schema already documents the parameters adequately.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (List) and the resource (pull requests) for a repository. This distinguishes it from sibling tools like get_pr (single PR), create_pr, comment_on_pr, and merge_pr, which have different purposes. The verb+resource pair is specific and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance on when to use this tool versus alternatives. No mention of, for example, 'use get_pr to retrieve details of a single PR' or 'use create_pr to open a new PR'. The description simply states what it does without context or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.