Server Quality Checklist

Tool Scores

• list_pull_requestsA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The readOnlyHint annotation already communicates that this is a safe read operation. The description itself adds no additional behavioral context, such as default filtering (state defaults to OPEN) or pagination behavior, which are captured in the schema. No contradiction exists.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single concise sentence that directly states the tool's purpose with zero redundant words. It is front-loaded and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a straightforward list operation with a rich schema (including defaults and filters) and a readOnlyHint annotation, the description provides enough context. It does not mention return values, but the name and tool nature imply a list of pull requests, and the schema covers behavior. Slight gap in not stating default state filtering, but acceptable.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema covers 100% of the 8 parameters with detailed descriptions, so the tool description does not need to explain parameters. It adds no parameter-level meaning beyond the schema, which aligns with the baseline 3 for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (List) and resource (pull requests) within a Bitbucket Cloud repository. It is specific but does not explicitly distinguish from the sibling tool get_pull_request, which presumably retrieves a single pull request.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage is implied: use this tool to list multiple pull requests, as opposed to getting a single pull request. However, there is no explicit guidance on when to use this versus alternatives, nor any exclusions or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_commentsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, so the read-only nature is covered. The description adds the useful detail that inline comments carry file and line, but does not discuss rate limits, pagination, or default behavior. This is adequate but not rich beyond the annotation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, front-loaded with the primary purpose, and contains no redundant information. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a read-only tool with a fully documented schema and readOnlyHint annotation, the description adequately conveys purpose and a key data detail (inline comments have file/line). It doesn't enumerate all return fields, but no output schema exists; this is a minor gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with each parameter clearly documented including defaults and examples. The description adds no parameter-specific meaning beyond what the schema already provides, so the baseline score of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb and resource: 'Read existing comments on a pull request.' It clearly distinguishes itself from sibling tools like post_comment (write) and get_pull_request (PR metadata), and adds the purpose of avoiding duplicate review points.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'so a review does not repeat points someone already raised' provides a clear use case: check existing comments before posting a review. It doesn't explicitly name alternative tools or exclusions, but the context is clear among the sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_diffA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The annotation readOnlyHint: true already indicates this is a safe read operation, so the description does not need to repeat that. The description adds no extra behavioral details (e.g., truncation via max_chars), but it does not contradict the annotation. This is acceptable given the annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that front-loads the core purpose. Every word earns its place, with no filler or redundancy. This is an example of ideal conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple nature of the tool, the annotations cover safety, and the schema fully documents parameters, the description is sufficient. The return format is implied (the diff), and no output schema exists, so a more detailed description could help but is not critical for this straightforward operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema description coverage is 100%, so all parameters (pull_request_id, max_chars, repo_slug, workspace) are already well-documented. The description adds no additional parameter meaning beyond what the schema provides, which meets the baseline for full schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function with a specific verb ('Fetch') and resource ('the unified diff for a pull request'). This distinguishes it from sibling tools like get_diffstat (statistics) and get_pull_request (metadata), making the purpose immediately clear.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use the tool: to fetch the unified diff of a pull request. It does not explicitly mention alternatives or exclusions, but the purpose is unambiguous. A 4 is appropriate because it lacks explicit when-not-to-use guidance, but the context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_diffstatA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The readOnlyHint annotation already declares the operation safe. The description adds valuable context about output scope (file-level line counts, not the diff content), clarifying behavior beyond the annotation. It does not address pagination or errors, but the safety profile is covered.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences: the first states purpose, the second gives a pragmatic usage tip. Front-loaded, no redundant words, and every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read-only list tool with annotations and a clear schema, the description covers core behavior and usage context. The absence of an output schema is mitigated by describing what is returned (files with line counts), making the tool's invocation and expected result reasonably clear.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 67%, covering repo_slug and workspace. The required pull_request_id lacks a description, but its meaning is obvious from the name and context. The description adds no parameter-level detail, relying on the schema, which is adequate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly specifies the tool's function: listing files changed in a pull request with per-file line gains/losses, explicitly excluding the full diff. This distinguishes it from the sibling tool get_diff by saying 'without the diff itself.'

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage guidance: 'Use this first on a large pull request to decide what is worth reading.' It gives a clear scenario but does not explicitly name alternatives or exclusions, though the contrast with get_diff is implied.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_fileA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already mark this as read-only, so the safety profile is covered. The description adds that it reads at the source commit, but it does not disclose truncation behavior via max_chars, error conditions, or return format. Minimal additional behavioral context beyond the annotation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the primary action and immediately followed by a practical use case. Every word earns its place with no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the schema covers parameters and the annotation covers safety, the description provides sufficient context for selection. However, the claim of reading a 'whole file' is slightly misleading because max_chars defaults to 50000 and can truncate; the description does not mention this or the return behavior, and there is no output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so all five parameters have descriptions. The tool description does not add extra parameter meaning beyond the schema, so the baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool reads a whole file at the pull request's source commit, distinguishing it from diff tools that show changed lines only. The verb 'Read' and resource 'whole file at the pull request's source commit' are specific and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly contrasts this tool with diff output and gives a concrete condition: 'when a hunk cannot be judged on its own.' This directly tells the agent when to use get_file versus get_diff, which is a sibling tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_pull_requestA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The readOnlyHint annotation already communicates the read-only nature, and the description adds a valuable behavioral nuance: the 'description' field is the change's claim, which is what reviews measure against. It also lists expected return fields, helping the agent understand what data will come back.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences: the first states the tool's purpose and scope, the second adds meaningful semantic context about the 'description' field. No wasted words or redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, the description compensates by listing the key returned fields and clarifying the meaning of 'description.' It is sufficiently complete for a simple read-only tool, though it could more explicitly frame this as a return value list.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema descriptions cover repo_slug and workspace, but pull_request_id has no description beyond its name. The description's 'one pull request' implies selection by ID but does not explicitly explain the parameter. Added value over the schema is minimal.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: 'Title, description, author, state, and branches for one pull request.' The explicit singular scope 'for one pull request' clearly distinguishes it from sibling tools like list_pull_requests and get_diff.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when a single pull request's details are needed and mentions 'one pull request,' but it does not explicitly contrast with alternatives or state when not to use this tool. It provides clear context but no exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• post_commentA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate a non-read-only, non-destructive operation. The description adds context beyond annotations by explaining the line/diff attachment behavior, which is a meaningful behavioral detail. No contradictory or surprising traits are disclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the main action and a concise usage hint. Every word earns its place, with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple write operation with full schema coverage and no output schema, the description covers the key usage (including inline comments). It does not describe the response format, but this is inferable and not a major gap given the tool's simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 100% description coverage, so a baseline of 3 applies. The description adds value by clarifying the relationship between path and line for attaching comments to a specific line, which the schema only hints at via 'requires path'.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb+resource ('Post a comment on a pull request') and clearly distinguishes itself from the sibling get_* tools, which are all read-oriented. It also adds the specific capability of attaching to a line of the diff.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is for posting comments, and adds a hint about using path and line for inline comments. However, it does not explicitly state when to prefer this tool over alternatives or provide exclusions. The guidance is implied rather than explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.