Server Quality Checklist

Tool Scores

• sonar_get_duplication_reportA
  3.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the full burden. It only states the basic purpose without disclosing behavior such as read-only guarantee, error handling, or performance characteristics. Minimal transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no filler. The first sentence states the purpose and output, the second gives usage guidance. Efficient and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with one parameter and no output schema. The description gives a high-level overview of the output (files and percentages) but lacks detailed structure or examples. Adequate but slightly incomplete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% for the single parameter 'pullRequest', and its schema description is clear. The description does not add any extra meaning beyond the schema, resulting in a baseline score of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'get' and the resource 'duplicated lines report for a pull request', specifying what the tool returns (files with duplicated code and percentage). It is distinct from sibling tools which focus on issues and metrics.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly advises using the tool when the user mentions 'duplication' or 'duplicated lines', providing clear context. However, it does not mention when not to use it compared to alternatives, leaving room for improvement.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sonar_get_file_issuesA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Describes the return format (line number, rule, severity, fix message) but no additional behavioral traits like read-only nature or rate limits. With no annotations, it's adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences: first defines core function, second adds usage and output shape. Efficient, no fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 2 params, no output schema, and no annotations, the description covers purpose, usage, and output shape well. Could mention error handling or preconditions, but not essential.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. Description does not add significant meaning beyond the schema; it merely restates filePath relative path and pullRequest scope. No extra constraints or examples.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states the verb 'Get' and resource 'open SonarQube issues for a specific source file'. Differentiates from siblings like sonar_get_pr_issues (PR-level) and sonar_get_duplication_report.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says 'Use this when the user asks about issues in a particular file they are editing or viewing', providing clear context. No explicit when-not-to-use or alternatives, but sufficient for typical use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sonar_get_pr_issuesA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must disclose behavioral traits. It mentions the return format (grouped by file with rule, line, severity, message) but does not explicitly state that the operation is read-only, safe, or describe potential side effects or limits. The 'get' verb implies read-only, but explicit safety information is missing.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is exceptionally concise, consisting of two sentences that deliver purpose, usage guidance, and output structure without any redundant words or filler. It is well-organized and front-loaded with the core action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (2 params, no output schema), the description adequately covers input requirements and output format (grouped by file with issue details). It does not address error conditions or absence of issues, but for a straightforward list retrieval, it is sufficiently complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the parameter descriptions are already clear in the schema. The description adds no new semantic value beyond restating that pullRequest is a number and severity is optional with enum values. It does not explain parameter usage nuances or format expectations beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves all open SonarQube issues for a specific PR, grouped by file. It distinguishes itself from siblings by specifying the use case (user asks about issues on PR, branch, or 'what does Sonar say') and mentions return structure, making the purpose distinct and actionable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly advises when to use the tool (when user asks about issues on PR, branch, or 'what does Sonar say'), providing clear context. However, it does not explicitly state when not to use it or mention alternative sibling tools for other needs, though the differentiation is implied by context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sonar_get_pr_metricsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It states the tool retrieves metrics (read-only), but does not mention authentication, rate limits, or error behavior. Adequate but could add more context about the system (e.g., SonarQube/Cloud).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences, front-loaded with purpose and metrics, followed by usage examples. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given a simple tool with one parameter and no output schema, the description explains the output (metrics list) and usage context. Lacks details on return format or pagination, but sufficient for typical use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with a clear param description. The tool description adds context about what metrics are returned but does not enhance parameter meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states 'Get quality metrics for a pull request' and lists specific metrics (coverage, duplication, issues). It distinguishes from sibling tools by specifying the scope (overall PR metrics vs. duplication or file-level issues).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit example queries when the tool should be used ('what is my coverage?', 'does Sonar pass?', etc.). Does not explicitly mention when not to use, but the examples cover common intents.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.