Server Quality Checklist

Tool Scores

• search_issuesA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral burden. It states the search scope, that it returns results per issue, and the specific fields provided, which strongly implies a read-only search operation. It does not mention pagination, sorting, or failure behavior, but these are secondary for a read-only query tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences with no filler. The first sentence states the action and scope; the second lists the return fields. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the six optional parameters, the 100% schema coverage, and the lack of an output schema, the description provides sufficient context to call the tool correctly, especially with its list of returned fields. Minor omissions — such as explicit guidance about sibling tools or project configuration prerequisites — prevent a perfect score.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so all six parameters (cid, limit, impact, offset, status, checker) are already documented in the schema. The description adds no parameter-level semantics, but it is not required to because the schema fully covers them.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the action ('Search'), the resource ('static analysis defects in the configured Coverity project'), and the returned fields (CID, checker, file, function, impact, status). This distinguishes it from sibling tools like list_projects and get_issue_details, though it does not explicitly name them.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies this tool is used when searching for static-analysis defects, but it gives no explicit guidance on when to prefer it over get_issue_details or when to use alternatives. There are no stated exclusions or prerequisites, leaving usage somewhat to inference.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_issue_detailsA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of behavioral disclosure. It transparently details the return content (event trace, triage info, file/line details) and the 'Get' wording signals a read-only operation. It doesn't explicitly state non-modification or error behavior, but for a getter this is adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single well-structured sentence that leads with the action and resource, then adds meaningful specifics. No filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with two required parameters and no output schema, the description covers the tool's purpose and expected return fields well. It doesn't mention related tools or potential errors, but the essentials for correct invocation and result understanding are present.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, with both parameters already described ('The Coverity Issue ID (CID)' and 'The stream name or ID containing the issue'). The tool description adds no parameter-specific meaning beyond the schema, so baseline 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: 'Get full details for a Coverity defect by CID', and enumerates what's included (event trace, triage information, file/line details). This distinguishes it from siblings like search_issues, which would locate issues rather than fetch full detail for one.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description conveys clear use context: call this when you have a specific CID and want comprehensive defect detail. It doesn't explicitly exclude alternatives or state when to prefer search_issues, but the intent is clear from the wording.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_projectsA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses that the operation is a read-only listing and importantly scopes results to what the authenticated user can access, which is meaningful authorization context. However, it doesn't mention pagination, ordering, or response format behavior — leaving an agent to guess at the return shape.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence with zero wasted words. The verb and resource are front-loaded, and the access-scoping qualifier adds the only needed nuance. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter, no-output-schema listing tool, the description covers the essential operational context: what is listed and under what authorization scope. The only gaps are response format and pagination details, which are minor for such a simple read-only tool and could plausibly follow conventional list semantics.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters, which sets the baseline at 4 per the rubric since there is nothing for the description to explain. The empty schema is fully covered at 100%, and the description correctly avoids fabricating parameter details that don't exist.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('List'), a distinct resource ('Coverity projects'), and a precise scope ('the authenticated user can access'). This clearly distinguishes it from siblings that target different resources: list_streams targets streams, while get_issue_details and search_issues target issues. An agent can select this tool without opening any schemas.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The context is clear — this is the tool for enumerating all projects visible to the authenticated user, and the resource distinction from siblings (streams vs. issues) makes the use case apparent. However, it doesn't explicitly state when not to use it or name an alternative, so it misses the top bar for explicit when/when-not guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_streamsA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of explaining behavior. 'List' implies a read-only enumeration operation, but the description does not explicitly mention lack of side effects, authentication requirements, or what happens if no project is configured. It is adequate but minimal.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no wasted words. It states the action, the object, and the scope in under ten words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter list operation, the description is nearly sufficient. It would be more complete with an explicit note about what is returned (e.g., stream names and metadata), but the lack of parameters and straightforward intent keep the gap small.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the schema is trivially complete and there is nothing for the description to add. A baseline of 4 is appropriate because no parameter ambiguity exists.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('List') and a clear resource ('Coverity streams') scoped to 'the configured project'. This clearly distinguishes it from siblings like list_projects, get_issue_details, and search_issues by identifying a distinct domain object.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'for the configured project' provides clear context for when this tool applies, and the contrast with sibling names makes its role evident. However, it does not explicitly state when not to use it or name alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.