Server Quality Checklist

Tool Scores

• debug_errorA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations declare readOnlyHint=true, and the description confirms this by stating it only analyzes and returns root cause and fixes. It adds context about the return format (up to 3 fixes with code edits) and supported languages. No contradictions or missing behavioral details that are critical.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded with the core purpose. It uses efficient sentences and a clear list of use cases. Every sentence adds value without redundancy, and the structure guides the agent on how to use the tool in sequence with a sibling.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (4 params, 1 required, no output schema), the description provides sufficient context: it explains the return value (root cause + ranked fixes), supported languages, and instructs the agent on the follow-up action (report_outcome). It is complete for the agent to use correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description does not add significant meaning beyond the schema; it mentions auto-detection for language and the need for full error text, but these are already in the schema descriptions. Therefore, it does not exceed baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's verb ('Analyze') and resource ('runtime error, exception, or stack trace') and distinguishes it from the sibling 'report_outcome' which is for reporting outcomes. It leaves no ambiguity about what the tool does.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly lists when to use the tool, including specific user phrasings and error types. It also provides a workflow instruction to report outcomes via the sibling tool. However, it does not explicitly mention when not to use it or list alternatives, keeping the score from being a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• report_outcomeA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The annotation has idempotentHint=true, and the description advises calling once, which is consistent. The description adds behavioral context about the feedback loop ('failed-fix follow-ups directly improve future answers') and the memory effect for rank-1 fixes, going beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, front-loaded with purpose, no redundant information, and efficient. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers what, when, how, why, and references the sibling tool. It lacks the return value description, but there is no output schema. Given the tool's simplicity, the context is largely complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description does not add new parameter details beyond schema, only contextualizes usage (e.g., 'passing the debug_log_id from that response'). No additional semantic depth.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Report whether a DebugAI fix actually worked'. It specifies the verb 'report' and resource 'fix outcome', and distinguishes from the sibling tool 'debug_error' by indicating it is a follow-up after applying a fix.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit instructions: 'Call this ONCE after applying (or abandoning) a fix from debug_error'. Provides context for when to use (post-fix) and why (to improve future answers and team memory). No ambiguity.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.