Server Quality Checklist

Tool Scores

• reset_loopA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It accurately describes the two clearing modes and scopes to 'this project', but stops short of explicitly noting irreversible destruction or permission requirements. The word 'wipe' implies destructiveness, but explicit disclosure would strengthen transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the main action and immediately explaining the parameter options. No redundant or filler content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple two-parameter tool with no output schema, the description covers the core purpose, parameter semantics, and project scope. It doesn't mention return values or side effects beyond clearing, but these are not critical given the schema richness. A small improvement would be an explicit 'cannot be undone' note.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline 3 applies. The description adds slight context (clears just that one vs. wipe everything) but essentially restates the schema. Minimal extra value beyond the existing parameter descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's action with a specific verb ('Clear') and resource ('tracked attempt history'), fully distinguishing it from siblings like report_attempt and loop_status. It also explains the intended outcome (no longer flagged as a repeat).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides clear context for when to use the tool and how to choose between the two parameters (error_signature for one, clear_all=true for everything). It doesn't explicitly name alternative tools, but the usage context is unambiguous.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• confirm_searchA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description carries the burden. It discloses the primary effect ('Clears the block') and the precondition ('error that report_attempt flagged as blocked'), which is useful. It does not cover edge cases like idempotency or failure behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no filler. The first sentence front-loads the trigger condition, the second explains the effect and benefit. Very efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple two-parameter tool with no output schema, the description covers purpose, when, and effect. It lacks explicit error cases or what happens if the signature doesn't match, but overall it's sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description covers 100% of parameters, and the tool description does not repeat or expand on them. The parameter descriptions already include the key relationship to report_attempt, so the description adds no extra semantic value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses specific language: 'Call this after using your own web search tool to look up an error that report_attempt flagged as blocked. Clears the block and lets you resume fixing.' This clearly identifies the action (clearing a block) and the resource (the error/loop state), and differentiates it from siblings like report_attempt and reset_loop.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly states the temporal placement: 'after using your own web search tool' and ties to report_attempt's flag, giving clear context for when to invoke. However, it does not mention alternative tools or explicitly say when not to use it, leaving the full decision tree to the agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• report_attemptA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses key behavioral aspects: the tool returns whether the exact problem has been tried before and whether web search is now required before retrying. It also implies statefulness by tracking previous attempts. This goes beyond a simple 'report' verb and gives context about consequences.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, with the first stating purpose and return values, and the second providing usage timing. Every sentence adds value, and it is appropriately front-loaded. No redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers purpose, usage timing, and return values, which is sufficient for a simple two-parameter tool with no output schema. It lacks exact return types (e.g., booleans) but the description 'returns whether...' is clear enough. Given the tool's simplicity, it is nearly complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, and the parameter descriptions are already clear (error_signature: 'The error message, test failure, or symptom text'; fix_summary: 'One-line description of the fix you just attempted'). The tool description adds no additional meaning beyond these, matching the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Report' and the resource 'that a fix attempt for an error/bug just failed.' It also explains what the tool returns (whether the problem has been tried before and whether web search is required), which distinguishes it from sibling tools like confirm_search, loop_status, and reset_loop. The purpose is specific and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit timing: 'Call this immediately after any failed fix, before writing more code.' This provides clear context for use. However, it does not explicitly mention alternatives or when not to use it, though the context strongly implies it is only for failed fixes.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• loop_statusA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of disclosure. It explicitly says 'View', indicating a non-destructive operation, and adds context about project scope and sorting order. It does not describe output format, but this is not critical for a simple status listing.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one efficient sentence with no filler. It front-loads the action and includes relevant details (project scope, sorting) without waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple no-parameter, read-only tool, the description adequately covers what the tool does and when to use it. It lacks an explicit output description, but since there is no output schema, this is a minor gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  This tool has zero parameters, so schema coverage is 100% and the description does not need to explain parameters. The baseline of 4 applies because there is nothing to document.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: viewing all currently tracked failed-attempt histories for the project, sorted by repeat count. This specific verb-resource combination distinguishes it from siblings like report_attempt and reset_loop.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives a clear use context ('checking what's been stuck') and implies it is for read-only status inspection. However, it does not explicitly mention alternatives or when not to use this tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.