Server Quality Checklist

Tool Scores

• audit_summaryC
  2.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It lists what the summary includes, implying a read-only query operation. However, it does not explicitly state idempotency, side effects, or access requirements. The mention of 'errors' and 'sensitive-path accesses' hints at safety-relevant output but not tool behavior itself.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single front-loaded sentence that efficiently lists key output fields. It avoids unnecessary words and is easy to scan, though it could benefit from a more structured format like bullet points.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple two-parameter input, no output schema, and zero parameter documentation, the description lacks completeness. It only covers the output contents, omitting parameter explanations, usage context, and behavioral details, which are needed for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description does not mention the parameters 'days' or 'project' at all. It provides no meaning or context for these inputs, leaving the agent to guess their purpose based on defaults alone.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's output components (command count, file writes, etc.) and implies it summarizes agent activity. The list differentiates it from sibling tools (cost_summary, efficiency_report, safety_report) by specifying distinct metrics. However, it lacks an explicit action verb like 'Get' or 'Retrieve.'

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus its siblings. The description does not specify context, prerequisites, or exclusions, leaving the agent to infer usage from the output contents alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• efficiency_reportB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden for behavioral disclosure. The description indicates the tool generates a report (presumably read-only) but does not mention side effects, data access permissions, or any other behavioral traits beyond the high-level output.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that conveys the core purpose without any wasted words. It is appropriately concise for the tool's function.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema and 0% parameter coverage, the description should provide more detail on what the report contains (e.g., specific metrics or format). It only gives high-level areas without specificity, leaving the agent with an incomplete understanding of the tool's output.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description does not explain the parameters (days, project). The agent cannot infer what 'days' or 'project' affect without additional information, making parameter usage unclear.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: identifying where tokens are wasted (re-read files, oversized output, cache hit rate) and providing recommendations. This verb+resource specification distinguishes it from sibling report tools like audit_summary, cost_summary, and safety_report.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is used for analyzing token waste and efficiency, but does not explicitly state when to use it vs alternatives. No exclusions or when-not-to-use guidance are provided, making it ambiguous for an agent to choose among siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• cost_summaryA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so the description carries full burden. It discloses the output metrics but does not state behavioral traits like read-only nature, side effects, authentication needs, or whether data is aggregated or real-time. More transparency is needed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, front-loaded with main purpose, no wasted words. Every part is informative.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with no output schema, the description adequately covers what the tool returns. It mentions key metrics and filtering by project. Could explicitly state the return format or that it's a summary object, but overall sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0% (no parameter descriptions). The description hints at project filtering ('top projects') and time scope ('across'), but does not explicitly explain 'days' parameter meaning or default behavior. It adds some value but is incomplete.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool provides a token cost summary across Claude Code projects, listing specific metrics (output/input tokens, cache hit rate, estimated dollars, top projects and tools). This distinguishes it from siblings like audit_summary and efficiency_report.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage is implied for cost analysis, but no explicit guidance on when to use this versus sibling tools (audit_summary, efficiency_report, safety_report). The description lacks when-not-to-use or alternative recommendations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• safety_reportA
  3.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses that tool shows both blocked and unblocked dangerous actions. No annotations exist, so description carries full burden. Missing details like whether empty results are possible, but core behavior is clear.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, no wasted words. Could be slightly improved by front-loading key info, but it is concise and readable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Tool is simple with two optional parameters, but no output schema and no parameter descriptions. While description suffices for basic understanding, it lacks completeness for parameter semantics. Sibling tools are named but not differentiated.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, but description adds no parameter information. The two optional parameters (days, project) are not explained, leaving the agent to guess their purpose (e.g., date range vs. project filtering).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description specifies verb ('reports'), resource ('dangerous actions attempted by agent'), and includes examples (force-push, rm -rf on risky paths, curl|sh, chmod 777). Clearly distinguishes from sibling tools (audit_summary, cost_summary, efficiency_report) by focusing on safety incidents.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Implied usage: use to review dangerous agent attempts. However, no explicit guidance on when to use this tool vs siblings like audit_summary or efficiency_report, nor conditions like minimum severity threshold.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.