Server Quality Checklist

Tool Scores

• scan_junkD
  1.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must cover behavior. It says 'Scan' which implies a read-only operation, but it does not explicitly state that it does not modify anything, nor does it disclose what it returns (e.g., a list of caches, sizes). No permissions or side effects are mentioned, leaving significant behavioral ambiguity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness2/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence, which is concise, but it is under-specified rather than efficiently detailed. It front-loads the verb 'Scan' but provides no useful structure or breakdown of capabilities. It is more an under-specification than a model of conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations, no output schema, and a parameter with zero schema descriptions, the description is grossly incomplete. An agent cannot determine what the tool returns, how to use the categories parameter, or how this tool relates to cleaning and disk usage. Essential information is entirely absent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The only parameter, 'categories', is an array with enum values, but the schema lacks any descriptions (coverage 0%). The description does not mention the parameter at all, so the agent has no idea what values to pass or what they control. Description fails to compensate for the schema gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the verb 'Scan' and the resource 'macOS developer/system caches and reclaimable space', which conveys a broad intent but is vague. It does not distinguish from sibling tools like clean_junk or get_disk_usage, and the phrase 'reclaimable space' is not a concrete resource. It is not a tautology, but it lacks specificity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines1/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives such as clean_junk or get_disk_usage. There is no mention of prerequisites, typical use cases, or exclusions. The agent is left to infer the tool's role in the workflow.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_disk_usageC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It implies a read-only operation ('report') but doesn't describe the output format, whether it scans the entire disk or just a mount point, or any side effects. This minimal disclosure is insufficient for a tool with no annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that fronts the core action. There is no fluff. However, its brevity omits essential context, making it minimal rather than well-structured; it earns its place but doesn't provide enough value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with one optional parameter and no output schema, the description is incomplete. It fails to clarify whether the report covers the whole disk, a specific path, or what the output looks like. It doesn't explain how the path parameter affects results, leaving the agent without enough information to use the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for the undocumented 'path' parameter. However, the description makes no mention of 'path' whatsoever. An agent receives no explanation of what the parameter does, its default value, or how it affects the report.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Report disk capacity and free space.' It uses a specific verb (report) and resource (disk usage), and the subject matter distinctly separates it from siblings scan_junk and clean_junk, which deal with junk files. However, it doesn't explicitly mention the path parameter or differentiate itself by name, so it falls short of a 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. The description doesn't mention the siblings or any conditions for when checking disk usage would be appropriate, leaving the agent to infer when this tool should be invoked.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• clean_junkA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose behavior on its own. It hints at safety with 'Safely' and the dry-run instruction, which implies this is a destructive operation that can be previewed. But it does not state what gets permanently destroyed, whether confirmation is needed, or any permission requirements.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two short, front-loaded sentences deliver purpose and the critical safety tip with zero waste. The instruction is immediately actionable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is a simple mutation with two parameters and no output schema. The description covers the essential safety guidance but omits what happens after running (e.g., return value, confirmation prompts) and does not specify any prerequisites or side effects beyond the dry-run caution. It is adequate but not exhaustive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description should compensate, but it adds no parameter-specific information. The schema's enum values (e.g., 'derivedData', 'brew') are self-explanatory, and the default for dryRun is documented in the schema, but the description does not clarify how parameters interact or what the consequence of each selection is.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear verb ('remove') and resource ('selected caches'), and the qualifier 'selected' signals user-controlled scope. It distinguishes this from scan_junk (which likely scans) and get_disk_usage (which measures usage) via the explicit removal action.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The instruction 'Always dry-run first' provides an important safety guideline for usage. However, it does not explicitly mention when to use this tool versus alternatives (e.g., 'run scan_junk first to identify caches'), leaving that routing implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.