Server Quality Checklist

Tool Scores

• defaults-readD
  1.9/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure but provides none. It doesn't indicate whether this is a read-only operation, whether it requires specific permissions, what happens if the domain/key doesn't exist, or what format the output takes. The description is purely syntactic with no behavioral information.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise - a single sentence showing command syntax. There's no wasted verbiage or unnecessary information. However, it's so minimal that it borders on under-specification rather than effective conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no annotations and no output schema, the description is completely inadequate. It doesn't explain what the tool returns, what 'defaults' refers to, or provide any context about the operation. The description fails to compensate for the lack of structured metadata, leaving the agent with insufficient information to understand the tool's purpose and behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 100% description coverage with clear parameter documentation, so the baseline is 3. The description adds minimal value beyond the schema - it shows the command syntax order (domain then key) which isn't in the schema, but doesn't explain what domains or keys represent, provide examples, or clarify the optional nature of the key parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose2/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'use the `defaults read <domain> <key>` command' is tautological - it restates the tool name 'defaults-read' with command syntax but doesn't explain what the tool actually does. It doesn't specify what 'defaults' refers to (system defaults, application defaults, configuration defaults) or what resource is being read. The description fails to provide meaningful purpose beyond the name.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines1/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention sibling tools like 'defaults-write', 'find', or 'list-domains', nor does it provide any context about appropriate use cases. There's no indication of when this tool should be selected over other options.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• findC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden but provides minimal behavioral insight. It mentions searching but doesn't disclose traits like read-only vs. destructive, permissions needed, rate limits, or what 'find' entails (e.g., exact match, partial). This leaves significant gaps for a tool with unknown behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, compact sentence with no wasted words, making it efficient. However, it lacks front-loading of critical details and could be more structured to clarify purpose upfront.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations, no output schema, and a vague purpose, the description is incomplete. It doesn't explain what 'entries container' refers to, the return format, or behavioral context, making it inadequate for effective tool selection and invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with the parameter 'word' documented as 'Word to search for'. The description adds no meaning beyond this, merely restating 'given word'. Baseline 3 is appropriate as the schema handles parameter documentation adequately.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the action ('find') and target ('entries container'), but it's vague about what 'entries container' means and doesn't distinguish from sibling tools like 'list-domains'. The phrase 'given word' is redundant with the parameter name, reducing specificity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like 'list-domains' or 'defaults-read'. The description implies usage for searching by word, but lacks context about prerequisites, exclusions, or comparison to siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• defaults-writeC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It mentions 'write', implying a mutation operation, but doesn't disclose behavioral traits such as permissions required, whether changes are persistent or reversible, potential side effects, or error handling. The description lacks any context beyond the basic command, leaving critical behavioral aspects undocumented.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise—a single sentence that directly states the command. There is no wasted text, and it's front-loaded with the essential information. Every word earns its place, making it highly efficient in structure.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a write operation with no annotations and no output schema, the description is incomplete. It fails to explain what the tool does beyond the command, omits details on return values or errors, and doesn't provide necessary context for safe usage. For a mutation tool with zero annotation coverage, this minimal description is insufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with clear descriptions for 'domain', 'key', and 'value'. The description adds no additional meaning beyond what the schema provides—it merely echoes the parameter names in the command syntax. Since the schema adequately documents parameters, the baseline score of 3 is appropriate, but the description doesn't enhance understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the tool executes a specific command 'defaults write <domain> <key> <value>', which implies writing configuration settings. However, it's vague about what 'defaults' refers to (likely macOS defaults system) and doesn't distinguish from sibling tools like 'defaults-read' beyond the command name difference. It provides a verb+resource but lacks specificity about the resource's nature.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. It doesn't mention when to choose 'defaults-write' over 'defaults-read' or other siblings, nor does it specify prerequisites or appropriate contexts for writing defaults. The description merely restates the command syntax without usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list-domainsB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It only states what the tool does ('List all available macOS domains') without mentioning output format, potential errors, or system dependencies. The 'same as `defaults domains`' hint adds minimal context but doesn't adequately describe behavior for an AI agent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise with just one sentence that directly states the tool's function. Every word earns its place, and there's no unnecessary elaboration or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no annotations, no output schema, and zero parameters, the description is insufficiently complete. It doesn't explain what format the domain list returns, whether there are any constraints or dependencies, or how this integrates with the macOS defaults system context implied by the sibling tools.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters with 100% schema description coverage, so the schema already fully documents the input requirements. The description appropriately doesn't add parameter information beyond what's in the schema, maintaining focus on the tool's purpose.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('List') and resource ('all available macOS domains'), making the purpose unambiguous. However, it doesn't explicitly differentiate from sibling tools like 'find' or explain how this differs from 'defaults-read' for domain-related operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It mentions 'same as `defaults domains`' which is a technical reference but doesn't help an AI agent decide between this tool and siblings like 'defaults-read' or 'find' for domain-related tasks.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.