Server Quality Checklist

Tool Scores

• get_security_profilesA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of disclosing behavior. The verb 'get' implies a read-only operation, and the mention of 'Returns' indicates no side effects, but there is no explicit statement about permissions, side effects, or data freshness. This is adequate but lacks the richness seen in well-disclosed tools.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences: the first gives the main purpose, the second elaborates on the content, and the third states the return type. Every sentence earns its place without redundancy, and the key purpose is front-loaded in the first sentence.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that this is a simple zero-parameter getter with an output schema present, the description is complete. It covers what the tool does and what it returns, and the lack of usage guidance is not a completeness gap for such a straightforward operation. The output schema already provides structural detail.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and the schema provides 100% coverage (empty object). With 0 params, the baseline is 4. The description correctly adds no parameter details because there are none, and the return description is sufficient for the tool's purpose.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Get information about all available security profiles' with a specific verb (get) and resource (security profiles). It distinguishes itself from the sibling tool set_security_profile by focusing on retrieval rather than modification, and the scope 'all available' is clear.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It does not mention set_security_profile or any conditions under which this getter should be invoked, such as before setting a profile. The agent must infer usage solely from the tool name and sibling names.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• execute_osascriptA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full behavioral disclosure. It explicitly mentions automatic TCC permission dialog handling, dry-run mode, security risk assessment, error recovery, and logging—significant behavioral details beyond simple execution. It does not cover every possible side effect, but the disclosure is substantial and well above a bare-bones description.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is organized with clear sections (Security Profiles, Features, Args, Returns) and front-loads the core purpose. Some redundancy exists (e.g., Features list partially repeats Args), but each section contains necessary information for a complex tool without excessive verbosity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (5 parameters, output schema present), the description covers the main behavioral aspects: execution scope, security modes, dry-run, and return type. It does not delve into all edge cases or output schema specifics, but the combination of features, security profiles, and parameter explanations provides a solid contextual foundation for invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description fully compensates with an 'Args:' section defining each parameter (script, execution_timeout, security_profile, enable_auto_permissions, dry_run) including defaults and allowed security profile values. This adds meaning beyond the schema, which only lists properties without descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Execute or analyze scripts using osascript with comprehensive functionality and automatic permission handling,' which clearly identifies the action (execute/analyze), resource (osascript scripts), and distinguishes from siblings by also covering security analysis and profile handling. The tool's unified nature makes its purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains security profiles and dry-run mode but never explicitly states when to choose this tool over the sibling tools get_security_profiles or set_security_profile. Usage context is implied through features, but there are no direct alternatives, exclusions, or selection criteria provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• set_security_profileA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that the tool changes the default configuration and that the effect is scoped to calls lacking an explicit profile. It also discloses the return value. It does not mention side effects or permissions, but for a configuration setter, this is adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded. The first sentence states the core purpose, followed by a clarifying sentence about when it applies, and then a well-structured Args/Returns section. Every sentence contributes value without fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter setter with an output schema, the description covers the action, condition of applicability, parameter values, and return behavior. It is complete in context; no additional details are needed for the agent to use this tool safely and correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema only provides a string type with no description or enum. The description compensates by listing the allowed values: 'strict', 'balanced', 'permissive'. This is essential information for the agent to invoke the tool correctly and is not present in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's action: 'Set the default security profile for script execution.' This is a specific verb+resource. It distinguishes itself from siblings by referencing execute_applescript and implying get_security_profiles is for retrieval, so there is no ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains when this setting applies: 'when no explicit profile is specified in execute_applescript calls.' This provides clear context for usage. It does not explicitly name alternatives or exclusions, but the context is sufficient for a simple setter.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.