Server Quality Checklist

Tool Scores

• open_shortcutC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure but offers minimal information. It states the action but doesn't explain what 'opening' entails (does it launch the app? display details? require specific permissions?), whether it's read-only or has side effects, or what happens if the shortcut doesn't exist.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise - a single sentence that directly states the tool's purpose with zero wasted words. It's front-loaded and efficiently communicates the core functionality without unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no annotations, no output schema, and sibling tools that perform related but distinct operations, the description is insufficiently complete. It doesn't explain what 'opening' means operationally, how it differs from 'running', what the expected outcome is, or address potential error conditions.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, with the single parameter 'name' clearly documented in the schema. The description doesn't add any parameter-specific information beyond what's already in the structured data, so it meets the baseline for high schema coverage without compensating value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Open') and target resource ('a shortcut in the Shortcuts app'), making the purpose immediately understandable. However, it doesn't explicitly differentiate from sibling tools like 'run_shortcut' - both involve interacting with shortcuts but serve different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'run_shortcut' or 'list_shortcuts'. There's no mention of prerequisites, appropriate contexts, or what distinguishes 'opening' from 'running' a shortcut, leaving the agent with insufficient usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• run_shortcutC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden but offers minimal behavioral insight. It mentions 'optional input and output parameters' but doesn't clarify execution effects (e.g., whether it's read-only, requires permissions, has side effects, or handles errors). This is inadequate for a tool that likely performs actions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core action and key details. Every word earns its place with no redundancy, making it highly concise and well-structured for quick understanding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations, no output schema, and a tool that performs an action ('run'), the description is incomplete. It lacks details on execution behavior, return values, error handling, or security implications, which are critical for an agent to use it correctly in context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents both parameters. The description adds no additional meaning beyond implying 'name' can be an identifier and 'input' is optional, which is already covered in the schema. Baseline 3 is appropriate as the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('run') and resource ('shortcut'), specifying it can be executed by name or UUID. It distinguishes from siblings like 'list_shortcuts' (listing) and 'open_shortcut' (opening), though it doesn't explicitly contrast them. The purpose is specific but lacks explicit sibling differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like 'list_shortcuts' or 'open_shortcut'. The description mentions optional input and output parameters but doesn't explain contexts or prerequisites for usage, leaving the agent to infer from tool names alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_shortcutsB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It states the action 'List all available Siri shortcuts' but doesn't describe what 'available' means, whether it requires permissions, how results are returned, or any rate limits. For a tool with zero annotation coverage, this leaves significant behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero waste: 'List all available Siri shortcuts.' It's front-loaded with the core action and resource, making it immediately understandable. No extraneous information or structural issues are present.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (0 parameters, no output schema, no annotations), the description is incomplete. It doesn't explain what 'available' means, how results are structured, or any behavioral constraints. For a listing tool with no structured support, more context about output format or scope would be needed for adequate completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters with 100% schema description coverage, so the schema fully documents the absence of inputs. The description doesn't need to compensate for parameter documentation, and it appropriately doesn't mention parameters. Baseline for 0 parameters is 4, as it avoids unnecessary parameter discussion.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'List' and the resource 'Siri shortcuts' with the qualifier 'all available', making the purpose unambiguous. It distinguishes from siblings open_shortcut and run_shortcut by focusing on listing rather than execution. However, it doesn't specify output format or scope beyond 'available', leaving minor room for improvement.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance is provided on when to use this tool versus alternatives like open_shortcut or run_shortcut. The description implies it's for listing shortcuts, but doesn't clarify prerequisites, context, or exclusions. Usage is implied rather than explicitly stated, which could lead to confusion in tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.