Server Quality Checklist

Tool Scores

• open_with_appB
  3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations provide key behavioral hints: readOnlyHint=false (implies mutation), openWorldHint=true (suggests external app interaction), idempotentHint=false (non-idempotent), destructiveHint=false (safe). The description adds minimal context beyond this, stating the action but not detailing side effects (e.g., app launching, file locking) or constraints (e.g., app availability). It doesn't contradict annotations, but offers little extra behavioral insight.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence that directly states the tool's function without unnecessary words. It's front-loaded and efficiently conveys the core purpose, making it easy for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (interacting with external apps, mutating state per annotations), lack of output schema, and poor parameter documentation, the description is insufficient. It doesn't address return values, error conditions, or practical usage details needed for reliable agent invocation, leaving significant gaps in understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the schema provides no parameter details. The description mentions 'appName' and 'filePath' implicitly but doesn't explain their semantics (e.g., format of filePath, valid app names). It fails to compensate for the schema's lack of documentation, leaving parameters largely undefined.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Open') and the target ('a file or folder with a specific application'), making the purpose immediately understandable. However, it doesn't explicitly differentiate from sibling tools like 'launch_app' (which might launch an app without opening a file) or 'list_applications' (which lists apps), leaving room for ambiguity in tool selection.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'launch_app' or 'list_applications'. It lacks context about prerequisites (e.g., whether the app must be installed) or exclusions (e.g., not for remote files), leaving the agent to infer usage based on the name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• launch_appB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations provide readOnlyHint=false, destructiveHint=false, and openWorldHint=true, indicating this is a non-destructive, non-idempotent operation that may have side effects. The description adds that it launches applications, which aligns with these annotations but doesn't provide additional behavioral context like what happens if the app is already running, permission requirements, or system-level effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence with zero wasted words. It's front-loaded with the core functionality and appropriately sized for a simple tool with one parameter.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no output schema and minimal annotations, the description covers the basic action but lacks details about return values, error conditions, or system dependencies. It's adequate for the simplest use case but doesn't provide enough context for robust agent decision-making given the tool's potential side effects.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage for the single parameter 'appName', the description doesn't add any parameter-specific information beyond what's implied by the tool name. It mentions 'by name' which echoes the parameter name but doesn't clarify format, examples, or constraints. The baseline is appropriate given the schema's lack of descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Launch') and resource ('a Mac application by name'), making the purpose immediately understandable. It doesn't differentiate from sibling tools like 'list_applications' or 'open_with_app', but the verb+resource combination is specific enough for basic understanding.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'open_with_app' or 'list_applications'. There's no mention of prerequisites, context for launching applications, or any exclusions that would help an agent choose between available options.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_applicationsA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds valuable context beyond annotations by specifying the scope ('/Applications folder'), which annotations don't cover. Annotations already declare readOnlyHint=true, destructiveHint=false, etc., so the description appropriately supplements rather than contradicts them.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose ('List all applications') and adds necessary scope detail ('installed in the /Applications folder') with zero wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read-only tool with no parameters and good annotations, the description is mostly complete. However, without an output schema, it could benefit from mentioning the return format (e.g., list of app names/paths), though the scope detail partially compensates.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0 parameters and 100% schema description coverage, the baseline is 4. The description confirms no parameters are needed, as it describes a simple listing operation without any filtering or options.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('List all applications') and resource ('installed in the /Applications folder'), distinguishing it from siblings like 'launch_app' and 'open_with_app' which perform different operations on applications.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context by specifying '/Applications folder', but provides no explicit guidance on when to use this tool versus alternatives like 'launch_app' or 'open_with_app', nor any prerequisites or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.