Server Quality Checklist

Tool Scores

• list_spacesB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It states it will 'List all Arc browser spaces' but doesn't specify what 'all' means (e.g., current user's spaces, all accessible spaces), whether there are pagination limits, what format the output takes, or any error conditions. The description is minimal and lacks important operational context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that gets straight to the point with no wasted words. It's front-loaded with the core functionality and doesn't include unnecessary elaboration or redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple listing tool with no parameters and no output schema, the description is adequate but minimal. It tells the agent what the tool does but lacks details about output format, scope limitations, or behavioral characteristics that would be helpful for proper invocation. Given the tool's simplicity, it's minimally viable but could provide more context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters with 100% schema description coverage, so the schema already fully documents the parameter situation. The description appropriately doesn't waste space discussing non-existent parameters, maintaining focus on the tool's purpose. This meets the baseline expectation for parameterless tools.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('List') and resource ('Arc browser spaces'), making the purpose immediately understandable. It doesn't differentiate from siblings like 'list_tabs' or 'focus_space', but it's specific enough to avoid confusion with creation/deletion tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'list_tabs' or 'focus_space'. It doesn't mention prerequisites, timing considerations, or any contextual factors that would help an agent choose between similar listing operations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• focus_spaceB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It mentions AppleScript implementation but doesn't describe what happens during the switch (e.g., does it close current space, require specific permissions, have side effects on tabs, or show error messages for invalid spaces).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero wasted words. It's front-loaded with the core purpose and includes only essential technical context about AppleScript.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a mutation tool with no annotations and no output schema, the description is incomplete. It doesn't explain what happens after switching (success indicators, error conditions, or return values), leaving significant gaps for agent understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents the 'space' parameter fully. The description doesn't add any additional meaning about parameter usage beyond what the schema provides, maintaining the baseline score.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Switch to') and resource ('Arc browser space'), distinguishing it from siblings like 'create_space' or 'list_spaces'. It explicitly mentions the implementation method ('uses AppleScript'), which adds technical context.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when needing to change the active space in Arc browser, but doesn't explicitly state when to use this vs. alternatives like 'create_space' or 'list_spaces'. No guidance on prerequisites or exclusions is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• delete_spaceA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses that the deletion requires an Arc restart to take effect, which is a key behavioral trait. However, it lacks details on permissions needed, error conditions, or what happens to associated data, leaving gaps in behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with zero waste, front-loading the core action and following with a critical behavioral note. Every sentence earns its place by providing essential information efficiently.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description is moderately complete for a destructive tool. It covers the action and a key side effect (restart requirement) but lacks details on permissions, error handling, or return values, which are important for a deletion operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents the 'space' parameter fully. The description adds no additional meaning beyond what the schema provides, such as format examples or constraints, meeting the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Delete') and target resource ('an Arc browser space'), with the method ('by name or ID') distinguishing it from potential alternatives. It precisely communicates what the tool does without being vague or tautological.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context by mentioning 'Requires Arc restart to take effect,' which suggests when to expect results, but it does not explicitly state when to use this tool versus alternatives like 'delete_tab' or 'create_space.' No exclusions or clear alternatives are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• delete_tabA
  3.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It clearly indicates this is a destructive operation ('Delete'), which is crucial for a mutation tool. It also adds valuable context about the restart requirement, which isn't obvious from the schema. However, it doesn't cover potential side effects (e.g., data loss, error handling) or response behavior, leaving some gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise with just two sentences that each serve a clear purpose: the first states the core functionality, and the second provides critical behavioral context. There's zero wasted language, and the most important information (the deletion action) is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a destructive mutation tool with no annotations and no output schema, the description does an adequate but incomplete job. It covers the basic purpose and a key behavioral constraint (restart requirement), but lacks information about what happens after deletion, error conditions, or confirmation of success. Given the tool's potential impact, more completeness would be beneficial.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with the single parameter 'tabId' fully documented in the schema as 'ID of the tab to delete'. The description doesn't add any additional semantic information about the parameter beyond what's already in the schema, so it meets the baseline for high schema coverage without compensating value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Delete a tab') and the target resource ('from Arc browser by its ID'), providing a specific verb+resource combination. However, it doesn't explicitly differentiate from sibling tools like 'delete_space' or 'list_tabs' beyond the obvious resource difference, so it doesn't reach the highest level of sibling distinction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description includes an important usage constraint ('Requires Arc restart to take effect'), which provides context about when effects become visible. However, it doesn't offer guidance on when to use this tool versus alternatives like 'list_tabs' for checking tabs first or 'delete_space' for different resources, nor does it mention prerequisites beyond the ID requirement.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_tabsA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses behavioral traits such as the hierarchical output and inclusion of pinned/unpinned sections, but it lacks details on permissions, rate limits, or response format. The description adds some context but is incomplete for a tool with no annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized with two sentences that are front-loaded and efficient. Each sentence adds value: the first states the purpose and scope, and the second specifies the output details, with zero wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (1 optional parameter, no output schema, no annotations), the description is reasonably complete. It explains what the tool does and the output structure, but it could benefit from more behavioral details (e.g., error handling or default behavior) to be fully comprehensive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents the single parameter 'space' with its description. The description does not add any additional meaning or details about the parameter beyond what the schema provides, resulting in a baseline score of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('List tabs and folders'), the resource ('in an Arc browser space'), and the scope ('with full hierarchy'). It distinguishes from siblings like 'list_spaces' (which lists spaces, not tabs) and 'add_tab' (which adds rather than lists).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by specifying the context ('in an Arc browser space') and what it shows ('pinned and unpinned sections with nested folders'), but it does not explicitly state when to use this tool versus alternatives like 'list_spaces' or provide exclusions. The guidance is contextual but not explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• open_urlA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses the immediate action and AppleScript mechanism, but doesn't mention potential side effects, error conditions, or what happens if the space doesn't exist. It's adequate but lacks depth for a tool that interacts with external applications.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly concise - one sentence that efficiently communicates the core functionality and key optional feature. Every word earns its place with zero waste or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (interacting with external browser via AppleScript), no annotations, and no output schema, the description is reasonably complete. It covers the main action and key optional parameter, though could benefit from more behavioral context about error handling or limitations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters thoroughly. The description adds minimal value by mentioning the optional space parameter but doesn't provide additional context beyond what's in the schema. Baseline 3 is appropriate when schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Open a URL'), the target resource ('in Arc browser'), and the mechanism ('uses AppleScript'). It distinguishes itself from siblings like 'add_tab' by specifying immediate opening in Arc rather than generic tab management.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when needing to open a URL in Arc browser, particularly with optional space targeting. However, it lacks explicit guidance on when to use this versus alternatives like 'add_tab' or 'focus_space', and doesn't mention prerequisites or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_workspaceA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It adds important context about the requirement for 'Arc restart' (a system-level impact) and implies a mutation operation ('Create'), but doesn't cover other behavioral aspects like error handling, permissions needed, or what happens if the workspace already exists. This leaves gaps for a tool with significant side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise (two sentences) and front-loaded with the core purpose. Every sentence adds value: the first defines the tool's function, and the second provides usage context and a critical behavioral constraint ('Requires Arc restart'). There's no wasted text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (a mutation tool with system restart requirement) and no annotations or output schema, the description is incomplete. It covers the purpose and a key behavioral trait but lacks details on error conditions, return values, or prerequisites beyond the restart. For a tool with such impactful behavior, more context would be helpful.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents all parameters. The description doesn't add any parameter-specific details beyond what's in the schema (e.g., it doesn't explain the relationship between 'name' and 'urls' or provide examples for 'icon'). This meets the baseline for high schema coverage but doesn't enhance understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Create a new workspace'), specifies what it creates ('space + tabs'), and distinguishes it from siblings like 'create_space' (which presumably only creates a space) and 'add_tab' (which adds to existing spaces). The phrase 'in one command' further differentiates it from using multiple tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use it ('Useful for setting up a focused context for a ticket or task'), which helps the agent understand appropriate scenarios. However, it doesn't explicitly state when NOT to use it or name specific alternatives among the siblings (e.g., 'create_space' vs. 'add_tab'), which prevents a perfect score.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• add_tabA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes a key behavioral trait: 'Requires Arc restart to see in sidebar,' which is crucial for understanding the tool's impact. However, it doesn't mention other potential behaviors like error conditions, permissions needed, or what happens if the space doesn't exist.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with zero waste: the first states the purpose, and the second provides critical behavioral and usage guidance. Every sentence earns its place, and it's appropriately front-loaded with the core action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (4 parameters, no output schema, no annotations), the description is mostly complete. It covers purpose, key behavioral constraint, and usage guidelines. However, it lacks details on error handling or response format, which would be helpful for a mutation tool with no annotations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, so all parameters are documented in the schema. The description doesn't add any parameter-specific information beyond what's in the schema (e.g., it doesn't explain 'space' or 'url' further). This meets the baseline of 3 when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Add a new tab') and resource ('to an Arc browser space'), distinguishing it from sibling tools like 'open_url' (for immediate opening) and 'create_space' (for creating spaces rather than tabs). It provides a complete verb+resource+scope statement.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use this tool ('Add a new tab to an Arc browser space') and when not to use it ('Requires Arc restart to see in sidebar. For immediate opening, use open_url instead'), providing clear alternatives and exclusions. It directly addresses the trade-off between this tool and 'open_url'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_spaceA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes two important behavioral traits: the restart requirement and sync disabling requirement. However, it doesn't mention error conditions, permissions needed, or what happens on failure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, both essential. The first states the core purpose, the second provides critical behavioral constraints. No wasted words, perfectly front-loaded with the main action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a creation tool with no annotations and no output schema, the description provides good context about prerequisites and side effects. However, it doesn't describe what happens on success (e.g., returns the created space object) or failure conditions, leaving some gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already fully documents both parameters. The description adds no additional parameter information beyond what's in the schema. This meets the baseline expectation when schema coverage is complete.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Create a new Arc browser space') with the exact resource ('space'), distinguishing it from siblings like 'create_workspace' (different resource) or 'add_tab' (different operation). The verb+resource combination is unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit when-to-use guidance: 'Requires Arc restart to take effect' and 'Arc sync must be disabled for changes to persist.' These are crucial prerequisites that tell the agent exactly when this tool is appropriate versus alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.