Server Quality Checklist

Tool Scores

• screenshotB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided; description does not disclose any behavioral traits (e.g., permissions, side effects). It only states the basic function.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence with no wasted words, but could benefit from more structure or context without being verbose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 5 parameters, no output schema, and no annotations, the description is too minimal—fails to explain return behavior or parameter usage beyond what schema provides.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline 3. Description adds minimal meaning beyond schema (e.g., 'window or element' maps to target and selector).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states 'Capture screenshot of Tabby terminal window or specific element', which is a specific verb-resource pair that distinguishes it from sibling tools (execute_js, list_targets, query).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives, no mention of prerequisites or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_targetsA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It accurately describes a read-only operation returning target details, but does not disclose any side effects, permission requirements, or behavior in edge cases (e.g., no targets). The description is truthful but minimally transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with no wasted words. It front-loads the action and immediately conveys what the tool does and what it returns. Achieves high conciseness while remaining clear.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given zero parameters and no output schema, the description adequately specifies the tool's purpose and the information returned (index, URL, WebSocket URL). It could be improved by noting the return format (e.g., array of objects), but still sufficient for a simple list tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and the schema coverage is 100% (vacuously). The description does not need to add param information. Baseline score of 4 is appropriate since nothing is missing.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists available CDP targets (tabs) and specifies the returned information (index, URL, WebSocket URL). It uses a specific verb+resource structure and distinguishes well from sibling tools like execute_js, query, and screenshot, which serve different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no explicit guidance on when to use this tool versus alternatives or any prerequisites. It only states what the tool does, leaving the agent to infer usage context. For a simple enumeration tool with no parameters, this is a noticeable gap.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• queryA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses automatic waiting behavior, but does not mention error handling, return value format, or what happens if the element is not found. This leaves notable gaps in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, front-loading the purpose and adding key behavioral info without any fluff. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description should explain the return value and structure, but it does not. It also lacks error handling details and guidance among siblings, making it incomplete for a tool with 5 parameters and no annotations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds no extra meaning beyond the parameter descriptions already in the schema, providing no additional semantic value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Query DOM elements by CSS selector,' specifying the verb (query), resource (DOM elements), and mechanism (CSS selector). This distinguishes it from sibling tools like execute_js (execute arbitrary JS), list_targets (list available targets), and screenshot (capture image).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description says 'Automatically waits for Angular and element to exist,' which implies it is ideal when elements may load asynchronously. However, it does not explicitly state when not to use it or compare it to siblings, missing explicit exclusion guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• execute_jsA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description must carry behavioral disclosure. It correctly reveals default async IIFE wrapping and await support. However, it omits details on error handling, side effects, or security implications, which are relevant for a code execution tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two efficient sentences: the first front-loads the primary purpose, the second adds critical behavioral detail. No filler, every word contributes.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 3-parameter tool with no output schema or annotations, the description is moderately complete. It covers the main action and the wrap behavior, but lacks information on error handling, return format details, and potential side effects, which are needed given the tool's power.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds context by explaining the wrap parameter's purpose (async IIFE for fresh scope and await support), going beyond the schema. However, it does not add meaning to target or code beyond what the schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool executes JavaScript code in the Tabby terminal context and returns results, distinguishing it from siblings like list_targets (listing) and screenshot (capturing). The mention of async IIFE adds specificity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use this tool (for executing JS code) versus siblings, but lacks explicit when-not or alternative guidance. The context from sibling names helps, but the description itself doesn't provide exclusion criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.