Server Quality Checklist

Tool Scores

• list_targetsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It states the action ('List') but lacks behavioral details such as whether this requires an active browser connection, if it's read-only (implied but not confirmed), what the output format looks like (e.g., list of objects with IDs), or any rate limits. The description is minimal and doesn't compensate for the absence of annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose ('List all available browser targets') and adds clarifying examples ('pages, service workers, etc.') without unnecessary details. Every word earns its place, making it highly concise and well-structured for quick understanding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (1 parameter, 100% schema coverage, no output schema), the description is incomplete. It lacks context on behavioral aspects like connection requirements, output format, or error handling. Without annotations or an output schema, the description should provide more guidance on what to expect after invocation, but it does not, leaving gaps for an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with the single parameter 'port' fully documented in the schema as 'Chrome CDP port' with a default. The description adds no parameter-specific information beyond what the schema provides, such as explaining why the port matters or how it relates to targets. Baseline score of 3 is appropriate since the schema handles the parameter documentation adequately.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('List') and resource ('all available browser targets'), with examples of what targets include ('pages, service workers, etc.'). It distinguishes from siblings like 'execute_js' or 'navigate' by focusing on enumeration rather than interaction or navigation. However, it doesn't explicitly differentiate from 'get_console_logs', which might also involve listing logs from targets, slightly reducing specificity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing a browser instance connected), exclusions (e.g., not for modifying targets), or suggest sibling tools like 'navigate' for interacting with listed pages. Usage is implied as a starting point for target selection but not explicitly stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• execute_jsB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions that the tool 'Returns the result of the expression,' which is helpful, but lacks critical details such as execution context (e.g., sandboxing, permissions), error handling, timeout behavior, or security implications. For a tool that executes arbitrary JavaScript code, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized with three sentences that are front-loaded: the first states the core functionality, the second explains the return value, and the third provides usage examples. There's no wasted text, though the examples could be more structured or prioritized.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of executing JavaScript code in a page context (with no annotations and no output schema), the description is incomplete. It lacks details on execution safety, error responses, or output formatting. The examples help but don't compensate for missing behavioral context, making this inadequate for a tool with such potential impact.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, providing clear documentation for all three parameters (code, targetIndex, port). The description doesn't add any parameter-specific information beyond what's in the schema, such as code examples or constraints. With high schema coverage, the baseline score of 3 is appropriate as the schema handles the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Execute JavaScript code') and resource ('in the page context'), with a specific verb+resource combination. However, it doesn't explicitly distinguish this tool from its siblings (get_console_logs, list_targets, navigate), which all operate in the same browser automation context but serve different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides implied usage guidance through examples ('clicking buttons, filling forms, or calling page functions'), suggesting this tool is for interacting with page elements. However, it doesn't explicitly state when to use this tool versus alternatives like navigate for page navigation or list_targets for target selection, nor does it mention any exclusions or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• navigateB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions the reload functionality but fails to describe other critical behaviors such as navigation errors, timeouts, authentication needs, or what happens with invalid URLs. For a tool with no annotation coverage, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise and front-loaded, consisting of two clear sentences that directly convey the tool's functionality without any wasted words. Every sentence earns its place by providing essential information efficiently.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a navigation tool with no annotations and no output schema, the description is incomplete. It doesn't explain return values, error handling, or behavioral nuances like page loading states. For a tool that interacts with web pages, more context is needed to ensure proper usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description adds minimal value by reiterating the 'reload' option for the URL parameter but doesn't provide additional meaning beyond what's in the schema. This meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Navigate to a URL or reload the current page.' It specifies the verb ('navigate'/'reload') and resource ('URL'/'current page'), making the action explicit. However, it doesn't distinguish this from sibling tools like 'list_targets' or 'execute_js', which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides implied usage guidance by mentioning 'Use "reload" as URL to refresh,' which suggests when to use this specific functionality. However, it lacks explicit guidance on when to choose this tool over alternatives like 'execute_js' or 'list_targets', and doesn't mention prerequisites or exclusions, leaving gaps in context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_console_logsB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It adds important context about 'Starts monitoring on first call' which reveals this isn't just a passive read but initiates an active monitoring session. However, it doesn't disclose other critical behaviors: whether this is read-only or has side effects, authentication needs, rate limits, or what happens on subsequent calls. For a tool with no annotations, this leaves significant gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise (two short sentences) with zero wasted words. The first sentence states the core purpose, and the second adds critical behavioral context. Every sentence earns its place, and the information is front-loaded appropriately.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 4 parameters with 100% schema coverage but no annotations and no output schema, the description is minimally adequate. It covers the basic purpose and one behavioral aspect (monitoring initiation) but leaves significant gaps: no output format description, no error handling information, no guidance on tool sequencing with siblings, and incomplete behavioral transparency. For a tool that initiates monitoring, this is borderline sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already fully documents all 4 parameters. The description adds no additional parameter semantics beyond what's in the schema - it doesn't explain relationships between parameters, provide examples, or clarify edge cases. With complete schema coverage, the baseline is 3 even without parameter details in the description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get console logs') and resource ('from a browser target'), providing a specific verb+resource combination. However, it doesn't explicitly differentiate from sibling tools like list_targets or navigate, which would require a 5. The phrase 'Starts monitoring on first call' adds useful behavioral context but doesn't address sibling differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like list_targets (which might provide target indices) or execute_js (which might generate logs). There's no mention of prerequisites (e.g., needing to call list_targets first to get targetIndex) or contextual constraints. The single sentence offers no usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.