Server Quality Checklist

Tool Scores

• comet_screenshotB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It states the action but doesn't disclose behavioral traits like whether this requires user permissions, what format the screenshot returns, or if it has side effects. This is inadequate for a tool with zero annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero wasted words. It's front-loaded with the core action and target, making it immediately clear without unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description is incomplete. It doesn't explain what the tool returns (e.g., image data, file path) or any behavioral context, which is insufficient for a tool that likely produces visual output.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters with 100% schema description coverage, so no parameter documentation is needed. The description doesn't add param info, but that's appropriate here, earning a baseline score above minimum viable.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('capture') and target ('screenshot of current page'), making the tool's purpose immediately understandable. However, it doesn't differentiate from sibling tools like comet_ask or comet_mode, which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives or what context it requires. With siblings like comet_ask and comet_poll available, there's no indication of when screenshot capture is appropriate versus other interactions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• comet_stopB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It states the tool stops tasks conditionally, but doesn't disclose behavioral traits like whether this is reversible, what happens to task state/data, permissions required, or side effects. For a potentially disruptive tool with zero annotation coverage, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core action ('Stop the current agent task') and adds a conditional clause for context. Every word earns its place with no redundancy or unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's potential complexity (stopping agent tasks conditionally), no annotations, no output schema, and minimal behavioral disclosure, the description is incomplete. It doesn't explain what 'off track' means, how stopping is implemented, or what the agent should expect after invocation, leaving critical gaps for safe and effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, so no parameter documentation is needed. The description doesn't add parameter details, which is appropriate. Baseline is 4 for zero parameters, as the schema fully covers the absence of inputs.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Stop') and target ('the current agent task'), with a specific condition ('if it's going off track'). It distinguishes from siblings by focusing on task termination rather than communication (comet_ask), connection (comet_connect), mode changes (comet_mode), polling (comet_poll), or screenshots (comet_screenshot). However, it doesn't specify what constitutes 'off track' or the exact mechanism of stopping.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when the agent task is 'going off track', providing some contextual guidance. However, it lacks explicit alternatives (e.g., whether to use comet_ask for clarification first), prerequisites, or clear exclusions (e.g., when not to stop). The guidance is situational but not comprehensive.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• comet_connectB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses that the tool connects to the browser and may auto-start it, which is useful behavioral context. However, it lacks details on potential side effects (e.g., resource usage, permissions needed), error handling, or what 'connect' entails operationally, leaving gaps in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core action ('Connect to Comet browser') and adds a key behavioral note ('auto-starts if needed') without any wasted words. It's appropriately sized for a simple tool with no parameters.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (0 parameters, no annotations, no output schema), the description is minimally adequate. It explains what the tool does but lacks context on when to use it relative to siblings, potential outcomes, or integration details. For a connection tool, more guidance on state management or error scenarios would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters, and the schema description coverage is 100%, so there's no need for parameter documentation in the description. The description appropriately avoids discussing parameters, focusing instead on the tool's action. A baseline of 4 is applied since no parameters exist, and the description doesn't add unnecessary details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Connect to Comet browser') and the conditional behavior ('auto-starts if needed'), which provides a specific verb+resource combination. However, it doesn't explicitly differentiate this tool from its siblings (like comet_ask or comet_stop), which would be needed for a score of 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when needing to connect to the browser, but provides no explicit guidance on when to use this tool versus alternatives (e.g., comet_ask for queries or comet_stop for termination). There's no mention of prerequisites, exclusions, or specific contexts, leaving the agent with minimal direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• comet_pollA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It describes the tool's function as checking status/progress and being callable repeatedly, which is useful context. However, it lacks details on what specific status/progress information is returned, whether there are rate limits for repeated calls, or any error conditions—leaving behavioral gaps for a monitoring tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise—two short sentences that are front-loaded with the core purpose and followed by usage guidance. Every word earns its place with no redundancy or fluff, making it highly efficient and well-structured for quick understanding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (simple monitoring with no parameters) and lack of annotations/output schema, the description is minimally adequate. It covers the what and how-to-use but lacks details on return values, error handling, or specific monitoring aspects. For a tool with no structured data support, this leaves room for improvement in completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters with 100% schema description coverage, so the schema fully documents the lack of inputs. The description adds no parameter information, which is appropriate here. According to the rules, 0 parameters warrants a baseline score of 4, as there's nothing to compensate for and the description doesn't need to cover parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('check agent status and progress') and identifies the resource ('agent'). It distinguishes from siblings by focusing on monitoring rather than asking, connecting, changing modes, capturing screens, or stopping. However, it doesn't explicitly differentiate from all siblings by name, keeping it at a 4 rather than 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for usage ('monitor agentic tasks') and suggests a pattern ('call repeatedly'), which helps guide when to use it. It implies this is for ongoing monitoring rather than one-time actions. However, it doesn't explicitly state when not to use it or name alternatives among siblings, so it falls short of a perfect 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• comet_askA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes key behavioral traits: it's a blocking operation ('wait for the complete response'), mentions use cases involving browser interaction (login walls, dynamic content, forms), and implies it's for complex tasks. It doesn't cover aspects like rate limits or error handling, but provides substantial context beyond basic functionality.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly concise and front-loaded: the first sentence states the core functionality, and the second sentence provides crucial usage context. Every sentence earns its place with no wasted words or redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 3 parameters, 100% schema coverage, but no output schema and no annotations, the description provides good purpose and usage context. However, it doesn't describe what the response looks like (format, structure, potential errors) or address authentication needs, which would be helpful given the browser interaction mention. It's adequate but has clear gaps in output expectations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all three parameters thoroughly. The description doesn't add any parameter-specific information beyond what's in the schema descriptions. This meets the baseline expectation when schema coverage is high.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Send a prompt to Comet/Perplexity and wait for the complete response') and distinguishes it from siblings by specifying it's 'blocking' and 'ideal for tasks requiring real browser interaction or deep research with agentic browsing', which differentiates it from non-blocking or simpler query tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context on when to use this tool ('Ideal for tasks requiring real browser interaction... or deep research with agentic browsing'), which helps differentiate it from alternatives. However, it doesn't explicitly state when not to use it or name specific sibling alternatives for comparison.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• comet_modeA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes the tool's dual behavior: switching modes when a parameter is provided, and querying current mode when no parameter is given. However, it doesn't mention side effects, permissions needed, or rate limits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly concise with two sentences that each serve distinct purposes: the first explains the tool's purpose and mode options, the second explains the optional parameter behavior. Every word earns its place with zero redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter tool with good schema coverage but no output schema or annotations, the description provides excellent context about the tool's purpose, usage patterns, and parameter semantics. The only gap is lack of information about return values or error conditions.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 100% description coverage, so the baseline is 3. The description adds value by explaining what each enum value represents ('search' = basic, 'research' = deep research, etc.), which provides semantic context beyond the schema's technical enum definition.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Switch') and resource ('Perplexity search mode'), and specifies the four possible modes with brief explanations. It also distinguishes this tool from its siblings by focusing on mode switching rather than asking questions, connecting, polling, etc.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use the tool ('Switch Perplexity search mode') and provides clear alternatives for different scenarios: use with a mode parameter to switch modes, or call without mode to see current mode. This gives complete guidance on usage patterns.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.