Server Quality Checklist

Tool Scores

• init-browserC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the action ('Initialize a browser') but doesn't explain what this entails—whether it launches a new browser instance, reuses an existing one, requires specific permissions, has side effects like opening windows, or what happens on failure. This leaves significant gaps in understanding the tool's behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero wasted words. It's front-loaded with the core action and resource, making it easy to parse quickly. Every word earns its place by conveying essential information without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of initializing a browser (which could involve launching processes, handling errors, or managing sessions) and the lack of annotations and output schema, the description is incomplete. It doesn't address what the tool returns, potential errors, or behavioral nuances, leaving the agent with insufficient context for reliable use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, with the 'url' parameter fully documented in the schema as 'The URL to navigate to'. The description adds no additional meaning beyond this, such as URL format examples or constraints. Since the schema does the heavy lifting, the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Initialize a browser') and the target resource ('with a URL'), making the purpose immediately understandable. However, it doesn't differentiate from sibling tools like 'get-screenshot' or 'get-full-dom' which might also involve browser operations, so it doesn't fully distinguish itself from alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'get-screenshot' or 'get-full-dom'. It doesn't mention prerequisites (e.g., needing a browser session first) or exclusions, leaving the agent to infer usage context from the tool name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get-contextB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It states the tool retrieves website context for testcase writing, but doesn't disclose behavioral traits like whether it's read-only, requires browser initialization, has side effects, or what format the context is returned in. This leaves significant gaps for an agent to understand how to use it effectively.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence that efficiently conveys the core purpose without any wasted words. It's front-loaded with the main action and context, making it easy for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is incomplete. It doesn't explain what 'website context' entails (e.g., metadata, state, or specific data), how it relates to sibling tools, or what the return value looks like. For a tool in a browser automation context with siblings like get-full-dom, more detail is needed to guide proper usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, so no parameter documentation is needed. The description appropriately doesn't discuss parameters, focusing instead on the tool's purpose. This meets the baseline for tools with no parameters, as it doesn't add unnecessary details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get the website context which would be used to write the testcase.' It specifies the verb ('Get') and resource ('website context'), and indicates its use for testcase writing. However, it doesn't explicitly differentiate from sibling tools like get-full-dom or get-screenshot, which might provide related but different website information.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides minimal guidance by implying usage in testcase writing contexts, but it lacks explicit when-to-use instructions, alternatives, or exclusions compared to sibling tools. There's no mention of prerequisites, such as whether init-browser must be called first, or how this differs from get-full-dom for DOM retrieval.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get-screenshotB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states what the tool does but lacks details on traits like whether it requires a browser session to be active, if it captures the entire viewport or a specific area, potential delays or failures, or the format of the output (e.g., image data, file path). This is a significant gap for a tool with no annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence with zero waste—it directly states the tool's function without unnecessary words. It is appropriately sized and front-loaded, making it easy for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (capturing a screenshot likely involves browser state and output handling), lack of annotations, and no output schema, the description is incomplete. It doesn't address behavioral aspects like dependencies on an active page or what the output contains, leaving the agent with insufficient context for reliable use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, so no parameter documentation is needed. The description doesn't add param info, which is appropriate here. Baseline is 4 for 0 params, as the description doesn't need to compensate for any schema gaps.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get') and resource ('screenshot of the current page'), making the tool's purpose immediately understandable. It doesn't explicitly differentiate from siblings like 'get-full-dom' or 'get-context', but the specificity of 'screenshot' provides inherent distinction. This is not a tautology of the name and is appropriately specific.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'get-full-dom' (which might provide DOM structure) or 'get-context' (which might provide textual context). It implies usage when a visual capture is needed, but lacks explicit context, prerequisites, or exclusions, leaving the agent to infer based on tool names alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• execute-codeA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden and does well by disclosing key behavioral traits: it executes code asynchronously, requires a specific function structure (async with page parameter), and details the return format (result, logs, errors). It doesn't mention rate limits, auth needs, or destructive effects, but covers execution mechanics adequately for a tool with no annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that directly states the tool's purpose with zero waste. Every word earns its place by specifying the action, code type, and target, making it highly efficient and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given one parameter with full schema coverage and no output schema, the description is reasonably complete. It explains the execution process and return format, compensating for the lack of output schema. However, it doesn't address potential errors or side effects beyond the listed response fields, leaving minor gaps for a code execution tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds value by emphasizing the code must be 'Playwright JS' and executed 'against the current page', providing context beyond the schema's technical details. It doesn't elaborate on parameter syntax beyond the example, but enhances understanding of the tool's scope.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('execute') and resource ('custom Playwright JS code') with the target ('against the current page'). It distinguishes from siblings like get-context or get-screenshot by focusing on code execution rather than retrieval or capture. However, it doesn't explicitly differentiate from init-browser in terms of browser lifecycle management.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for executing custom code on a page, suggesting it's for dynamic interactions beyond simple data fetching. No explicit guidance on when to use vs. alternatives like get-full-dom for static content or prerequisites (e.g., requires an initialized browser). The context is clear but lacks specific exclusions or named alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get-full-domA
  4.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool's purpose and deprecation status but lacks details about what 'full DOM' includes (e.g., structure, limitations), performance implications, or error handling. The description adds some context (deprecation) but doesn't fully compensate for the missing annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise with just two sentences that each serve a distinct purpose: the first states the tool's function, and the second provides critical usage guidance. There is zero wasted text, and the information is front-loaded with the core purpose immediately clear.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 0 parameters, no annotations, and no output schema, the description provides adequate context for a simple, deprecated tool. It clearly explains what the tool does and why it shouldn't be used, which is sufficient for its complexity level. However, it could be more complete by explaining what 'full DOM' entails or linking to documentation for the replacement tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, so no parameter documentation is needed. The description appropriately doesn't discuss parameters, focusing instead on the tool's purpose and usage guidance. This meets the baseline expectation for tools without parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Get the full DOM') and the target resource ('of the current page'), providing a complete verb+resource combination. It also explicitly distinguishes from its sibling 'get-context' by marking itself as deprecated and recommending the alternative, which enhances clarity about its role relative to other tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when not to use this tool ('Deprecated, use get-context instead'), directly naming the alternative sibling tool. This gives clear, actionable advice for tool selection, helping the agent avoid deprecated functionality in favor of the recommended option.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.