Server Quality Checklist

Tool Scores

• clickB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses the identification method but omits side effects (e.g., navigation, visibility requirements, return value, or error behavior). The description is minimally transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence that conveys the essential purpose and identification method without any fluff or redundancy. Highly concise and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has no annotations and no output schema, yet the description only covers the basic action. It fails to clarify that at least one of selector/text must be provided (both are optional in schema), nor does it explain return values or potential failure modes. For a 3-param tool, this leaves significant gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with each parameter (selector, text, waitForNavigation) having a clear description. The tool description echoes the selector/text content choice but adds no meaning beyond what the schema already provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (click), the resource (element on the page), and how it is identified (CSS selector or text content). This distinguishes it from siblings like fill or evaluate.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage (clicking an element) but offers no explicit guidance on when to choose this tool over alternatives such as fill for form inputs or evaluate for JavaScript. Context is clear but exclusions and alternative mentions are absent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• fillA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden for behavioral disclosure. It only states the core action of typing text and does not mention the default clearing behavior (clearFirst=true) or any side effects like event triggering. This is a significant gap for a mutation operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single clear sentence, front-loaded with the verb, and contains no extraneous information. It is concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is adequate for a simple tool, but it omits key behavioral context such as the default clearing behavior. The schema compensates for parameter details, yet the overall context is still somewhat sparse given the lack of annotations or output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so each parameter has a description. The tool description does not add further parameter-specific meaning, but the baseline of 3 applies because the schema already documents the parameters effectively.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Type text' and the resource types ('input field, textarea, or contenteditable element'), distinguishing it from sibling tools like 'click' or 'navigate'. It is specific and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The intended use is implied by the element types mentioned, but there is no explicit guidance on when to use this tool versus alternatives, nor any exclusions or prerequisites. The description does not reference sibling tools or contrast with them.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• navigateA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden of disclosing behavior. It mentions returning the final URL after redirects, which is useful, but it does not disclose side effects like replacing the current page state or potential navigation failures. Still, the core behavior is reasonably transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that front-loads the main action and returns information without any wasted words or redundant details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple, with only two parameters and no output schema. The description covers purpose and return values but lacks guidance on when to use navigation versus other tools and does not mention state-changing side effects or error conditions. It is adequate but not fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents both parameters (url and waitUntil). The description adds no extra parameter-specific semantics beyond what the schema already provides, so the baseline score of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Navigate the browser to a URL') and the resource, plus the specific return values (page title and final URL after redirects). This distinguishes it from sibling tools like screenshot or click, which have different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives no explicit guidance on when to use this tool versus the sibling tools (e.g., get_page_state, screenshot). It implies the use case of loading a page but does not state exclusions or alternatives, leaving the agent to infer.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• screenshotA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the return format (base64-encoded PNG) but does not disclose side effects, whether the screenshot captures the viewport or full page by default, implications of the savePath parameter, or any waiting/network behavior. This is a significant gap for a tool with no annotation safety hints.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise, consisting of two short sentences that front-load the core purpose. Every word earns its place: 'Take a screenshot of the current page or a specific element' immediately conveys scope, and the return type is specified in the second sentence. No filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is relatively simple, and the schema fully documents all three optional parameters, so the description does not need to repeat those details. However, with no output schema and no annotations, the description should provide more context on return value usage, potential errors, or behavior nuances (e.g., element screenshots may require visibility). The current description is minimal but not fully inadequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema covers 100% of the parameters with descriptions, so the baseline is 3. The tool description adds no additional meaning beyond the schema; it references 'specific element' which aligns with the selector parameter, but does not elaborate on fullPage or savePath semantics. It neither compensates for schema gaps nor adds extra value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Take a screenshot of the current page or a specific element.' It uses a specific verb ('take a screenshot') and names the resources (current page or element). This distinguishes it from sibling tools like navigate, fill, or evaluate, which serve different functions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by mentioning 'current page or a specific element,' but it does not explicitly state when to prefer screenshot over alternatives like get_page_state, nor does it provide exclusions or conditions. There is no 'use this instead of X' guidance, so the agent must infer usage from the tool's purpose.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• evaluateA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden for behavioral disclosure. It mentions 'Execute JavaScript in the browser page context and return the result,' but fails to disclose that arbitrary JavaScript may have side effects on the page, potential error behavior, or that the result format is unspecified. This is a significant gap for a tool that can modify the page.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that front-loads the core function and follows with use cases. Every word earns its place; there is no redundancy or unnecessary detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description provides enough to understand the tool's purpose and general usage, but lacks critical context about potential side effects, error handling, and return value shape. With no annotations and no output schema, the description should do more to fully set expectations for a JavaScript execution tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% with the 'expression' parameter fully described (valid JS expression, examples). The tool description adds usage context but no additional parameter-specific meaning. Per the rubric, high schema coverage sets a baseline of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action: 'Execute JavaScript in the browser page context and return the result.' This clearly distinguishes it from sibling tools like navigate, fill, and click, which handle other browser interactions. It also lists concrete use cases ('assertions, reading DOM values, or custom interactions'), making the tool's purpose unmistakable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Use for assertions, reading DOM values, or custom interactions,' providing clear context for when to use this tool. However, it does not mention when not to use it or name alternative tools (e.g., assert or get_page_state), so it lacks explicit exclusions. This aligns with 'clear context, no exclusions.'

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• assertA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that the tool 'returns pass/fail with details,' which is a key behavioral trait. However, it does not mention whether the tool is read-only, how it behaves on failure (e.g., if it throws or just returns), or any side effects. The disclosure is adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the tool's purpose, and every word earns its place. No fluff or unnecessary detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity and 100% schema coverage, the description is largely complete. It clarifies the return format ('pass/fail with details') even though there is no output schema. It could mention usage relative to get_page_state or waiting behavior, but these are not essential for an assert tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description does not add any parameter-specific details beyond what the schema already provides. The schema's descriptions are clear, but the tool description itself offers no added semantic value for parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Assert a condition about the current page state.' It uses a specific verb (assert) and resource (current page state), and distinguishes itself from sibling tools like get_page_state by mentioning it returns pass/fail and is for verifying test expectations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear usage context: 'use this to verify test expectations.' It does not explicitly compare to alternatives or state when not to use, but this guidance is sufficient for an assertion tool in a test automation context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_page_stateA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses what the tool returns (title, URL, visible text, etc.) and mentions truncated HTML, which gives useful context. However, it does not explicitly state that the tool is non-mutating or describe any side effects, permissions, or limitations beyond truncation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences. The first states what it does and what it returns; the second provides usage guidance. There is no redundancy or extraneous detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read tool with no output schema, the description lists the main return components and usage context. It could be more explicit about response structure or the meaning of 'interactive elements', but it is sufficiently complete for an agent to select and invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Both parameters have 100% schema coverage, so the baseline is 3. The description does not add significant meaning beyond the schema; it only mentions the tool's overall output, not how the parameters affect the result. The schema already explains selector and includeHtml.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool gets the current page state and enumerates specific components (title, URL, visible text, interactive elements, console errors). This is more specific than just 'get state' and distinguishes it from siblings like screenshot (visual) and evaluate (JavaScript).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly advises to use this tool to understand what is on screen before deciding the next action, providing a clear context. It does not explicitly name alternatives or exclusions, but the instruction is enough to guide appropriate use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• close_browserA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description carries the burden of behavioral disclosure. It explicitly mentions 'release all resources,' which goes beyond the simple 'close' and indicates side effects such as freeing memory and connections. While it does not detail all consequences (e.g., state loss), the statement is sufficient for this simple tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence that front-loads the action and includes only essential information. Every word contributes to understanding the tool's purpose and usage.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no parameters, no output schema, and a simple cleanup function, the description covers the necessary information: what it does, what happens (releases resources), and when to use it. It is complete for the tool's complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has no parameters, and the schema coverage is 100%. According to the baseline, a score of 4 is appropriate. The description does not need to explain parameters that do not exist.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool closes the browser and releases all resources, using a specific verb and resource. It distinguishes itself from the sibling tools which are all about navigation, inspection, and interaction rather than session cleanup.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly advises calling this tool when the test session is complete, providing clear timing guidance. There are no alternative tools for closing the browser, so no exclusions are needed.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.