Server Quality Checklist

Tool Scores

• inspect_pageB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already cover read-only/open-world/non-destructive; the description adds that it does not intentionally modify the target or local filesystem, and that allowPrivateNetwork is deliberately scoped to loopback targets - useful detail. However, idempotence (the annotation idempotentHint is false) is not addressed, and any side effects of network read activity are not covered.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Four crisp sentences: intent, output capsules, the private-network safety cap, safety promise. No filler or repetition; a concrete list of returned elements is valuable rather than boilerplate.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description gives a reasonable list of returned elements and a security warning, but does not describe the output format/structure of the result, and does not contrast itself with the comparable viewer scan. Given no output schema, the return type is undefined, so one could get by with more contextual build information.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema describes the 5 parameters, but the text label is only explicitly mapped to it, and adds a safety nuance for allowPrivateNetwork. The 0% schema description coverage means the description must carry semantics for all parameters; viewport, includeForms, allowPrivateNetwork, includeAccessibility are not explained anywhere, making them partly guessable.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Read page through outbound network requests and return...') and resource (page elements) with concrete outputs (headings, forms, buttons, links, inputs, role/name selectors, data-testid candidates). It is clearly distinct from scan_url and the Playwright repro/test tools, though it does not explicitly differentiate itself from sibling inspect/read tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage is implied by the scope (inspect page structure/DOM candidates) and by the conservative network-only fetch, but there is no explicit rule for 'use this instead of scan_url' or any when-not/when-not guidance. Unlike scan_url, it presents concrete candidates for tests, but the exclusivity is not mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• scan_urlB
  3.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already flag readOnlyHint=false and idempotentHint=false, but the description adds valuable behavioral detail: it may write a local screenshot artifact when screenshot:true, and it makes outbound browser and HTTP network requests. It also warns that allowPrivateNetwork:true is only for deliberate loopback development targets. These disclosures go beyond the annotations, giving the agent a realistic picture of side effects and network activity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the primary purpose and then adds usage hints. It is three sentences long, each earning its place: the first lists the checks, the second notes the screenshot side effect, and the third gives format and private-network guidance. No fluff or redundancy; the structure is logical and scannable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (8 parameters, nested objects, no output schema), the description is insufficient. It does not explain the output format beyond mentioning a 'shareable graded report' for markdown, nor does it describe the JSON structure. It omits critical parameters (maxLinks, viewport, weightBudget) and their constraints. Without an output schema, the description should clarify what the result contains, but it doesn't. The description covers the main checks but leaves too many operational details undefined.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must carry the burden of explaining parameters. It mentions a few parameters implicitly (screenshot, format, allowPrivateNetwork) and the 'checks' implied by the list of checks, but it does not describe key parameters like maxLinks, viewport, or weightBudget. The meaning of these nested objects and integers is left entirely to the schema, which lacks descriptions. The description only partially compensates for the missing schema documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Inspect a URL with outbound browser and HTTP network requests' and enumerates a comprehensive list of checks (console errors, links, accessibility, Core Web Vitals, SEO, security headers, etc.). It distinguishes itself from siblings by focusing on network-level scanning and the breadth of checks, though it does not explicitly name alternatives. The verb 'Inspect' plus the specific resource (URL) and scope make the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides some usage context, such as setting format to 'markdown' for a shareable report and noting that allowPrivateNetwork is limited to loopback development targets. However, it does not explicitly state when to use this tool over siblings like inspect_page or run_playwright_test. The guidance is more about configuration than about routing the agent to the right tool, leaving some ambiguity about alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• run_playwright_testA
  3.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the annotations, the description adds valuable behavioral context: it is a code-execution and artifact-writing boundary, requires MCP human-approval elicitation bound to the exact test digest, and may make outbound network requests. This meaningfully explains the side-effect and approval profile beyond the raw annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three short sentences, front-loaded with the core action and followed by the two most important behavioral constraints. No filler or repetition; every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a destructive, open-world code-execution tool with 8 parameters and no output schema, the description omits critical operational details: what is returned (test output, logs, screenshots?), where artifacts are written, how failures surface, and how timeout or environment parameters behave. The approval and network notes are good, but the overall definition is not complete enough for reliable invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0% across 8 parameters, so the description must compensate, but it only vaguely maps to 'code' and 'test file'. It offers no guidance for headed, baseUrl, browser, timeoutMs, allowedEnv, or wallClockTimeoutMs, leaving an agent without enough information to configure the call correctly.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific action—'Execute supplied local Playwright code or a local test file'—and clearly identifies the resource (local Playwright code/tests). This distinguishes it from siblings like generate_playwright_repro (creation), inspect_page (inspection), and scan_url (scanning), making the tool's role immediately obvious.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description conveys clear context: this is the tool for executing local Playwright code or a test file. However, it never explicitly says when to prefer this over inspect_page, scan_url, or generate_playwright_repro, nor does it state any exclusions or prerequisites beyond 'supplied local' code.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• generate_playwright_reproA
  4.4/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  It adds substantial behavior beyond the destructiveHint annotation: it states the exact write location, mandates a relative outputPath, requires overwrite:true to replace existing files, and explicitly discloses that no outbound network request occurs. This safely frames a destructive write operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences carry the full message: the first front-loads the primary action and inputs, and the second adds the critical constraints. No filler or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Although there is no output schema, the description covers the key operational risks: destination, relative path, overwrite behavior, and lack of network access. It does not describe the return value or whether outputPath has a default, but the missing details are minor relative to what is provided for a moderately complex, destructive tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description must compensate for all six parameters. It does clarify url/goal/finding as alternative input sources and gives explicit constraints for outputPath and overwrite, but it leaves testName and the internal structure of the finding object unaddressed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Generate a minimal Playwright test...' and clearly names the output destination. It distinguishes itself from siblings like run_playwright_test by making clear it writes reproductions rather than executing them, and lists the three accepted input forms (scan finding, URL, plain-English goal).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use the tool: when the goal is to generate a reproduction from a finding, URL, or plain-English goal. It does not explicitly name alternative siblings or state when not to use it, so it stops short of full exclusionary guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.