Server Quality Checklist

Tool Scores

• scrapeA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description carries the behavioral disclosure burden. It adds the claims of being the 'lowest-cost call' and returning 'main content as Markdown,' which provide some insight. However, it omits important behaviors like handling redirects, size limits, or error scenarios. The parameter descriptions complete the picture for the browser parameter, but the tool description itself lacks comprehensive behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise: two sentences with no wasted words. The first sentence directly states the action and output; the second adds a cost claim and use case. Every sentence is meaningful and earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with two parameters, no output schema, and no annotations, the description covers the core purpose and output format. It mentions the key extra feature of cost. However, it could be more complete by hinting at limitations or example use cases. Still, it is fairly complete for its complexity level.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The tool description does not add meaning beyond the parameter descriptions in the schema (url and browser). It repeats the general idea of fetching a URL but provides no new semantic detail.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly specifies the action ('Fetch a URL'), the resource ('return its main content'), and the output format ('LLM-ready Markdown'). It also distinguishes this tool as the 'lowest-cost call,' which differentiates it from sibling tools like crawl or extract.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for simple page reading ('useful for reading a page into context'), but provides no explicit guidance on when to use this tool versus alternatives (e.g., crawl for multiple pages, extract for structured data). No when-not-to-use information is given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• mapA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden for behavioral traits. It discloses the method (sitemap + link extraction) and output format (newline-separated list). However, it does not cover potential issues like rate limits, authentication, or behavior when sitemap is missing. The schema covers parameters well, but behavioral context is sparse.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise at one sentence, front-loading the core action and output. Every word serves a purpose with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 4 well-documented parameters, no output schema, and no annotations, the description provides enough information to use the tool effectively. It explains the output format, which is not in the schema. However, it lacks guidance on when to use this tool versus siblings, slightly reducing completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with descriptions for all 4 parameters. The description adds minimal additional meaning beyond the schema, only mentioning the newline-separated output format. It does not elaborate on parameter usage or constraints beyond what the schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Discover URLs on a site via sitemap.xml + root-page link extraction.' It specifies the method and output format, distinguishing it from sibling tools like 'crawl' or 'scrape' which have different behaviors.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not provide explicit when-to-use or when-not-to-use guidance compared to siblings. While it mentions the method, it lacks context on when 'map' is preferred over 'crawl' or 'extract'. The sibling tool names are given but no differentiation is described.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• extractA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description reveals it uses WebReaper and returns JSON Lines, but lacks details on error handling, rate limits, or side effects. The `browser` parameter behavior is documented in parameters but not in the main description. Without annotations, the description carries the disclosure burden but is only partially transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the core purpose. Every sentence adds value—defining inputs, schema format, and outputs. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema and moderate complexity, the description covers inputs, schema format, and output format. It does not mention potential failure modes or prerequisites (e.g., browser installation), but the parameter description fills in the browser requirement. Overall, it is adequately complete for basic use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, and the description adds clarity by explaining the schema shape and referencing WebReaper docs. This goes beyond the schema's property descriptions, which are mostly self-explanatory. The browser parameter's default behavior is noted in the description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Extract structured fields from a URL using a JSON schema.' It specifies the schema shape (WebReaper) and return format (JSON Lines), distinguishing it from sibling tools like `extract_inferred` and `extract_with_prompt`.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when you have a JSON schema, but does not explicitly state when to choose this over siblings or provide exclusions. No guidance on prerequisites (e.g., need for Chromium if using browser parameter), though that is in the parameter description.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• extract_inferredA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden. It discloses the use of LLM for schema inference, deterministic extraction, dependencies on environment variables, and the cost/consistency benefits. It does not mention potential failure modes, but overall provides good behavioral insight beyond just the basic purpose.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise (3 sentences) and front-loaded with key information: the core behavior, comparison to alternatives, and configuration requirements. Every sentence adds value with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the tool's purpose, mechanism, comparison to a sibling, and configuration needs. It mentions the output format (JSON Lines). However, it does not address what happens on failure (e.g., if LLM call fails or page is inaccessible) or provide details on error handling. Minor gap for a tool with no output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%; the schema already describes each parameter. The description mentions the goal parameter ('optionally steered by a goal') but does not add significant meaning beyond what the schema provides. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool extracts structured data from a URL without writing a schema, using an LLM to infer the schema. It distinguishes itself from sibling tools, especially extract_with_prompt, by noting it's cheaper and more consistent for similarly shaped pages.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context on when to use this tool (when no schema is needed, for similarly shaped pages) and mentions requirements (OpenAI-compatible LLM endpoint). It explicitly compares to extract_with_prompt but does not address other siblings like crawl, scrape, or map, nor does it state when not to use this tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• extract_with_promptA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully discloses behavior: uses LLM, costs one call, returns JSON Lines, requires specific environment variables, optional browser flag. Does not mention error handling, but sufficient for selection.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Concise and well-structured: first sentence states purpose, then configuration details, then optional parameters. No fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description covers return format (JSON Lines), cost, and all prerequisites. It is complete for an understanding of tool capabilities and requirements.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, but description adds value beyond schemas: model parameter includes override explanation and API key clarification; browser parameter adds 'auto-spawns system Chrome'; prompt parameter includes example. Justifies score above baseline 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool extracts structured data using natural-language instructions, distinguishing it from sibling 'extract' which uses a CSS schema. The verb 'Extract' and resource 'URL with an LLM' are specific.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly describes when to use (natural-language extraction) and provides configuration prerequisites (LLM setup). Mentions optional model override. Lacks explicit when-not-to-use for siblings like extract_inferred, but still clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• crawlA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses critical behavioral traits: single long blocking call, MCP progress notifications, bounded max pages (default 50, hard cap 1000). Since no annotations are provided, the description fully carries the burden of transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise yet packed with essential information. It is front-loaded with purpose, followed by warnings and alternatives. Every sentence earns its place without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (blocking crawl, progress notifications, resource-heavy), the description adequately covers the blocking nature, bounds, progress reporting, and alternative approach. The return format is stated as 'Markdown record per page as JSON Lines', which is sufficient. Minor gaps in error handling and rate limiting are acceptable for this context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% coverage with descriptions for all 3 parameters. The description reinforces the maxPages cap and default but does not add new semantic information beyond what the schema already provides. Baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('crawl'), resource ('whole site'), and outcome ('return one Markdown record per page as JSON Lines'). It explicitly distinguishes from siblings like 'map' and 'scrape' by suggesting an alternative approach for large sites.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit when-to-use guidance (crawl entire site) and when-not-to ('For a large site prefer map to list URLs, then scrape each URL'). It also warns about the blocking nature and gives an alternative strategy.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.