Server Quality Checklist

Tool Scores

• scrapyard_mapA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations include readOnlyHint=true and openWorldHint=true, so safety profile is already declared. Description adds that it intentionally does not follow links or crawl, and reports source, count, filtered_count, and entries. This adds behavioral context beyond annotations without contradicting them.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two compact paragraphs. First sentence states purpose and scope immediately. Second paragraph gives usage guidance and output details. No fluff, all sentences earn their place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With an output schema present, the description need not detail return values, and it only briefly mentions the source/count fields. It effectively communicates the tool's non-crawling nature and inventory use case. Slight gap: no mention of rate limits or error behavior, but minor given annotations and schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with descriptions for both parameters. The description mentions the discovery source and counts, and while it implies the output shape, it doesn't add parameter-level syntax beyond schema. Baseline 3 is appropriate when schema covers all parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clear verb+resource: 'Discover a site's published URLs from robots.txt and sitemap XML'. Distinguishes from crawling. Sibling tools include scrapyard_extract and scrapyard_search, and description explicitly states it does not follow page links or crawl, so differentiation is clear.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  States when to use: 'inventory documentation, blog, or product URLs before selecting pages to extract'. Implies alternative scrapyard_extract but doesn't explicitly name alternatives. Provides context but no exclusions or direct comparison to siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• scrapyard_researchA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint and openWorldHint, but the description adds valuable behavioral context: it explains the response structure (engine_used, per-source method, content_source, extraction metadata) and warns about overstating provenance by distinguishing parsed content from snippet fallback. This goes beyond the annotations and helps the agent use results correctly. Minor absence of rate-limit/auth notes, but for a read-only tool this is sufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact—two sentences—yet packs the core action, usage trigger, and behavioral caveat. Every sentence earns its place, and the most critical information (when to use) is front-loaded. No filler or repetition of schema details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given there is a schema for parameters and an output schema (indicated), the description need not restate return types. It still covers when to use, how it differs from siblings, and the key behavioral nuance about provenance. For a read-only, bounded tool, nothing essential is missing—an agent can confidently decide and invoke it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so each parameter (query, n, render, char_limit) already has a clear description. The tool description does not add additional semantics beyond what the schema provides, but it also doesn't conflict. Baseline 3 is appropriate because the schema carries the weight; the description offers only a high-level hint about 'bounded' which is already reflected in min/max constraints.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('Search and extract'), a resource ('several domain-deduplicated sources'), and a scope ('in one bounded request'). It also names sibling tools by implication, distinguishing this combined operation from separate search and extract tools. The purpose is unambiguous and differentiating.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use this tool: 'when the task needs source text rather than only result snippets.' It also provides an exclusion: 'For exhaustive work, search first and extract selected URLs individually instead,' routing the agent to an alternative approach. This is clear, direct guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• scrapyard_searchA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations (readOnlyHint, openWorldHint) cover the read-only and open-world nature, but the description adds valuable behavioral context beyond these: it discloses that the response retains `engine_used`, `adapter_used`, and `cached` flags, enabling the agent to report provenance and caching behavior. It also explicitly states a limitation (does not extract page bodies). No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, each earning its place: purpose, usage guidance with response metadata, and a limitation. The core purpose is front-loaded in the first sentence. No filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity, the presence of an output schema (which obviates the need to detail return values), and the thorough schema descriptions, the description covers all necessary context: when to use, how it behaves, what metadata it returns, and what it does not do. An agent has everything needed to invoke correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so every parameter is already fully explained in the schema (e.g., 'type' enum, 'limit' range, 'query' natural-language advice, 'engine' fallback chain). The tool description adds no parameter-specific detail beyond what the schema provides, so the baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Search'), a resource ('the live web or news index'), and the output ('ranked URLs, titles, and snippets'). It clearly distinguishes from siblings like scrapyard_extract and scrapyard_map by focusing on web/news search rather than extraction or mapping. No ambiguity about what the tool accomplishes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use this tool: 'Prefer this before extraction when you do not already know the target URL.' It also names the alternative (extraction) and provides exclusion guidance ('This tool does not extract page bodies.'). This leaves no inference needed for an agent to decide whether to call search or reactive siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• scrapyard_statusA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The annotations already declare readOnlyHint=true (safe read) and openWorldHint=false (does not depend on external world state). The description adds concrete behavioral context by listing what the tool returns (version, breaker state, registry, counters) and clarifies it is operational status, not a liveness probe. It does not dig into error conditions or rate limits, but given the read-only nature is already covered, a 4 is appropriate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is just two sentences. The first sentence front-loads the return payload; the second gives usage context. Every sentence earns its place with no fluff. The structure is excellent.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple status tool with no parameters and an output schema already provided, the description covers all necessary context: what is returned, when to use it, and how it differs from siblings. Nothing an agent needs to correctly invoke it is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so there is nothing for the description to explain about input. The description instead focuses on output, which is appropriate. The baseline for 0 params is 4, and the description does not need to add anything about parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific verb ('Return') and the exact resources: version, search-engine breaker state, adapter registry, and cache counters. It also explicitly differentiates itself from sibling tools by stating 'not a web search and not merely a liveness probe', which leaves no ambiguity about its role.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit when-to-use scenarios ('when search/research fails, results unexpectedly come from a fallback adapter, or you need to verify which engines are configured and healthy') and also what it is not for ('not a web search and not merely a liveness probe'). This gives an agent clear routing guidance relative to the sibling search/research tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• scrapyard_extractA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  While annotations declare readOnlyHint=true and openWorldHint=true, the description enriches beyond that by explaining what the output contains (provenance, error handling) and how to interpret fields like method, pipeline_steps, upstream_status, and final_url. It also discloses the rejection of private-network URLs. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is four sentences, each with a distinct purpose: primary action, usage context, caution about error handling, and a note on provenance. No fluff; front-loaded with the core function.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that an output schema exists and annotations cover read-only/open-world aspects, the description covers the essential operational details: single-URL fetching, return format (markdown + provenance), error handling, and URL restrictions. Nothing critical is missing for an agent to invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with both url and char_limit having descriptions. The description adds a small additional nuance by mentioning JavaScript-rendered pages, which is not in the schema's url description, but mostly reiterates the schema. Since the schema is strong, this earns a 4 rather than 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a clear verb-resource statement: 'Fetch one public URL and return clean readable markdown plus extraction provenance.' It further specifies the intended inputs (known article, documentation page, JavaScript-rendered page, or PDF) and implicitly differentiates from sibling tools (search, map, research, status) by focusing on a single URL extraction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use: 'Use for a known article, documentation page, JavaScript-rendered page, or PDF.' Also provides actionable guidance like 'Check error before relying on content' and warns that private-network and unsafe URLs are rejected, which helps the agent decide and handle failures correctly.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.