Server Quality Checklist

Tool Scores

• batch_scrapeB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose behavioral traits. It mentions 'efficiently' but lacks details on concurrency, rate limits, error handling, or return format beyond a vague 'list of scraping results'.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Very concise with clear Args/Returns structure. Every sentence is meaningful without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Adequate for a simple batch tool, but missing details on output format, error handling, and limits. With an output schema available, it could rely on that, but the description's return statement remains vague.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds meaning to both parameters: urls is a list of URLs, javascript is a boolean defaulting to False with a clear usage note. This compensates for the 0% schema description coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool scrapes multiple URLs efficiently, distinguishing it from single-URL tools like scrape_url and from crawling or extraction tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool over siblings like crawl_website or extract_data. It does not suggest prerequisites or alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• extract_dataA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description is the sole source of behavioral info. It reveals that JavaScript rendering can be enabled via parameter, and the return type is a dictionary. However, it does not disclose error handling, rate limiting, origin restrictions, or what happens when selectors find no elements. The example provides some clarity but not comprehensive transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a structured docstring with Args, Returns, and Example. It is concise (8 lines) and front-loaded with the purpose. Every sentence adds value, though the example could be slightly shorter. No redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that the tool has 4 parameters and an output schema, the description explains each parameter and the return structure with an example. It lacks details on limitations (e.g., no mention of authentication, timeout, or error handling), but for a standard scraping tool with an output schema, it covers the main aspects well.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must explain each parameter. It describes 'url' as the webpage, 'css_selectors' as list of selectors, 'attributes' as list of attributes to extract (defaults to 'text'), and 'javascript' as a toggle for JS-rendered sites. The example further clarifies usage. This fully compensates for the missing schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'scrape' and 'extract', the resource 'webpage', and the method 'using CSS selectors'. It distinguishes from siblings like batch_scrape (multiple pages) and crawl_website (following links) by focusing on a single-page, targeted extraction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus siblings such as batch_scrape, crawl_website, extract_first, or scrape_url. There is no mention of trade-offs, prerequisites, or exclusions. A tool with clear sibling alternatives should explicitly differentiate usage scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• crawl_websiteA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full weight. It lists parameters (max_pages, max_depth, same_domain_only) that hint at behavior but omits details on robots.txt compliance, rate limiting, or whether the crawl is polite.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is succinct and front-loaded with the main purpose, followed by a parameter list and return statement. Every sentence adds value with no extraneous content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description adequately covers the basic crawling behavior and return type ('Site map with discovered pages and statistics'). However, it lacks specifics on output structure and error handling, which would be helpful given the tool's complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description provides clear explanations for each parameter, including defaults. This adds significant meaning beyond the bare schema titles.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Crawl a website to discover its structure and pages,' using specific verb and resource. It distinguishes from sibling tools like scrape_url and extract_data, which focus on content extraction from single pages.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for site mapping but lacks explicit when-to-use versus alternative tools. No 'do not use' or 'instead use' guidance is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• extract_firstA
  4.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It mentions javascript handling for rendered sites and return format, but omits error handling, what happens on no match, or limits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise yet complete, using bullet-point Args, Returns, and an Example. Every sentence serves a purpose, with no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity, the description covers all necessary aspects: parameter semantics, return value, and usage context. The example solidifies understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description fully compensates by explaining each parameter, including defaults, examples, and attribute choices. This adds substantial value beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Extract the first matching element from a webpage' with a specific verb and resource. It provides examples and contrasts with batch operations, effectively distinguishing from sibling tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states it is for single values like page title or heading, giving clear context for when to use. However, it does not explicitly mention when not to use or directly reference sibling alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• scrape_urlA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Despite no annotations, the description discloses key behaviors: the effect of javascript=True (slower but handles dynamic content), the role of wait_seconds (only when javascript=True), and the return dictionary containing html, status code, and load time. It does not cover rate limits or auth, but given the tool's nature, this is sufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured with clearly labeled Args and Returns sections. Every sentence provides necessary information without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (3 parameters, output schema present) and no annotations, the description completely covers parameter roles, return value, and behavioral notes. No additional context is needed for effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds substantial meaning beyond the input schema: it explains the purpose of each parameter (url as webpage URL, javascript for JS rendering, wait_seconds for JS wait time) and their dependencies. This compensates for the schema's 0% description coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's action ('Scrape a webpage') and outcome ('return its HTML content'), using a specific verb and resource. It is distinct from siblings like batch_scrape (multiple pages) and extract_data (structured extraction).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not explicitly indicate when to use this tool versus alternatives like batch_scrape or crawl_website. Usage context is implied by the tool's name and the mention of 'simple' scraping, but no direct guidance is given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.