Server Quality Checklist

Tool Scores

• smartscraperA
  4.4/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds substantial behavioral context beyond what annotations provide. While annotations indicate read-only/idempotent/non-destructive operations, the description adds crucial details: cost (10 credits per page), processing time implications, rate limits (size limits of 2MB for HTML/markdown), authentication needs (none mentioned), and specific behavioral traits like mutual exclusivity of input sources, scroll behavior, pagination handling, and JavaScript rendering options.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely long (over 800 words) with extensive parameter documentation that might be better placed in a separate reference. While well-structured with clear sections, it's not front-loaded - the core purpose gets buried in verbose parameter details. Some sentences could be more concise while maintaining clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (9 parameters, AI-powered extraction, multiple input modes) and the presence of an output schema, the description is remarkably complete. It covers all parameters thoroughly, explains the return structure, documents errors/exceptions, provides cost information, and gives practical examples throughout. The output schema existence means the description doesn't need to detail return values, which it appropriately delegates.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage for 9 parameters, the description carries the full burden of explaining parameter semantics and does so comprehensively. Each parameter gets detailed explanations with examples, constraints, defaults, and practical usage guidance. The description transforms what would be opaque parameters into well-understood inputs.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose as 'Extract structured data from a webpage, HTML, or markdown using AI-powered extraction' with specific examples of use cases (product info, contact details, etc.). It distinguishes from siblings by mentioning 'AI-powered extraction' and 'structured data', but doesn't explicitly differentiate from all sibling tools like 'scrape' or 'searchscraper'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context about when to use this tool (for AI-powered structured extraction from web content) and includes some usage tips. However, it doesn't explicitly state when NOT to use it or name specific alternatives among the sibling tools, though it implies this is for structured extraction vs. other scraping approaches.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• markdownifyA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds significant behavioral context beyond what annotations provide. While annotations indicate read-only, idempotent, and non-destructive operations, the description adds: cost information ('Costs 2 credits per page'), timeout behavior ('>120 seconds'), error conditions (ValueError, HTTPError, TimeoutError), and specific constraints about protocol requirements. This provides rich operational context not captured in annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (purpose, usage, cost, args, returns, raises) and front-loaded key information. While comprehensive, it's appropriately sized for a tool with complex behavior and parameter requirements. Some sentences could potentially be more concise, but overall structure is excellent.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity, the description provides complete context. It covers purpose, usage guidelines, behavioral traits, parameter semantics, return values, and error conditions. With an output schema present, the return value documentation is appropriately detailed but not redundant. The description addresses all aspects needed for effective tool use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description fully compensates by providing extensive parameter documentation. It explains the website_url parameter's format requirements, provides valid and invalid examples, and details content type support. This adds substantial meaning beyond the bare schema definition.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verb ('convert') and resource ('webpage into clean, formatted markdown'). It distinguishes from sibling tools by specifying this is specifically for markdown conversion rather than general scraping or crawling operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool ('Useful for extracting content from documentation, articles, and web pages for further processing'). However, it doesn't explicitly state when NOT to use it or name specific alternatives among the sibling tools, though the purpose differentiation implies alternatives exist.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• scrapeA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, covering safety and idempotency. The description adds valuable context beyond this: it specifies cost (1 credit/page), performance impact (2-5 sec vs. 15-30 sec), and that it's 'lower cost than AI extraction,' which helps the agent make informed decisions without contradicting annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with sections (Args, Returns, Raises, Use Cases, Note) and front-loaded key information. However, it is lengthy with detailed examples and lists; while informative, some redundancy (e.g., repeating cost info) slightly reduces efficiency, though every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (web scraping with JS rendering), the description is highly complete. It covers purpose, usage, parameters, returns (though output schema exists), error handling, and sibling differentiation. With annotations and output schema provided, the description adds comprehensive context without gaps, making it fully adequate for agent use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description fully compensates by detailing both parameters. For website_url, it explains requirements (protocol inclusion), examples, supported/unsupported protocols, and behavior. For render_heavy_js, it provides default values, usage scenarios, performance impacts, and cost implications, adding significant meaning beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Fetch raw page content from any URL with optional JavaScript rendering' and 'performs basic web scraping to retrieve the raw HTML content of a webpage.' It distinguishes from siblings by mentioning alternatives like smartscraper for structured data extraction and markdownify for clean content, making the scope specific.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use this tool vs. alternatives: 'Use smartscraper for structured data extraction' and 'Use markdownify for clean, readable content.' It also details when to enable JavaScript rendering for SPAs vs. static sites, offering clear context and exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sitemapA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds valuable behavioral context beyond annotations. Annotations indicate read-only, idempotent, and non-destructive operations, but the description elaborates on cost ('1 credit per request'), discovery methods (e.g., checking robots.txt, crawling links), limitations (pages requiring authentication won't be discovered), and performance considerations (large sites may have thousands of URLs). No contradictions with annotations exist.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (Args, Returns, Raises, Use Cases, Best Practices, Note), but it is lengthy. While most sentences add value (e.g., explaining cost-effectiveness, discovery methods, limitations), some redundancy exists (e.g., repeating 'cost-effective' in multiple sections), slightly reducing efficiency.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (website mapping), the description is highly complete. It covers purpose, usage, parameters, return values (detailed in Returns section), error handling (Raises), practical applications (Use Cases), and operational notes. With annotations and an output schema present, the description provides all necessary contextual information without gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage for the single parameter 'website_url', the description fully compensates by providing extensive semantic details. It explains the parameter's purpose, format requirements (must include protocol), usage examples (e.g., root domain vs. subdomain), best practices, and discovery methods, adding significant value beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Extract and discover the complete sitemap structure of any website.' It specifies the verb ('extract and discover'), resource ('sitemap structure'), and scope ('any website'), and distinguishes it from siblings like 'smartcrawler_initiate' by focusing on comprehensive mapping rather than targeted crawling.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use this tool versus alternatives. It states: 'Useful for understanding site architecture before crawling or for discovering all available content,' and under 'Best Practices' advises: 'Run sitemap before using smartcrawler_initiate for better planning.' This clearly positions it as a preparatory tool for other scraping operations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• smartcrawler_fetch_resultsA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  While annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, the description adds valuable behavioral context beyond annotations: it explains the asynchronous nature of the operation, polling requirements, status tracking, and safe retrieval without side effects. The description doesn't contradict annotations and provides operational details not captured in structured fields.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with clear sections (purpose, usage instructions, Args, Returns) and front-loaded key information. Some repetition of 'Keep polling until status is 'completed'' could be reduced, but overall each sentence adds value. The description is appropriately sized for the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has annotations covering safety profile, an output schema exists (so return values don't need explanation), and the description provides comprehensive parameter semantics and usage guidelines, this description is complete. It covers the asynchronous nature, polling behavior, parameter meaning, and distinguishes from sibling tools - all essential context for proper tool invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage for the single parameter, the description fully compensates by providing comprehensive semantic information: it explains what request_id is ('unique request ID returned by smartcrawler_initiate'), its purpose ('to retrieve the crawling results'), usage guidance ('Keep polling until status is 'completed''), and provides an example ('req_abc123xyz'). This adds significant value beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('retrieve results'), resource ('asynchronous SmartCrawler operation'), and distinguishes it from sibling tools by explicitly mentioning it fetches results from a previously initiated operation using request_id. It differentiates from smartcrawler_initiate by focusing on result retrieval rather than initiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use this tool ('to retrieve the results from a previously initiated crawling operation using the request_id') and provides clear alternatives by naming the initiation tool ('smartcrawler_initiate'). It also provides specific usage instructions about polling behavior and when to stop ('Keep polling until status is 'completed'').

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• agentic_scrapperA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds significant behavioral context beyond annotations. While annotations indicate non-readOnly and non-idempotent, the description elaborates on credit costs, anti-automation measures, result variability, and specific capabilities like session persistence and AI extraction modes. It also details return structure and potential errors, providing comprehensive operational insight.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (Args, Returns, Raises, Use Cases, Best Practices, Note) and front-loads key information. However, it is lengthy due to comprehensive parameter details; some redundancy exists (e.g., repeating examples), but most content earns its place by adding value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (7 parameters, no schema descriptions, non-readOnly/non-idempotent annotations), the description is exceptionally complete. It covers purpose, usage, parameters, returns, errors, use cases, best practices, and limitations. The presence of an output schema reduces need to explain returns, and the description fills all other gaps thoroughly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description fully compensates by providing detailed semantics for all 7 parameters. Each parameter includes explanations, examples, tips, and default behaviors (e.g., ai_extraction default true, timeout_seconds default 120). This adds substantial meaning beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Execute complex multi-step web scraping workflows with AI-powered automation.' It specifies the verb ('execute'), resource ('web scraping workflows'), and distinguishes from siblings by emphasizing multi-step, interactive capabilities versus simpler extraction tools like smartscraper or markdownify.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly provides when to use this tool ('Ideal for complex scraping scenarios requiring user interaction simulation, form submissions, or multi-page navigation flows') and when not to use it ('Consider using simpler tools (smartscraper, markdownify) for basic extraction needs'). It also lists specific use cases and best practices for guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• searchscraperA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds substantial behavioral context beyond annotations: cost structure (10 credits per website), non-idempotence details ('results may vary over time'), processing time implications, accessibility constraints ('some websites may be inaccessible'), and specific error conditions. While annotations cover read-only and non-idempotent hints, the description provides operational details that help the agent make informed decisions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (Args, Returns, Raises, Note) and front-loaded key information. While comprehensive, some sections could be more concise (e.g., multiple similar examples). Every sentence adds value, but the overall length is substantial for a tool description.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (web search with AI extraction, cost structure, multiple parameters) and the presence of output schema, the description is exceptionally complete. It covers purpose, usage, parameters, returns, errors, cost implications, performance considerations, and sibling tool differentiation - providing everything an agent needs to use this tool effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description fully compensates by providing comprehensive parameter documentation: detailed explanations, examples, tips, ranges, defaults, and cost implications for all three parameters. The description adds significant meaning beyond the bare schema, including practical guidance for better results and trade-offs between parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool performs 'AI-powered web searches with structured data extraction' - a specific verb (search/extract) and resource (web). It distinguishes from siblings like 'smartscraper' (for specific URLs) and 'scrape' (generic scraping) by emphasizing AI-powered search and extraction from search results.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit guidance is provided: 'Ideal for research, competitive analysis, and gathering information from multiple sources' and 'Consider using smartscraper on specific URLs if you know the target sites.' The description clearly distinguishes when to use this tool (broad web searches) vs. alternatives (targeted scraping with smartscraper).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• smartcrawler_initiateA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds significant behavioral context beyond annotations: it explains the asynchronous nature, credit costs per mode (10 vs 2 credits/page), non-idempotent behavior, estimated processing time, and need for polling with fetch_results. Annotations only indicate it's not read-only/idempotent/destructive, so this provides crucial operational details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (overview, modes, args, returns, raises, notes) but is quite lengthy. While every sentence adds value, it could be more front-loaded; the core functionality is clear early, but parameter details are extensive. Still, no wasted text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (6 parameters, async operation, cost implications) and 0% schema coverage, the description is exceptionally complete. It covers purpose, usage, all parameters, return values, errors, costs, and next steps. The output schema exists but the description still usefully summarizes returns.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description fully compensates by providing detailed semantics for all 6 parameters: url requirements, prompt usage, extraction_mode options with costs, depth levels, max_pages ranges, and same_domain_only implications. Each parameter includes examples, defaults, and practical guidance.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool 'starts an intelligent crawler that discovers and processes multiple pages from a starting URL' and distinguishes it from siblings by specifying it's for 'asynchronous multi-page web crawling' with AI extraction or markdown conversion, unlike simpler scraping tools like 'scrape' or 'smartscraper'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use this tool vs alternatives: it specifies this is for 'asynchronous multi-page web crawling' and directs users to 'use smartcrawler_fetch_results to retrieve results'. It also contrasts modes (AI vs markdown) with cost implications and best-use cases.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.