Server Quality Checklist

Tool Scores

• crawl_statusA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  There are no annotations, so the description carries the disclosure burden. 'So far' does convey that results may be incremental and that polling may need to be repeated, and it lists the returned progress data. However, it does not describe error cases, status value vocabulary, or how the output evolves between polls.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The entire description is one dense, front-loaded sentence. It names the action, the resource, and the expected output fields with no filler or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter read-only poll with high schema coverage and no output schema, the description covers purpose and the main return fields adequately. It falls slightly short by not giving exact output field names for the pages fetched or the possible status values, but it is still sufficient for an agent to call the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and jobId is already documented as 'The id returned by crawl', which orients the agent well. The description adds no additional parameter meaning beyond the schema, so the baseline of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Poll a crawl job', and it names the key monitored fields (status, pagesCrawled, pagesQueued, pages fetched). It is clear, but it does not explicitly differentiate from sibling tools such as collector_run_status.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The verb 'Poll' implies the tool is used after starting a crawl and to monitor progress, but the description never explicitly states when to use it instead of alternatives, nor does it mention any exclusions or related tools like crawl or collector_run_status.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• searchA
  3.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations present, the description carries the burden of explaining behavior, and it does a solid job: it discloses the response shape, the engines covered, the verticals, and the relevant pricing. It does not mention rate limits, caching, or failure modes, which is a gap, but there is no contradiction and the core operational behavior is clearly stated.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences deliver a large amount of information without filler. The first sentence front-loads the tool's purpose and response contents; the second adds the full vertical count and per-search cost. Every clause earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the full parameter schema and the absence of an output schema, the description covers the most important return values and the wide range of verticals well. The main missing element is selection guidance relative to sibling tools, but the information needed to actually invoke the tool is either in the schema or in the description.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already covers 100% of the parameters with helpful descriptions, so the baseline is met. The description only adds the high-level note that there are eighteen verticals and that search spans Google, Bing, DuckDuckGo, or Yandex, which is already visible in the schema. It adds no deeper parameter-specific detail.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly defines a search-engine resource and the exact output format: organic rows with position, title, link, snippet, and sitelinks, plus related searches and local/news/shopping blocks. It is distinct in substance from scraping or crawling, but it does not explicitly name sibling tools or show selection boundaries, so it stops short of perfect.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance about when to prefer this tool over scrape, map, crawl, or the other siblings. The description says it offers web search results and verticals, but it does not specify alternatives, exclusions, or conditions that would drive an agent toward another tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_collectorsA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Since there are no annotations, the description carries the full burden. It discloses that the tool is free ('Free to call'), and it details what each entry contains (input/output schema, examples, health, price). It does not mention rate limits, caching, or that it's a read-only operation, but for a listing tool these are minor. The description goes beyond a simple 'list' by explaining the structure of the response.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is somewhat long because it enumerates all 31 Collectors, but that list is arguably useful for an agent to know the available options upfront. The first sentence states the purpose, followed by a definition and the list. Each sentence adds value. It's not overly verbose given the need to identify the Collectors.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a parameterless listing tool with no output schema, the description is complete. It tells the agent what the response will contain (schema, examples, health, price) and that it's free. There are no missing pieces an agent needs to know before calling it, given its simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool takes no parameters, and the input schema is empty. The description doesn't need to explain parameter meaning. The baseline for zero parameters is 4, and the description adds context about what the tool returns, which is beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the exact purpose: listing 31 ready-made Collectors. It clearly differentiates them from the sibling tools (run_collector, collector_run_status) by describing what a Collector is and that this tool only lists them, not runs them. The verb 'List' and the resource 'Collectors' are specific and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  While it's evident that this tool is meant to be called before running a Collector (given the siblings run_collector and collector_run_status), the description does not explicitly state 'use this to see available Collectors before running one' or contrast it with alternatives. It only implies usage by listing capabilities. No explicit guidance on when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• seo_auditA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It describes the dual fetch behavior, the returned diff types, the meta extraction, and even the cost per call. It does not explicitly state whether the operation is read-only, but the language (fetch, return) implies a safe audit. The level of detail about internal mechanics is strong for a tool of this type.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact—two sentences—and every element earns its place: the action, the outputs, example diff types, the use case, and the cost. It front-loads the core behavior and then answers 'why use this' with a clear value proposition. There is no filler or redundant phrasing.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given there is no output schema, the description does a good job of indicating what will be returned (both views, diff items, meta types). It names specific diff categories, making the output shape imaginable. However, it does not detail the structure of the returned views/meta or explain each diff type, which leaves some ambiguity for an agent attempting precise invocation. Still, for a tool with three simple parameters and a clear output sketch, it is largely sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so all three parameters (url, country, no_render) already have descriptions. The tool description adds minimal parameter-level detail beyond the schema—it mentions the rendered vs no-JS contrast, which implicitly relates to no_render, but does not elaborate on country or additional constraints. Per the rubric, baseline 3 is appropriate given high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb (fetch), the resource (URL twice, bot vs rendered), and the resulting outputs (views, diff, meta). It also names concrete diff types, making the tool's purpose unmistakable. This clearly distinguishes it from siblings like scrape or crawl, which focus on general data extraction rather than SEO indexability analysis.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly frames the tool as 'The fastest way to answer is this page indexable without JavaScript', giving a clear target scenario. However, it does not explicitly state when not to use this tool or mention alternative siblings, so it stops short of full exclusionary guidance. The context is clear enough for an agent to select it over other tools in many situations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• crawlA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so the description carries the burden. It discloses async behavior, immediate job id return, per-page cost, budget refund, and output format (Markdown). Does not mention potential side effects like rate limits or data retention, but covers the key operational traits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Concise single paragraph, front-loaded with action and resource, then key details on costs and alternatives. No fluff, but could be slightly more structured (e.g., separating usage guidance).

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Description covers the async nature, job id, cost, and alternative tools. It does not explain parameter semantics, but schema does. No output schema provided; description mentions 'poll with crawl_status' so return behavior is clear. For a complex tool, it's reasonably complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema already provides full descriptions for all 8 parameters (100% coverage). The description does not add parameter-specific meaning; it only clarifies the overall outcomeщаться, which is adequate given high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states it starts an asynchronous BFS crawl from a seed URL, returns Markdown, and explicitly contrasts with map/batch for filtered URL lists, distinguishing it from siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly advises 'Prefer map + batch when you can filter the URL list first,' and notes the async nature with polling via crawl_status. Clear when-to-use and alternative guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• mapA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral burden and discloses the data sources, deduplication, output contents (list, total, per-section summary), and pricing. It does not mention error behavior or rate limits, but for a read-only discovery call these are minor gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Four tight sentences front-load the core behavior, then return shape, cost comparison, and price. No filler or redundant restatement of the schema.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having no output schema or annotations, the description covers sources, deduplication behavior, return contents, pricing, and the main use case, while the schema covers params. This is enough for an agent to select and invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so all six parameters are already documented in the input schema. The description adds output context but no new parameter-level semantics, which meets the high-coverage baseline of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The opening 'Discover every URL of a site in one call' names the action and resource, and the following colon specifies the exact discovery sources (robots.txt sitemaps, /sitemap.xml, nested indexes, same-domain homepage links). This clearly distinguishes map from siblings like crawl and search.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'Always cheaper than crawling to find out how big a site is' gives a concrete use case and names crawl as the alternative. It does not explicitly address when to prefer search or seo_audit, but the context makes the primary selection criterion clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• run_collectorA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden. It discloses billing per delivered row (zero rows cost zero), and the sync/async behavior with run_id. It doesn't mention error handling or rate limits, but the key behaviors are covered. No contradiction with annotations since none exist.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, each earns its place: the core action, the sync/async distinction, and the billing model. Front-loaded with the key action and then important behavior. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema and 100% param schema coverage, the description covers the essential behaviors: sync vs async, billing, and how to poll. It could mention error conditions or rate limits, but for a run tool it's reasonably complete. The sibling list_collectors is hinted for slug discovery.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the schema documents the parameters well. The description adds meaning for the 'async' parameter (force background processing), and explains the input is semantic and matches the collector's published input_schema. It provides value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states clearly it runs a Collector with its semantic input, and distinguishes the sync vs async behavior. It also names the sibling collector_run_status for polling, which helps differentiate from other tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives context on when to use synchronous vs asynchronous (short vs long runs), and implicitly suggests using list_collectors first if unsure about the slug. It doesn't explicitly say when not to use this tool or mention alternatives beyond the polling tool, so a slight gap.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• scrapeA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses that fetches go through residential proxies with real-browser TLS fingerprints (implying anti-blocking), the cost per page ($0.0002), and that failed fetches are free. This goes beyond obvious behavior, though it doesn't mention rate limits, auth, or detailed error handling. Still, it provides meaningful operational context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences with zero wasted words. The main purpose is front-loaded; then usage guidance, alternatives, and cost are presented logically. The pricing note is a useful addition without bloat.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite 9 parameters and no output schema, the description covers the critical decision points (render, extract, ai_prompt, cost) enough for an agent to call correctly. Some parameters (country, contentMode, waitForSelector) are not elaborated, but they are well-described in the schema with 100% coverage. The description adds sufficient contextual guidance for a complex tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds value by explaining when to set render (only if absent from raw HTML), and how extract and ai_prompt are used for structured data. It also clarifies that summary mode returns metadata only via schema description. This enriches the schema's parameter meanings.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the tool fetches a single URL and returns it as Markdown, HTML, or plain text. This is a specific verb+resource, and it implicitly distinguishes from siblings like crawl (which implies multiple pages) by saying 'one URL'. The mention of extract and ai_prompt for structured data further clarifies the tool's scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides concrete guidance for when to use render (only if content absent from raw HTML) and when to use extract/ai_prompt for structured data. However, it does not explicitly contrast with sibling tools (e.g., crawl for multi-page) or state when not to use this tool. The guidance is clear for parameter usage but not tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• collector_run_statusA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It explicitly states the operation is a 'fetch', implying a read-only, non-destructive behavior. It does not disclose response format or edge cases, but the simple nature of the operation and the clear 'fetch' wording provide sufficient transparency for an agent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with zero filler. It immediately states what the tool does and how, leaving no wasted words. It earns full marks for conciseness and clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, no output schema) and the absence of annotations, the description provides all essential information: what it fetches, what it returns (run and delivered rows), and how to identify the run. Nothing critical is missing for a tool of this scope.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so runId is already documented. The description adds value by specifying that the run_id is the one returned by run_collector, establishing the provenance and correct value source. This extra context meaningfully aids correct invocation beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action (fetch), the resource (one Collector run and its delivered rows), and the method (by run_id). This distinguishes it from siblings like crawl_status and run_collector, making its purpose unambiguous without needing to open the schema.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context through the parameter note 'returned by run_collector', signaling this is meant to be called after run_collector to retrieve results. It does not explicitly list alternatives or when-not-to-use scenarios, but for a simple status-like fetch the implicit linkage is adequate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.