Server Quality Checklist

Tool Scores

• crw_scrapeA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the agent knows this is a safe, non-destructive operation. The description adds minimal behavioral context beyond the 'one URL' scope, but does not disclose any additional behavior such as JS rendering defaults, pagination, or output limits. It does not contradict the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single 9-word sentence that is direct and front-loaded. Every word contributes to the core purpose. It is extremely concise without being verbose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (9 parameters, 4 output formats, multiple renderer options), the description is quite thin. The rich schema compensates for parameter details, but the description fails to mention the 'images' format and provides no context about renderer behavior or when to use optional parameters. For a simple scraper, it may be sufficient, but for an AI agent selecting among multiple similar tools, more context would be helpful.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema covers 100% of parameters with descriptions, so the baseline is 3. The description mentions markdown, HTML, and links, but omits 'images' from the formats enum, which is a minor gap. The description does not add meaning beyond the schema for the remaining parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (scrape), the resource (one URL), and the output formats (markdown, HTML, or links). The phrase 'one URL' distinguishes it from siblings like crw_crawl, which implies multi-page crawling. It is specific and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies this tool is for scraping a single URL, since it says 'one URL'. However, it does not explicitly mention when to prefer this over crw_crawl, crw_map, or crw_extract, nor does it list any exclusions or prerequisites. Usage is implied rather than stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• crw_mapA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already provide readOnly, idempotent, and non-destructive hints. The description adds that output is URL list only, consistent with annotations, but no extra behavioral details beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise, front-loaded sentences with no wasted words; every part adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity and rich schema+annotations, the description covers the main action and output. Minor gap: interaction of sitemap and crawl fallback is explained in schema, so description is complete enough.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 100% parameter coverage with clear descriptions (e.g., limit 0 = unbounded, defaults). The description adds no further parameter meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool discovers URLs on a site via sitemap/crawl and explicitly says it returns only URLs, no content, distinguishing it from sibling crw_scrape and implying it's different from crw_crawl.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for URL discovery without content, but does not explicitly contrast with siblings like crw_crawl or provide when-to-use/not-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• crw_cancel_extractA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Despite annotations already indicating destructive and non-read-only behavior, the description adds valuable context: cancellation is non-terminal until the claimed URL settles, and it returns canonical status. This goes beyond the structured annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two tightly worded sentences. The first states the action; the second adds a critical caveat. No redundant information; front-loaded with purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a cancellation tool, the description covers purpose and the important async behavior. An output schema exists, so return values need not be described. Slightly more guidance on subsequent steps (e.g., checking status) could help, but it is not essential.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with the single 'id' parameter documented as 'Extract job id from crw_extract.' The description text itself adds no parameter details, so it relies entirely on the schema, which is adequate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states the action: 'Request cancellation of an extract job.' The verb 'cancel' and resource 'extract job' are specific, and it is distinct from sibling tools like crw_extract (create) and crw_check_extract_status (status check).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use when you want to cancel an extract job, but it does not explicitly contrast with alternatives or mention when not to use it. No sibling tool is referenced, so guidance is implied rather than explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• crw_check_crawl_statusA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint and idempotentHint, so the description adds only moderate behavioral context by confirming it polls and retrieves pages. The description does not disclose any additional traits beyond the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that is front-loaded and contains no unnecessary words. It efficiently conveys the tool's purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple polling tool with two parameters and no output schema, the description adequately explains the action (poll and retrieve pages). It could hint at the return format, but 'retrieve its pages' is sufficient for an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, with clear parameter descriptions in the schema. The tool description adds no additional meaning beyond what is already in the schema, meeting the baseline for high coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool polls an async crawl job and retrieves its pages, providing a specific verb and resource. It distinguishes from siblings like crw_crawl (start) and crw_scrape (synchronous scrape).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context by mentioning 'poll an async crawl job', indicating it's for checking ongoing crawls. However, it does not explicitly state when not to use or name alternatives, which is a minor gap.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• crw_crawlA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations declare readOnlyHint=false, destructiveHint=false, idempotentHint=false, openWorldHint=true. The description adds the important behavioral trait that the crawl is async and returns a job id for polling, which informs the agent about the non-immediate nature and required follow-up. This goes beyond what annotations alone provide.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no superfluous text. The first sentence states the action and output, the second ties it to the sibling tool for follow-up. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having 7 parameters and no output schema, the tool description covers the key context: async nature, job id return, and polling via a specific sibling. It does not explain the crawling scope or error handling, but the input schema fills in parameter details. For a start-crawl tool, this is fairly complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% (all 7 parameters have descriptions). The tool description does not add any additional parameter semantics beyond the schema, so baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Start') and resource ('async site crawl'), clearly states the output (job id), and distinguishes from the sibling tool crw_check_crawl_status by indicating the polling workflow.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for usage: start a crawl then poll with the sibling tool. It does not explicitly list when to avoid using it or compare with other siblings like crw_map or crw_scrape, but the implication is strong enough for a straightforward tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• crw_extractA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate readOnlyHint=false and openWorldHint=true. The description adds behavioral nuance by disclosing the async job nature ('Async job — poll crw_check_extract_status with the returned id') and a key prerequisite ('Needs an LLM'). This provides context beyond the annotations, such as the non-blocking execution model and the requirement to track progress via a returned identifier.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is exactly two sentences: the first states the core purpose and method, the second covers the async workflow and prerequisite. Every clause adds value, and the most important information is front-loaded. There is no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 7 parameters, nested objects, and an output schema, the description efficiently covers the essential workflow (async, polling, LLM requirement). The existence of an output schema means return values are documented elsewhere. It does not discuss error handling or rate limits, but these are less critical given the asynchronous pattern and available schema documentation. Overall, it is complete enough for an agent to select and invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema covers 71% of parameters with descriptions, so the baseline is 3. The description adds meaningful semantics by explaining that extraction works 'via a prompt and/or JSON schema', clarifying the relationship between prompt and schema parameters. It also highlights the LLM dependency, tying together llmModel/llmProvider/llmApiKey even though the schema doesn't explicitly state they are required. This goes beyond the schema's bare parameter list.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description begins with 'Extract structured JSON from URLs via a prompt and/or JSON schema', which clearly states the verb ('extract'), the resource ('URLs'), and the output format ('structured JSON'). It also distinguishes from sibling tools like crw_scrape (raw scraping) and crw_map by emphasizing structured extraction. The async note differentiates it from synchronous tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides context that it is an async job and must be polled via crw_check_extract_status, and that it needs an LLM. However, it does not explicitly state when to use this tool over alternatives such as crw_scrape or crw_crawl, nor any exclusions. The usage context is clear but lacks explicit when-to-use/when-not-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• crw_parse_fileA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The annotations already declare readOnlyHint=true, idempotentHint=true, destructiveHint=false, so the safety profile is clear. The description adds the behavioral trait that no OCR is performed and scanned PDFs return empty markdown with a warning, which is valuable context beyond annotations. It also clarifies that the input is base64-encoded local PDF bytes, but that's already in schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one sentence, front-loaded with the main action ('Parse a local PDF...to markdown') and then a brief caveat. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 6 parameters and no output schema, the description covers the core functionality and a key limitation. However, it doesn't describe the response structure or that other output formats (json, summary) require a server LLM, though that's in the schema. The description is adequate for a simple parse tool but leaves some gap regarding output shape.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema provides descriptions for all 6 parameters (100% coverage), so the description doesn't need to add much. The description does reference contentBase64 and the markdown output, implicitly mapping to formats, but it doesn't explain the formats, jsonSchema, or maxLength parameters – though those are well-documented in the schema. Thus, the description adds minimal additional parameter semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool parses a local PDF (base64 in contentBase64) to markdown, which is a specific verb with resource and output format. It distinguishes itself from sibling tools by specifying 'local PDF' rather than URLs, aligning with crw_scrape/crw_crawl. The 'No OCR' caveat further defines its scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use it: when you have a local PDF as base64 and want markdown. It explicitly states a when-not scenario: scanned PDFs return empty markdown with a warning, which tells the agent to avoid using it for those. However, it doesn't name alternative tools for OCR or other formats, so it falls just short of explicit alternatives guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• crw_check_extract_statusA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The annotations already indicate a safe, idempotent read operation. The description adds the behavioral detail that the tool returns status immediately and, upon completion, includes a per-URL results array, which is useful for understanding polling behavior. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that directly states the action and outcome with no extraneous text. It earns a top score.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that an output schema exists and the tool has a single parameter, the description adequately covers the tool's purpose, return behavior, and usage context. It is complete for the tool's complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already defines 'id' as the extract job id from crw_extract, and the description does not add any additional parameter semantics. With 100% schema coverage, the baseline of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'Poll' and identifies the resource as 'an extract job,' clearly distinguishing it from the similar sibling 'crw_check_crawl_status' by specifying 'extract' and the per-URL results array. It states exactly what the tool does and its output.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It clearly implies usage when checking the status of a previously submitted extract job, providing clear context. However, it does not explicitly mention alternatives or when not to use this tool, such as for crawl jobs, so it lacks explicit exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.