Server Quality Checklist

Tool Scores

• get_usageB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that the tool returns usage statistics, which implies it's a read-only operation, but doesn't specify behavioral traits like authentication needs, rate limits, or whether it's real-time data. The description adds basic context but lacks details on how the tool behaves in practice, such as update frequency or error handling.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded, starting with the core purpose in the first sentence. The bullet points for returns are structured and add clarity without redundancy. However, the second part could be slightly more integrated into the flow, but overall, it's efficient with minimal waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (0 parameters, no output schema, no annotations), the description is somewhat complete but has gaps. It explains what the tool does and what it returns, which is adequate for a simple read operation. However, without annotations or output schema, it could benefit from more context on usage scenarios or limitations, making it just minimally viable.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, so no parameter documentation is needed. The description doesn't add parameter details beyond the schema, but since there are no parameters, it appropriately focuses on the output. This meets the baseline for tools with no parameters, as it doesn't mislead or omit necessary information.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get current API usage and token statistics.' It specifies the verb ('Get') and resource ('API usage and token statistics'), making the function immediately understandable. However, it doesn't explicitly differentiate from sibling tools like 'get_job_status' or 'batch_scrape,' which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention context, prerequisites, or exclusions, such as whether it's for monitoring usage limits or checking after API calls. With sibling tools like 'scrape_url' or 'get_job_status,' there's no indication of how this tool fits into the workflow, leaving usage unclear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• scrape_urlA
  4.1/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure and excels. It details advanced features like antibot bypass, JavaScript rendering, captcha solving, and proxy support, and includes critical cost information (token costs) that isn't in the schema. This provides rich context on performance, capabilities, and resource implications beyond basic scraping.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with sections (Features, Use cases, Token costs) and front-loaded with the core purpose. However, the token costs section is lengthy and could be more concise, and some details (like specific browser profiles) might be excessive. Overall, it's efficient but has minor verbosity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of the tool (8 parameters, advanced features) and no output schema, the description is largely complete: it covers purpose, usage, behavioral traits, and costs. The main gap is the lack of output details (e.g., what content is returned), but this is partially mitigated by the clear purpose. It's robust but not fully exhaustive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all 8 parameters thoroughly. The description doesn't add specific parameter semantics beyond what's in the schema (e.g., it mentions features like JS rendering but doesn't explain parameters like 'js_wait_for'). This meets the baseline of 3 since the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose as 'Scrape content from any web page with advanced antibot bypass,' which is a specific verb+resource combination. It distinguishes itself from siblings like 'batch_scrape' and 'scrape_async' by emphasizing advanced features, but doesn't explicitly contrast with them. The title is null, so the description carries the full burden of clarity effectively.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The 'Use cases' section provides clear context for when to use this tool (e.g., for JavaScript-rendered content, pages behind protections, or with captchas), which helps guide selection. However, it doesn't explicitly mention when NOT to use it or name alternatives like 'batch_scrape' for bulk operations or 'scrape_async' for asynchronous tasks, missing explicit sibling differentiation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• batch_scrapeA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It effectively discloses key behavioral traits: batch processing capability (up to 100 URLs), parallel execution for speed, and webhook notification upon completion. It also mentions return values (batch_id and job_ids), though it lacks details on error handling, rate limits, or authentication needs.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and front-loaded with the core purpose, followed by a bulleted list of benefits and return details. Every sentence adds value without redundancy, making it efficient and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description does a good job covering the tool's behavior and parameters. However, it lacks details on error cases, rate limits, or authentication requirements, which are important for a batch processing tool with potential complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds value by explaining the benefits related to parameters: 'Submit up to 100 URLs at once' clarifies the urls parameter limit, 'Parallel processing for speed' implies performance aspects, and 'Single webhook notification when all complete' relates to callback_url usage, enhancing understanding beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Scrape multiple URLs in a single batch request.' It specifies the verb ('scrape'), resource ('URLs'), and scope ('multiple'/'batch'), distinguishing it from sibling tools like scrape_url (single URL) and scrape_async (asynchronous single).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool: for batch processing of up to 100 URLs with parallel speed and single notification. However, it does not explicitly state when NOT to use it or name alternatives (e.g., use scrape_url for single URLs), though the sibling tool names imply differentiation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_job_statusA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden and does well by documenting the possible status values and their meanings (pending, processing, completed, failed, cancelled). It also specifies the polling behavior requirement. However, it doesn't mention rate limits, authentication needs, or error handling specifics that would be valuable for a polling tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly structured: a clear purpose statement, a well-organized status enumeration, and a specific usage instruction. Every sentence earns its place, with no redundant information. The status list uses bullet points for readability without being verbose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter tool with no output schema, the description provides excellent context about what the tool returns (status values with explanations) and how to use it (polling guidance). The only minor gap is lack of information about the return format/structure when status is 'completed' (though this is partially mitigated by referencing scrape_async as the job source).

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% with the job_id parameter fully documented as 'Job ID returned from scrape_async'. The description doesn't add any additional parameter information beyond what the schema provides, so it meets the baseline expectation when schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Get the status') and resource ('async scraping job'), distinguishing it from sibling tools like scrape_async (which creates jobs) and scrape_url (which performs immediate scraping). It explicitly identifies the tool's role in checking job progress rather than initiating or performing scraping.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage guidance: 'Poll this endpoint until status is 'completed' or 'failed''. This tells the agent when to use this tool (for monitoring async jobs) and when to stop (when job reaches terminal states). It also implicitly distinguishes from batch_scrape and scrape_url which are for different workflows.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• scrape_asyncA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes key behavioral traits: the async/long-running nature, the need to poll for results via 'get_job_status', and the return of a 'job_id'. It doesn't mention rate limits, authentication needs, or error handling, but covers the core operational behavior well.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and front-loaded with the core purpose, followed by clear usage guidelines and a note on the return value. Every sentence earns its place with no wasted words, making it highly efficient and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (async scraping with multiple parameters) and no annotations or output schema, the description does a good job of covering the essential context: purpose, usage scenarios, and result handling. It could benefit from mentioning error cases or prerequisites, but it's largely complete for agent use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all 7 parameters thoroughly. The description doesn't add any parameter-specific information beyond what's in the schema, but it doesn't need to since the schema is comprehensive. Baseline 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Submit an async scraping job for long-running requests.' It specifies the verb ('submit'), resource ('async scraping job'), and scope ('long-running requests'), and distinguishes it from immediate scraping alternatives like 'scrape_url'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage guidelines with a bulleted list: 'Use this for: - Pages that take > 60 seconds to load - Heavy JS rendering tasks - When you don't need immediate results - Batch processing workflows.' It clearly indicates when to use this tool versus alternatives like 'scrape_url' for immediate results.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.