Server Quality Checklist

Tool Scores

• firecrawl_crawl_statusC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, and the description only says 'check the status'. It doesn't describe whether this is a poll operation, if it might block, or what the response looks like. Minimal behavioral disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, no waste. But it could be slightly more informative without being verbose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple status check with one parameter and no output schema, the description is minimally complete. But it lacks information about the output format or behavior, leaving some gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with clear description for jobId. The tool description doesn't add new meaning beyond the schema, so baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool checks crawl job status, which differentiates it from sibling tools that start a crawl, map, or scrape. However, it doesn't explicitly contrast with siblings, so score is 4.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use or not use this tool, nor mention of alternatives. It's implied it's used after starting a crawl, but no explicit instructions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• firecrawl_mapC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description should disclose behavioral traits. It fails to mention whether the operation is read-only, requires authentication, or has rate limits. The phrase 'all accessible URLs' is vague.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence, concise and front-loaded. No wasted words, though it could benefit from slightly more detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 5 parameters, no output schema, and no annotations, the description is too brief. It lacks context on how parameters like 'ignoreSitemap' or 'includeSubdomains' fit into the operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline 3. The description adds no additional meaning beyond the schema parameter descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Map a website') and the outcome ('get a list of all accessible URLs'). It is specific enough to distinguish from sibling tools like firecrawl_crawl, though no explicit differentiation is provided.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives (e.g., firecrawl_crawl). No exclusions or prerequisites are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• firecrawl_scrapeC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must carry the full behavioral burden. It only states the basic action (scrape and return content) but omits details on error handling, rate limits, authentication, or what happens with complex pages. The minimal disclosure does not adequately inform an AI agent about potential behaviors.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with no wasted words. It is concise, but the brevity sacrifices some useful detail. Still, it is appropriately sized for a straightforward scrape operation.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the core function, but given no output schema and multiple parameters, more context would be helpful (e.g., typical use cases, behavior with dynamic content). It is minimally complete but could be improved.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%—all 5 parameters have descriptions. The tool description adds no additional meaning beyond the schema, so a baseline score of 3 is appropriate. It does not elaborate on parameter usage or relationships.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it scrapes a single webpage and returns markdown content. The verb 'scrape' and resource 'single webpage' are specific. However, it does not explicitly differentiate from sibling tools like firecrawl_crawl (multiple pages) or firecrawl_map, but the distinction is implied.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus its siblings (crawl, crawl_status, map). There is no mention of prerequisites, limitations, or context for selection, which is critical given the related tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• firecrawl_crawlB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden but only mentions basic purpose. It omits important traits like rate limits, robots.txt compliance, async behavior (given the existence of firecrawl_crawl_status), or failure handling.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence that immediately conveys the core action and result. No wasted words, though slightly more detail would be welcome without ruining conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a crawl tool with 7 parameters and no output schema, the description fails to mention return format, async behavior (hinted by sibling status tool), or any operational context. Incomplete for effective agent use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers all parameters (100% coverage), so description adds no extra meaning. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('crawl') and resource ('website starting from a URL') with a clear outcome ('return content from multiple pages'). This distinguishes it from siblings like firecrawl_scrape (single page) and firecrawl_map (probably URL listing) by emphasizing multi-page content extraction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool over siblings (e.g., firecrawl_scrape for single page, firecrawl_map for link mapping). Description only states what it does, not context for selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.