@crawlbrulee/mcp

Official

Overview Schema Source Related Servers Discussions Admin

Scrape a URL asynchronously

scrape_async

Submit a scrape job asynchronously, receive a job_id instantly for long-running tasks like JS rendering or full-page screenshots. Poll for status or get a webhook on completion.

Instructions

Submit a scrape job to run ASYNCHRONOUSLY and return a job_id immediately, instead of holding the connection open. Use this for long-running scrapes (heavy JS rendering, full-page screenshots of long pages) — for a quick one-shot fetch prefer the synchronous scrape tool, which blocks and returns the page directly. Poll the job with scrape_status and fetch the page with scrape_result once done. Optionally attach a per-job completion webhook: we deliver a single signed scrape.complete POST to your endpoint when the job finishes (HTTPS required in production), and echoes your opaque webhook.metadata back in the delivery (under data.metadata, alongside data.response_meta.usage).

Input Schema

TableJSON Schema

Name	Required	Description	Default
`url`	Yes	The URL to scrape. Known tracking parameters are removed before the page is fetched, so they are neither sent to the target site nor part of the cache key. Every other query parameter is kept verbatim and is part of the cache key.
`cache`	No	Cache settings for this request
`proxy`	No	Proxy tier to use for fetching	auto
`cleanup`	No	What is removed from the page before any output is built. Applies to markdown, cleaned_html, links and images on every engine, and to the screenshot. Never applies to raw_html, which is always the page before we removed anything.
`extract`	No	Which content formats to extract. Defaults to metadata + cleaned_html.
`webhook`	No	Completion webhook delivered when this async job finishes. Async-only: the synchronous `scrape` tool does not accept it.
`location`	No	Optional locale + country emulation for the scrape
`require_js`	No	Use a headless browser to render JavaScript before scraping

Output Schema

TableJSON Schema

Name	Required	Description	Default
`job_id`	Yes	Job identifier — pass it to the `scrape_status` / `scrape_result` tools

Schema Changelog

Changes observed during successful MCP inspections.

First observedv1.0.3Sep 25, 2026

TDQS

A4.4/5.0

Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden. It discloses async behavior, immediate return of job_id, webhook delivery with signature, HTTPS production requirement, and metadata echo. It doesn't cover failure modes or retries, but the core behavioral profile is well conveyed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two dense sentences front-load the async behavior WF and workflow guidance, then cover the webhook option. Every clause adds value; no repetition or filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For an 8-param async tool with no annotations but a rich schema and output schema, the description covers the essential decision factors (async vs sync, polling, webhook). It doesn't mention error handling or limits, but the existing coverage is strong.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline is 3. The description does not add much beyond the schema's rich parameter descriptions, except tying the webhook metadata echo to the job lifecycle. It relies on the schema for parameter-level detail, which is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's function: submitting a scrape job that runs asynchronously and returns a job_id immediately. It also distinguishes itself from the synchronous `scrape` tool, making its purpose unambiguous even among siblings.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It explicitly states when to use this tool (long-running scrapes) and when to prefer the synchronous alternative, and it outlines the full workflow: poll with scrape_status, fetch with scrape_result. No guesses needed.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Deploy Server