This server is a local Windows MCP stdio server for Crawl4ai that lets agents crawl web pages, extract content, and take screenshots.
crawl_markdown: Fetch a URL and return the page content as Markdown.
crawl_structured (documented in README): Extract repeated elements as JSON using CSS selectors and field specs.
crawl_screenshot: Capture a full-page screenshot and save it to a file, auto-creating parent directories.
Supports waiting options for dynamic content: wait for a fixed delay or for a CSS selector to appear.
Reuses a browser instance for the process and automatically recovers from crashes with limited retries.
Runs as a stdio MCP server, so agents like Claude Code or Hermes launch it on demand.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type @ followed by the MCP server name and your instructions, e.g., "@crawl4ai-local-for-windows Crawl https://example.com and summarize the main points"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
Local MCP server for Crawl4ai on Windows. Runs directly on Windows without WSL.
π€ Most of the code was generated with AI assistance.
See also:
Python 3.14 (installed automatically via uv)
Related MCP server: crawl4ai-mcp-server
uv syncπ¨ This server is a stdio server β no need to keep it running like a network server. When connected to an agent as MCP, the agent automatically spins up the server instance.
# Check for the "π Crawl4ai MCP server started." message, then exit
uv run mainβ
After confirming the welcome message, press Ctrl+C to exit, then add the MCP to your agent.
# Tip: debugging mode (MCP Inspector)
uv run mcp dev src/crawl4ai_local_for_windows/server.pyclaude mcp add --transport stdio --scope user crawl4ai -- uv run --directory <parent_location>/crawl4ai-local-for-windows mainparent_location: Write it like C:/dev/repo
e.g. claude mcp add --transport stdio --scope user crawl4ai -- uv run --directory C:/dev/repo/crawl4ai-local-for-windows main
# Check MCP installation
claude mcp list
claude mcp get crawl4aihermes mcp add crawl4ai --command "uv" --args "run" "--directory" "<parent_location>/crawl4ai-local-for-windows" "main"parent_location: Write it like C:/dev/repo
e.g. C:/dev/repo/crawl4ai-local-for-windows
# Check MCP installation
hermes mcp listcrawl_markdown(url, wait_seconds=0, wait_selector="")Crawl a URL and return the content as Markdown.
crawl_structured(url, selector, fields, wait_seconds=0, wait_selector="")Extract repeated elements as JSON using CSS selectors.
crawl_structured(
url="https://books.toscrape.com/",
selector="article.product_pod",
fields={"μ λͺ©": "h3@title", "κ°κ²©": ".price_color:text"},
)
# β [{"μ λͺ©": "A Light in the Attic", "κ°κ²©": "Β£51.77"}, ...]Field spec syntax:
Spec | Meaning |
| Text of the matched element |
| Attribute of a child element ( |
| Attribute of the base element itself |
| Nth element β use standard CSS ( |
Both tools share the waiting options (see below).
crawl_screenshot(url, output_path)Capture a full-page screenshot and save it to a file. Parent directories are created automatically.
π‘οΈ output_path is checked against a blocklist of known Windows system directories (C:\Windows, C:\Program Files, C:\Program Files (x86), C:\ProgramData, C:\System Volume Information, C:\$Recycle.Bin, C:\Users\All Users, C:\Users\Default) before the crawl even runs. A path that resolves into one of these (including via .. traversal) is rejected with an error message, so an agent can't accidentally overwrite system files. This is a blocklist, not a full sandbox β it guards against mistakes, not a determined attacker.
Instead of launching Chromium on every call (which is expensive), the server keeps one browser instance for the process lifetime. If the shared browser crashes mid-session, the server detects it and recovers automatically:
Crash detection matches Playwright collapse signatures only β Target page, context or browser has been closed, browser has crashed, browsertype.launch failures, etc. Network errors, anti-bot blocks, and timeouts are page-side causes and are never retried with a fresh browser.
On crash: dispose the dead instance β launch a new one β retry, up to _MAX_RECREATE_ATTEMPTS = 2 attempts.
Both failure paths are checked equally on every attempt: exceptions raised by arun() and result.success=False + crash message in error_message.
On the last attempt, a crash is still reported as a failure, but the crawler is not reset again β that cleanup already happened on the prior attempt, and resetting a second time could tear down an instance a concurrent request just recreated.
Preventive recycling (e.g., refresh every N pages) is intentionally left to Crawl4ai's built-in browser recycling; the server only reacts to actual collapses.
Concurrency note: multiple simultaneous arun() calls on the shared crawler are safe β Crawl4ai serializes page creation internally (_page_lock, GH-1198 fix) and manages context lifecycle with refcounting + LRU. Verified with concurrent multi-site smoke tests.
For dynamic pages that render content late using JS, use the waiting options of crawl_markdown / crawl_structured.
# Method 1: Wait for a fixed duration (seconds)
crawl_markdown(url="https://example.com", wait_seconds=5)
# Method 2: Wait until a CSS selector appears (takes precedence over wait_seconds when specified)
crawl_markdown(url="https://example.com", wait_selector="div.result-list")Comparison of 3 cases on a local test page that updates content via JS after 3 seconds:
Case | success | Captures Dynamic Content |
No waiting option | True | β Returns only "Loading..." |
| True | β Accurately captures final content |
| True | β Passes immediately if the element already exists in the initial HTML |
wait_selector is only valid for "newly created elements." If the text of an element already present in the initial HTML (e.g., <article id="dynamic">Loading...</article>) changes later, the selector matches immediately and passes without waiting. For pages that update text dynamically, use wait_seconds.
Extreme pages trigger anti-bot heuristics. Pages with minimal content and many script tags are flagged by Crawl4ai's anti-bot detector as Blocked by anti-bot protection: Structural: no_content_elements, script_heavy_shell, resulting in success=False. Although rare in production pages, testing should be done on pages containing basic static content (navigation bars, paragraphs, etc.).
μ£Όμ΄μ§ URLμ ν¬λ‘€λ§ν΄μ λ§ν¬λ€μ΄μΌλ‘ λ°ννλ€.
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes |
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must carry the full disclosure burden. It mentions crawling and markdown output but does not reveal network side effects, rate limits, following redirects, failure modes (unreachable URLs, non-HTML content), or whether it executes JavaScript. This leaves important behavioral traits undisclosed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single, concise sentence fully states the action, the object, and the return format. There is no wasted text or irrelevant detail.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple one-parameter tool this is minimally viable, but it lacks context about the expected markdown structure (full page vs filtered content), input URL restrictions, or output schema-level details beyond what the output schema likely provides. The existence of a sibling tool also suggests usage guidance would have made the description more complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has one `url` parameter with 0% description coverage. The description references 'given URL' but adds no extra meaning beyond the obvious fact that `url` is the input. Since the parameter is self-explanatory, this is adequate but adds little beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific action ('crawl') on a specific resource (the given URL) and a clear output format (markdown). This naturally distinguishes it from the sibling tool 'crawl_screenshot', so an agent can immediately tell the two apart.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no guidance about when to use this tool versus crawl_screenshot or any other alternative. The description only states what it does, not the conditions under which an agent should prefer it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
μ€ν¬λ¦°μ·μ μ°μ΄ νμΌλ‘ μ μ₯νλ€.
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | ||
| output_path | Yes |
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It only states that a screenshot is saved to a file; it does not disclose overwrite behavior, file format, URL rendering expectations, or any side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single clear sentence with no filler. It is appropriately sized for a simple tool and states the core action upfront.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has only two required parameters and an output schema, so complexity is low. Still, with no annotations and no sibling comparison, the description leaves the agent to infer important operational details such as URL validity and output handling.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description should clarify the parameters. It does not explicitly explain that 'url' is the page to screenshot or that 'output_path' is the destination file, though the phrase 'saves to a file' hints at the latter. This is minimal but not nonexistent.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool takes a screenshot and saves it to a file. It names a specific verb ('screenshot') and resource (file output), making it easy to distinguish from the sibling crawl_markdown, which presumably produces markdown instead.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is given about when to use this tool versus crawl_markdown, or what requirements exist (e.g., valid URL, writable path). Usage must be inferred entirely from the tool name and terse description.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
v0.1.0crawl_markdowncrawl_screenshotScored across 2 tools
The two tools are clearly distinct: one returns markdown content, the other saves a screenshot. There is no overlap in their purposes or outputs.
Both tools follow the same verb_noun pattern with the 'crawl_' prefix, making the naming entirely predictable and consistent.
With only two tools, the server is slightly under the typical 3-15 range, but the minimal set is reasonable for a focused crawler that only offers markdown and screenshot output.
The core crawling workflow is covered, but there are noticeable gaps such as raw HTML output, link extraction, or crawling options like depth or custom selectors. Agents may need to work around these missing capabilities.
Scrape, crawl and search the web for AI agents via MCP.
MCP server for web extraction and rendering via AceDataCloud WebExtrator
Zenrows MCP server β Fetch, Extract, Batch, and Browser Sessions for AI coding assistants
Docs: https://docs.keenable.ai/mcp-server Keenable is a free, remote MCP server that gives agents access to the web index. Search the web with ranked results and date/site filters, then fetch any indexed page as clean markdown. Works out of the box with no account or API key.
