Run the native Crawl4AI MCP server

This repository maintains a pinned, locally verified distribution of Crawl4AI 0.9.1. It runs the upstream container and its native Model Context Protocol (MCP) server. It is not a custom MCP implementation.

Start the service

The local distribution requires Docker with Compose and an API token. Generate a token for the current shell, then start the pinned image:

export CRAWL4AI_API_TOKEN="$(openssl rand -hex 32)"
docker compose up -d --wait

The service listens on http://127.0.0.1:11235. Set CRAWL4AI_PORT before startup to choose another loopback port.

Related MCP server: MCP Deep Web Research Server

Verify the distribution

The smoke check starts an isolated stack, checks public health, verifies that unauthenticated MCP access fails, verifies authenticated schema access, and removes the stack. Run it while the regular stack is stopped:

export CRAWL4AI_API_TOKEN="$(openssl rand -hex 32)"
./scripts/smoke.sh

Use these requests to verify a regular running stack:

base_url="http://127.0.0.1:${CRAWL4AI_PORT:-11235}"
curl --fail --silent --show-error "${base_url}/health"
curl --fail --silent --show-error \
  --header "Authorization: Bearer ${CRAWL4AI_API_TOKEN}" \
  "${base_url}/mcp/schema"

Run the static Compose validation without starting the service:

./scripts/validate.sh

Connect an MCP client

Crawl4AI exposes these upstream-native endpoints:

• Server-Sent Events (SSE): http://127.0.0.1:11235/mcp/sse

• WebSocket: ws://127.0.0.1:11235/mcp/ws

• Tool schema: http://127.0.0.1:11235/mcp/schema

Send Authorization: Bearer ${CRAWL4AI_API_TOKEN} on every request except GET /health. The Crawl4AI 0.9 migration guide defines this authentication contract, and the upstream self-hosting guide documents the MCP endpoints and tools.

Crawl4AI 0.9.1 supports legacy HTTP with SSE and a custom WebSocket transport. It does not expose the single-endpoint Streamable HTTP transport defined by the current MCP transport specification. Use a client with legacy SSE or WebSocket support.

For example, add the local SSE endpoint to Claude Code with the token kept in the current shell:

claude mcp add --transport sse \
  --header "Authorization: Bearer ${CRAWL4AI_API_TOKEN}" \
  crawl4ai http://127.0.0.1:11235/mcp/sse

Claude Desktop limitation

This loopback-only distribution does not provide a Claude Desktop custom connector. Anthropic's custom connectors run from Anthropic's cloud, so they cannot reach 127.0.0.1; this repository also does not ship a local stdio bridge or desktop extension. Use Claude Code or another local client that supports authenticated legacy SSE. A separately reviewed remote boundary is tracked in issue #25.

Stop the service

Stop and remove the regular Compose stack:

docker compose down

Supported maintenance scope

This repository keeps a known upstream release reproducible through a pinned image, loopback-only Compose defaults, dependency update automation, and executable validation. It does not fork Crawl4AI, add custom tools, or maintain the retired TypeScript and Cloudflare Workers product.

Trusted remote authentication and deployment remain deferred to issue #25. They require an owned target environment, credential policy, TLS termination, and deployment-specific validation before implementation.

Update the pinned version

Use this procedure for each upstream release:

1. Read the upstream release notes and Docker migration guidance.

2. Update the complete unclecode/crawl4ai:<tag>@sha256:<digest> reference in compose.yaml, preferably through Dependabot.

3. Run ./scripts/validate.sh.

4. Export a fresh CRAWL4AI_API_TOKEN and run ./scripts/smoke.sh.

5. Update the version references in this README.

Do not move the pin until the image pull and both checks pass.

Ownership and license

Crawl4AI, its container image, and its MCP implementation belong to the upstream project. Report upstream runtime or tool behavior there. This repository owns only its distribution configuration and validation.

Repository-maintained files use the MIT License. The packaged upstream project retains its Apache 2.0 license and attribution terms.