# Crawl4AI Complete MCP

Floating latest-stable Docker deployment of Crawl4AI with the official MCP Python SDK stdio and Streamable HTTP servers. Builds resolve exact releases and source commits into generated metadata; manual build overrides are available only for rollback/debugging. The bridge does not expose arbitrary Python, shell, or HTTP proxying.

## Status

The current verified build resolves Crawl4AI **0.9.2** (`7e801521428ee12509994d39151006f64055ebe3`) and MCP Python SDK **2.0.0** (`6f69a3758ebf2ee55ce050f58b470ce11af71133`). These are build evidence, not permanent pins. The inspected source remains the implementation authority for each build. The upstream `deploy/docker` tree was inspected before implementation; its HTTP server is a separate FastAPI process with Redis-backed jobs and monitoring. This project provides a stdio-first runtime and a safe, typed subset/bridge of the same local Python capability surface, with unsupported operations explicitly reported rather than silently discarded.

## Quick start

```bash
docker compose build crawl4ai-mcp-stdio
# equivalent profile-aware form:
docker compose --profile stdio build crawl4ai-mcp-stdio
docker compose --profile stdio run --rm crawl4ai-mcp-stdio
```

For the separated native/Redis/HTTP topology, choose an unused host port instead of assuming `8000`:

```bash
CRAWL4AI_MCP_HTTP_PORT=37600 docker compose --profile redis --profile native-api --profile stdio --profile http-mcp up -d
```

The container speaks MCP JSON-RPC over stdin/stdout. Logs go to stderr. Configure `CRAWL4AI_API_TOKEN` only if your MCP host requires an application token; it is not printed or returned.

## Security defaults

- only `http` and `https` URLs;
- private, loopback, link-local, and cloud-metadata destinations blocked by default;
- optional domain allow-list;
- bounded URLs, scripts, output, timeout, concurrency, and crawl count;
- no arbitrary Python, shell, filesystem paths, Docker socket, or CAPTCHA/access-control bypass;
- declarative hooks only, disabled by default unless `CRAWL4AI_HOOKS_ENABLED=true`;
- proxy credentials must be supplied through environment-backed references, never MCP arguments;
- webhook delivery is not enabled in stdio mode (use the native HTTP job service for operator-managed webhooks).

See `docs/security.md`, `docs/configuration.md`, and `docs/deployment.md`.

## MCP

The complete tool inventory and parity claims are in `docs/mcp-tools.md` and `docs/parity-matrix.md`. The bridge exposes individual tools for crawling, streaming, rendering, JavaScript, extraction, configuration and capability inventory. It is not an arbitrary HTTP proxy: operations are explicit and documented. It does not claim that Redis jobs, webhooks, native monitoring, or the upstream artefact store are present in a single stdio process; those are documented as separate authenticated HTTP deployment operations.

## Native HTTP server

The pinned upstream release includes `deploy/docker/server.py`, a separate FastAPI service with `/health`, `/schema`, `/crawl`, `/crawl/stream`, `/md`, `/html`, `/screenshot`, `/pdf`, `/execute_js`, `/hooks/info`, job routes, monitoring routes, artefacts, and MCP SSE/WebSocket routes. It is documented in `docs/upstream-api.md`. This image deliberately does not start a hidden background HTTP process alongside stdio; run the upstream HTTP deployment separately when REST, SSE, Redis jobs, or Prometheus monitoring are required.

## Development

```bash
uv sync --locked
uv run pytest -q
uv run python -m crawl4ai_mcp.server < /dev/null
```

Docker integration requires Docker and a working Chromium sandbox configuration. See `docs/testing.md`.

## Licence

Apache-2.0. Crawl4AI is used under its upstream Apache-2.0 licence.