<div align="center">

<img src="docs/banner.jpg" alt="Firecrawl Lite MCP Server" width="100%" />

# Firecrawl Lite MCP Server

**Web scraping for AI agents, minus the infrastructure. One process. Your machine. No scraping cloud in the loop.**

[![npm version](https://img.shields.io/npm/v/firecrawl-lite-mcp-server?logo=npm&color=cb3837)](https://www.npmjs.com/package/firecrawl-lite-mcp-server)
[![Docker Pulls](https://img.shields.io/docker/pulls/ariangibson/firecrawl-lite-mcp-server?logo=docker&logoColor=white)](https://hub.docker.com/r/ariangibson/firecrawl-lite-mcp-server)
[![Image Size](https://img.shields.io/docker/image-size/ariangibson/firecrawl-lite-mcp-server/latest?logo=docker&logoColor=white&label=image%20size)](https://hub.docker.com/r/ariangibson/firecrawl-lite-mcp-server)
[![Build](https://img.shields.io/github/actions/workflow/status/ariangibson/firecrawl-lite-mcp-server/docker-build.yml?branch=main&logo=github&label=build)](https://github.com/ariangibson/firecrawl-lite-mcp-server/actions/workflows/docker-build.yml)
[![Tests](https://img.shields.io/github/actions/workflow/status/ariangibson/firecrawl-lite-mcp-server/test.yml?branch=main&logo=github&label=tests)](https://github.com/ariangibson/firecrawl-lite-mcp-server/actions/workflows/test.yml)
[![Node](https://img.shields.io/node/v/firecrawl-lite-mcp-server?logo=node.js&logoColor=white)](https://nodejs.org)
[![MCP](https://img.shields.io/badge/MCP-compatible-6E56CF)](https://modelcontextprotocol.io)
[![License: MIT](https://img.shields.io/badge/license-MIT-yellow.svg)](LICENSE)

</div>

---

Your agent needs to read web pages. Firecrawl is great at that, but self-hosting it means Redis, a Playwright service, API workers, and a weekend. Firecrawl Lite is the other option: a single Node.js process with headless Chrome (stealth-patched) that turns URLs into clean Markdown, and speaks the two protocols agents actually use:

- **MCP** — for Claude Desktop, Claude Code, Cursor, and anything else that talks [Model Context Protocol](https://modelcontextprotocol.io).
- **Firecrawl-compatible REST API** — for any agent or harness that already speaks the Firecrawl SDK. Point `FIRECRAWL_API_URL` at it and it quietly impersonates a self-hosted Firecrawl (scrape only — no crawl or search).

Nothing leaves your box except the page requests themselves. No Firecrawl account. No API keys required at all unless you want the optional LLM-powered extraction tools — and for those, you bring your own model.

## 60-second start

**As an MCP server** (Claude Code shown; Claude Desktop and Cursor configs are [below](#mcp-clients)):

```bash
claude mcp add firecrawl-lite npx -- -y firecrawl-lite-mcp-server
```

**As a Firecrawl-compatible API** (for agents, scripts, anything HTTP):

```bash
docker run -d -p 3000:3000 ariangibson/firecrawl-lite-mcp-server:latest
curl -X POST localhost:3000/v2/scrape -H 'Content-Type: application/json' -d '{"url":"https://example.com"}'
```

That's a working scraper. Everything below is optional.

## What you get

| Tool | Does | Needs an LLM? |
| --- | --- | --- |
| `scrape_page` | URL → clean Markdown. Renders JS, waits for the DOM to settle, strips the junk. | No |
| `batch_scrape` | Same, for up to 10 URLs, with polite delays between them. | No |
| `screenshot` | URL → PNG (base64). Viewport or full page. | No |
| `extract_data` | "Get me the price and the release date" → JSON, via your LLM. | Yes |
| `extract_with_schema` | Same, but you hand it a JSON Schema and get exactly that shape back. | Yes |

The Firecrawl-compatible API exposes `scrape_page` as `POST /v2/scrape`. See [the API section](#firecrawl-compatible-api) for the exact contract.

## Hook it up

### MCP clients

<details>
<summary><strong>Claude Desktop</strong> — <code>claude_desktop_config.json</code></summary>

The `env` block is only needed for the `extract_*` tools; leave it out otherwise.

macOS: `~/Library/Application Support/Claude/claude_desktop_config.json` · Windows: `%APPDATA%\Claude\claude_desktop_config.json`

```json
{
  "mcpServers": {
    "firecrawl-lite": {
      "command": "npx",
      "args": ["-y", "firecrawl-lite-mcp-server"],
      "env": {
        "LLM_API_KEY": "sk-...",
        "LLM_PROVIDER_BASE_URL": "https://api.openai.com/v1",
        "LLM_MODEL": "gpt-5.5"
      }
    }
  }
}
```
</details>

<details>
<summary><strong>Claude Code</strong> — one command</summary>

```bash
claude mcp add firecrawl-lite \
  --env LLM_API_KEY=your_key \
  --env LLM_PROVIDER_BASE_URL=https://api.openai.com/v1 \
  --env LLM_MODEL=gpt-5.5 \
  -- npx -y firecrawl-lite-mcp-server
```

Drop the `--env` lines if you don't need the LLM tools. For a remote instance: `claude mcp add -t http firecrawl-lite http://your-server:3000/mcp`.
</details>

<details>
<summary><strong>Cursor</strong> — <code>~/.cursor/mcp.json</code></summary>

The `env` block is only needed for the `extract_*` tools; leave it out otherwise.

```json
{
  "mcpServers": {
    "firecrawl-lite": {
      "command": "npx",
      "args": ["-y", "firecrawl-lite-mcp-server"],
      "env": {
        "LLM_API_KEY": "sk-...",
        "LLM_PROVIDER_BASE_URL": "https://api.openai.com/v1",
        "LLM_MODEL": "gpt-5.5"
      }
    }
  }
}
```
</details>

### Agents that speak Firecrawl

Any framework with a "self-hosted Firecrawl" option works unchanged — set its Firecrawl URL to your Firecrawl Lite instance. The server implements the scrape endpoint the SDKs call; search returns a clear `501` because this is a renderer, not a search engine, so pair it with whatever search provider your agent supports.

<details>
<summary><strong>Example: Hermes Agent</strong></summary>

[Hermes](https://hermes-agent.nousresearch.com) uses Firecrawl for `web_extract` and lets you split search and extract across providers.

```bash
# ~/.hermes/.env
FIRECRAWL_API_URL=http://your-server:3000
# FIRECRAWL_API_KEY=...   only if you set one on the server
```

```yaml
# ~/.hermes/config.yaml
web:
  search_backend: ddgs        # DuckDuckGo, no key. Or searxng, brave-free, ...
  extract_backend: firecrawl  # → Firecrawl Lite
```

Hermes's native `web_extract` now renders through your local browser. It also supports MCP servers if you want `screenshot` and `extract_with_schema` too.
</details>

<details>
<summary><strong>Example: Firecrawl SDKs</strong></summary>

```python
from firecrawl import Firecrawl

fc = Firecrawl(api_key="unused", api_url="http://your-server:3000")
doc = fc.scrape("https://example.com", formats=["markdown"])
print(doc.markdown)
```

The JS SDK works the same way with `apiUrl`.
</details>

## Configuration

Everything is an environment variable, and everything has a default. Full annotated list in [`.env.example`](.env.example).

### Endpoints

Running via `npx` with nothing set = MCP over stdio. Enable any of these and it becomes an HTTP server on `PORT` (default `3000`) instead. The Docker image turns on `/mcp` and the Firecrawl API out of the box.

| Variable | |
| --- | --- |
| `ENABLE_FIRECRAWL_API` | `POST /v2/scrape` — the Firecrawl-compatible API |
| `FIRECRAWL_API_KEY` | Optional. If set, that API requires `Authorization: Bearer <key>` |
| `ENABLE_HTTP_STREAMABLE_ENDPOINT` | `/mcp` — remote MCP for Claude Code and friends |
| `ENABLE_SSE_ENDPOINT` | `/sse` — legacy MCP transport (Claude Desktop via `mcp-proxy`). Deprecated. |

`/health` is always there when HTTP is on. Without Docker: `ENABLE_FIRECRAWL_API=true npx -y firecrawl-lite-mcp-server`.

### LLM (only for `extract_*`)

| Variable | |
| --- | --- |
| `LLM_PROVIDER_BASE_URL` | Any OpenAI-compatible base URL; the server calls `{base}/chat/completions` |
| `LLM_MODEL` | Model name |
| `LLM_API_KEY` | Your key |
| `LLM_TEMPERATURE` · `LLM_MAX_TOKENS` · `LLM_TOP_P` · `LLM_REASONING_EFFORT` | Optional tuning, passed straight through. Defaults `0.1` / `2000` / unset / unset. |

<details>
<summary>Provider cheat sheet</summary>

```bash
# OpenAI
LLM_PROVIDER_BASE_URL=https://api.openai.com/v1        LLM_MODEL=gpt-5.5
# Anthropic
LLM_PROVIDER_BASE_URL=https://api.anthropic.com/v1     LLM_MODEL=claude-haiku-4-5
# xAI
LLM_PROVIDER_BASE_URL=https://api.x.ai/v1              LLM_MODEL=grok-4
# OpenRouter
LLM_PROVIDER_BASE_URL=https://openrouter.ai/api/v1     LLM_MODEL=openai/gpt-5.5
# Ollama (local)
LLM_PROVIDER_BASE_URL=http://localhost:11434/v1        LLM_MODEL=llama3.3
```
</details>

### Scraping behaviour

| Variable | Default | |
| --- | --- | --- |
| `SCRAPE_USER_AGENT` | a current Chrome UA | One string, or a JSON array to rotate through (keep it on one line) |
| `SCRAPE_VIEWPORT_WIDTH` / `_HEIGHT` | `1920` / `1080` | |
| `SCRAPE_DELAY_MIN` / `_MAX` | `1000` / `3000` | Random pause before navigating (ms) |
| `SCRAPE_BATCH_DELAY_MIN` / `_MAX` | `2000` / `5000` | Random pause between batch URLs (ms) |
| `SCRAPE_SETTLE_MAX_MS` | `3000` | How long to wait for a page to stop changing. Raise it for sites that inject content on a slow `setTimeout`. |
| `SCRAPE_STRIP_LINK_URLS` | `false` | Replace `[text](url)` with `text` in the Markdown. Inline URLs are 40–60% of the tokens on link-heavy pages; the Firecrawl API returns them in `links` instead. |
| `SCRAPE_MAX_CHARS` | `0` (unlimited) | Hard cap on Markdown/text output, ending in a `[truncated — N more characters]` marker. |
| `FIRECRAWL_RETRY_MAX_ATTEMPTS` | `3` | Attempts per scrape, each advancing the proxy / UA rotation |

### Proxy

```bash
PROXY_SERVER_URL=http://proxy.example.com:10001-10010   # a port range = automatic rotation
PROXY_SERVER_USERNAME=...
PROXY_SERVER_PASSWORD=...
PROXY_LLM_API=false   # proxies are for target sites; LLM calls go direct unless you say otherwise
```

## Deploying

```bash
docker run -d -p 3000:3000 \
  -e LLM_API_KEY=... -e LLM_PROVIDER_BASE_URL=... -e LLM_MODEL=... \
  ariangibson/firecrawl-lite-mcp-server:latest
```

Or use the bundled [`docker-compose.yml`](docker-compose.yml) with a `.env` file. It uses a `wget` health check on purpose: the Alpine image has no `curl`, and a `curl` check will restart-loop the container (see [Troubleshooting](#troubleshooting)).

Images are multi-arch (`amd64` / `arm64`), published to `ariangibson/firecrawl-lite-mcp-server` on Docker Hub and `ghcr.io/ariangibson/firecrawl-lite-mcp-server`. Tags: `latest` moves on every push to `main`; **`stable` only moves on releases** — track `stable` in production (it pairs well with [Watchtower](https://containrrr.dev/watchtower/) for hands-off updates); version tags (`1.5.0`, `1.5`, `1`) pin exactly.

**Remote MCP clients:** Claude Code → `claude mcp add -t http firecrawl-lite http://your-server:3000/mcp`. Claude Desktop → Settings → Connectors with an HTTPS URL, or `mcp-proxy http://your-server:3000/sse` if you don't have a certificate (needs `ENABLE_SSE_ENDPOINT=true`).

## Firecrawl-compatible API

Implements the slice of the [Firecrawl v2 API](https://docs.firecrawl.dev/api-reference/endpoint/scrape) that SDK clients use for extraction.

| Endpoint | |
| --- | --- |
| `POST /v2/scrape` (alias `/v1/scrape`) | Request: `{ "url", "formats": ["markdown","html"], "onlyMainContent": true }`. Response: `{ "success": true, "data": Document }` with `markdown` / `html` / `rawHtml` / `links` per requested format, plus `metadata` (`title`, `description`, `language`, `sourceURL`, `url`, `statusCode`). |
| `POST /v2/search` (alias `/v1/search`) | `501` with a message telling you to use a real search backend. |

`formats` defaults to `["markdown"]` and accepts both `"markdown"` and `{ "type": "markdown" }`; `screenshot` is accepted but ignored. Two extensions beyond stock Firecrawl: `stripLinkUrls` (boolean) and `maxChars` (number) override the `SCRAPE_STRIP_LINK_URLS` / `SCRAPE_MAX_CHARS` defaults per request — and when link URLs are stripped, `links` is always included so nothing is lost. With `FIRECRAWL_API_KEY` set, both endpoints return `401` before anything else. Crawl, map, batch jobs, and the other async endpoints aren't implemented.

## Troubleshooting

**`Could not find Chrome`** — the `npx` postinstall step downloads Chrome the first time; if it's missing: `npx puppeteer browsers install chrome`. Corrupted? `rm -rf ~/.cache/puppeteer` and run it again.

**Scraping works, `extract_*` doesn't** — it's the LLM call. The server logs the provider's status and response body to stderr. `401` = bad key. `400` = wrong model name or a tuning param the model rejects. `429` = you're being rate limited.

**Container restart-loops with `SIGTERM` right after "listening on port 3000"** — a `curl` health check is failing because the image has no `curl`. Use `wget --spider http://localhost:3000/health` (the bundled compose file already does).

**Agent says Firecrawl isn't configured / extract fails** — `curl http://your-server:3000/health` should show `endpoints.firecrawlApi` = `"enabled"`. If you set `FIRECRAWL_API_KEY` on the server, the agent needs the same value. Search failing is expected unless the agent has a separate search provider.

## Under the hood

For the curious or the contributing: `config.ts` parses every env var once into a typed object; `browser.ts` owns one stealth browser session (launch flags, proxy, UA, navigation, cleanup, retries); `scraper.ts` puts rotation, retries, and scroll/settle heuristics behind two methods, `scrape` and `screenshot`; `htmlToMarkdown.ts` does the cleaning and conversion; `index.ts` is just MCP tool definitions, LLM extraction, and transports. The browser and clock are injected, so the whole scrape pipeline is tested through a stub browser — `npm test` needs no Chrome and no network.

```bash
npm install && npm run build && npm test
```

Releases: bump `version`, `git tag vX.Y.Z`, push the tag. CI publishes to npm (with provenance) and Docker Hub / GHCR.

## Credits

Inspired by the [Firecrawl](https://firecrawl.com) team and their official [MCP server](https://github.com/firecrawl/firecrawl-mcp-server). This is an independent, deliberately small take on the same idea. If you want the managed, enterprise-grade version with crawling, search, and a team behind it, that's [firecrawl.com](https://firecrawl.com).

## License

MIT — see [LICENSE](LICENSE).