Skip to main content
Glama
bidouilles

cloudflare-crawl-mcp

by bidouilles

cloudflare-crawl-mcp

MCP server for Cloudflare Browser Rendering Crawl API. Fetches and crawls web pages, returning clean Markdown optimized for LLM consumption.

Built with FastMCP and uv.

Tools

Tool

Description

scrape_url

Fetch a single page as Markdown. The primary tool — use this when you know the URL.

map_url

Discover URLs on a site without fetching content. Use to find the right page first.

crawl_url

Crawl multiple pages and return all content as Markdown.

Typical workflow: map_url to find pages → scrape_url to read the right one.

Related MCP server: superFetch MCP Server

Prerequisites

  1. A Cloudflare account with Browser Rendering enabled

  2. An API token with Account > Browser Rendering > Edit permission (create one here)

  3. uv installed

Configuration

Variable

Required

Description

CF_API_TOKEN

Yes

Cloudflare API token

CF_ACCOUNT_ID

Yes

Cloudflare Account ID

CF_RATE_LIMIT

No

API requests per minute (default: 6 for Free, set to 600 for Paid)

Setup

Claude Code

claude mcp add cloudflare-crawl \
  -e CF_API_TOKEN=your_api_token \
  -e CF_ACCOUNT_ID=your_account_id \
  -- uv run --directory /path/to/cloudflare-crawl-mcp python server.py

Codex

codex mcp add cloudflare-crawl \
  -- env CF_API_TOKEN="your_api_token" CF_ACCOUNT_ID="your_account_id" \
  uv run --directory /path/to/cloudflare-crawl-mcp python server.py

Claude Desktop

Add to ~/Library/Application Support/Claude/claude_desktop_config.json (macOS) or %APPDATA%\Claude\claude_desktop_config.json (Windows):

{
  "mcpServers": {
    "cloudflare-crawl": {
      "command": "uv",
      "args": ["run", "--directory", "/path/to/cloudflare-crawl-mcp", "python", "server.py"],
      "env": {
        "CF_API_TOKEN": "your_api_token",
        "CF_ACCOUNT_ID": "your_account_id"
      }
    }
  }
}

Cursor

Add to .cursor/mcp.json in your project or ~/.cursor/mcp.json globally:

{
  "mcpServers": {
    "cloudflare-crawl": {
      "command": "uv",
      "args": ["run", "--directory", "/path/to/cloudflare-crawl-mcp", "python", "server.py"],
      "env": {
        "CF_API_TOKEN": "your_api_token",
        "CF_ACCOUNT_ID": "your_account_id"
      }
    }
  }
}

Tool Reference

scrape_url

Fetch a single page and return its content as Markdown.

url (string, required)    — The URL to fetch
render (boolean, optional) — Render JavaScript with headless browser (default: true, set false for static pages)

map_url

Discover URLs on a website without fetching full content.

url (string, required)                  — Starting URL
limit (number, optional)                — Max URLs to discover (default: 50)
depth (number, optional)                — Link depth to follow (default: 2)
include_subdomains (boolean, optional)  — Follow subdomain links (default: false)
include_external_links (boolean, optional) — Follow external links (default: false)
include_patterns (string[], optional)   — Only visit matching URLs (e.g. "https://example.com/docs/**")
exclude_patterns (string[], optional)   — Skip matching URLs

crawl_url

Crawl multiple pages and return all content as Markdown.

url (string, required)                  — Starting URL
limit (number, optional)                — Max pages to crawl (default: 10)
depth (number, optional)                — Link depth (default: 1)
include_subdomains (boolean, optional)  — Follow subdomain links (default: false)
include_external_links (boolean, optional) — Follow external links (default: false)
include_patterns (string[], optional)   — Only visit matching URLs
exclude_patterns (string[], optional)   — Skip matching URLs
render (boolean, optional)              — Render JavaScript (default: true)

Cloudflare Plan Limits

Free

Paid

Browser time

10 min/day

10 hrs/month

API rate limit

6 req/min

600 req/min

Concurrent browsers

3

10

Max pages per job

100,000

100,000

Max job duration

7 days

7 days

Results available

14 days

14 days

render: false crawls run on Workers instead of a headless browser and do not consume browser time.

License

MIT

F
license - not found
-
quality - not tested
D
maintenance

Maintenance

Maintainers
Response time
Release cycle
Releases (12mo)
Commit activity

Resources

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/bidouilles/cloudflare-crawl-mcp'

If you have feedback or need assistance with the MCP directory API, please join our Discord server