cloudflare-crawl-mcp
Server Configuration
Describes the environment variables required to run the server.
| Name | Required | Description | Default |
|---|---|---|---|
| CF_API_TOKEN | Yes | Cloudflare API token with Browser Rendering Edit permission | |
| CF_ACCOUNT_ID | Yes | Cloudflare Account ID | |
| CF_RATE_LIMIT | No | API requests per minute (default: 6 for Free, 600 for Paid) | 6 |
Instructions
Guidance the server publishes about itself, which clients place ahead of the tool catalog so the model reads it before choosing anything.
This server publishes no instructions, or was last inspected before Glama recorded them.
Capabilities
Features and capabilities supported by this server
Protocol revision2025-11-25
| Capability | Details |
|---|---|
| tools | {
"listChanged": true
} |
| prompts | {
"listChanged": false
} |
| resources | {
"subscribe": false,
"listChanged": false
} |
| extensions | {
"io.modelcontextprotocol/ui": {}
} |
| experimental | {} |
Tools
Functions exposed to the LLM to take actions
| Name | Description |
|---|---|
| scrape_urlA | Fetch a single web page and return its content as Markdown. This is the primary tool for reading web content. Use this when you know which URL contains the information you need. Best for: Reading documentation pages, articles, blog posts, API references, or any single page where you know the URL. Not recommended for: When you don't know which page has the info (use map_url first), or when you need content from many pages (use crawl_url). Args: url: The URL to fetch. render: If true (default), renders JavaScript with a headless browser. Set to false for faster fetch of static pages. |
| map_urlA | Discover URLs on a website without fetching full page content. Returns a list of URLs found by crawling from the starting URL. Use this to find the right page before scraping it. Best for: Finding documentation pages, locating specific content on a site, understanding site structure before scraping. Typical workflow: map_url to find URLs -> scrape_url on the right page. Args: url: The starting URL to discover links from. limit: Maximum number of URLs to discover (default: 50, max: 100000). depth: How many links deep to follow (default: 2). include_subdomains: If true, follows links to subdomains. include_external_links: If true, follows links to external domains. include_patterns: Only visit URLs matching these wildcard patterns (e.g. "https://example.com/docs/**"). exclude_patterns: Skip URLs matching these wildcard patterns. |
| crawl_urlA | Crawl multiple pages starting from a URL and return all content as Markdown. Best for: Fetching entire documentation sections, blog archives, or multiple related pages at once. Not recommended for: Single pages (use scrape_url — it's faster). Large sites without filters (responses can be very large and exceed token limits). Tip: Use include_patterns to scope the crawl (e.g. "https://example.com/docs/**"). Args: url: The starting URL to crawl. limit: Maximum number of pages to crawl (default: 10, max: 100000). depth: Maximum link depth from the starting URL (default: 1). include_subdomains: If true, follows links to subdomains. include_external_links: If true, follows links to external domains. include_patterns: Only visit URLs matching these wildcard patterns. exclude_patterns: Skip URLs matching these wildcard patterns. render: If true (default), renders JavaScript. Set false for faster static fetch. |
Prompts
Interactive templates invoked by user choice
| Name | Description |
|---|---|
No prompts | |
Resources
Contextual data attached and managed by the client
| Name | Description |
|---|---|
No resources | |
TDQS
Scored across 3 tools
Each tool has a clear, distinct purpose: scrape_url fetches one page, map_url discovers URLs without fetching content, and crawl_url fetches multiple pages. The descriptions explicitly state when to use each, so there is no ambiguity.
All tool names follow a consistent verb_noun pattern with underscores: scrape_url, map_url, crawl_url. The style is uniform and predictable, making it easy to infer functionality.
Three tools is well-scoped for a crawling server. Each tool covers a distinct core operation—single fetch, URL discovery, and bulk crawl—without unnecessary redundancy or bloat.
The tools cover the essential crawl lifecycle: discover URLs, scrape a single page, and crawl multiple pages. Minor gaps exist (e.g., no sitemap parsing or headless options for custom headers), but the core workflows are complete.