FlyCrawl MCP Server
OfficialClick on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@FlyCrawl MCP Serverscrape https://example.com and convert to markdown"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
🐝 FlyCrawl Model Context Protocol (MCP) Server
Official Model Context Protocol (MCP) server for FlyCrawl — the high-performance Web-to-Markdown engine and AI crawling API. Directly connects Claude Desktop, Cursor AI, Windsurf, and autonomous LLM agents to the live web with zero token bloat and automatic anti-bot bypass.
⚡ Quick Start
1. Cursor IDE Setup
In Cursor: Settings -> Features -> MCP Servers -> + Add New MCP Server:
Name:
flycrawlType:
commandCommand:
npx -y flycrawl-mcpEnvironment Variables:
FLYCRAWL_API_KEY=fc_live_YOUR_API_KEY
2. Claude Desktop Setup
Add this block to your claude_desktop_config.json (%APPDATA%\Claude\claude_desktop_config.json on Windows or ~/Library/Application Support/Claude/claude_desktop_config.json on macOS):
{
"mcpServers": {
"flycrawl": {
"command": "npx",
"args": ["-y", "flycrawl-mcp"],
"env": {
"FLYCRAWL_API_KEY": "fc_live_YOUR_API_KEY"
}
}
}
}Related MCP server: icecrawl MCP server
🛠️ TDQS 5/5 Behavioral Disclosures & Tool Spec
Every FlyCrawl tool is certified with explicit behavioral disclosures, rate limits, side-effect boundaries, dynamic JS rendering specs, and zero-risk billing guarantees.
Tool Name | Behavior & Idempotency | Dynamic JS / Anti-bot | Auth Required | Billing & Fair Guarantee | Scope & Difference |
| Read-only, idempotent | Auto Playwright rendering + stealth residential proxies |
| 1 credit/page. 0 credits on 4xx/5xx failures. | Converts full page body to clean Fit-Markdown (90% token saving). |
| Read-only web traversal | Dynamic JS per page + domain boundary check |
| 1 credit/page. 0 credits on failed pages. | Traverses sub-pages recursively up to configured depth/limit. |
| Read-only reconnaissance | Headless sitemap + SPA route parsing |
| 1 credit per domain mapping | Discovers URL structure in < 2s without downloading full page bodies. |
| Read-only web search | Real-time index without search engine ads |
| 1 credit per search query | Searches live web and returns scraped top results in 1 atomic call. |
| Read-only, idempotent | Dynamic JS rendering + schema validation |
| 1 credit/call. 0 credits on failure. | Extracts validated JSON conforming to your prompt and JSON schema. |
🛡️ Fair Billing Guarantee
0-Credit Deduction on Failures: If an upstream target returns
404 Not Found,403 Forbidden,500 Server Error, or times out, exactly 0 credits are billed.Structured Error Schema: Failures return standard JSON indicating
{ isError: true, status: <CODE>, billed_credits: 0, retryable: boolean }.Free Credits: Claim 100 free credits on sign up at https://flycrawl.net.
📦 Canonical Package Ecosystem
TypeScript / Node.js SDK:
npm install flycrawlMCP Server (npx):
npx -y flycrawl-mcpPython SDK:
pip install flycrawlOfficial Documentation: https://flycrawl.net/docs
📄 License
MIT © FlyCrawl Core Team.
Available Tools
5 toolsflycrawl_crawlA
Recursively crawl multiple pages of an entire website following internal links up to a configured depth and page limit.
BEHAVIOR & SAFETY: Read-only web traversal. Strict domain boundary enforcement; will not traverse outside the target domain. AUTHENTICATION: Requires valid FLYCRAWL_API_KEY. RATE LIMITS & BILLING: Deducts 1 credit per successfully crawled page. 0 credits billed for failed pages. DYNAMIC JS RENDERING: Renders dynamic JavaScript on each crawled page with automated anti-bot bypass. DIFFERENCE FROM flycrawl_map: flycrawl_map only discovers URLs without downloading page contents; flycrawl_crawl fetches and converts full page bodies to Markdown. ERROR HANDLING: Returns aggregated Markdown or structured error JSON on network or authentication failure.
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | Starting root URL of the website to crawl. | |
| depth | No | Maximum traversal depth from root (0-3). Default is 1. | |
| limit | No | Maximum pages to crawl (1-50). Default is 10. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden and does so thoroughly: read-only traversal, strict domain-boundary enforcement, API key requirement, credit billing (including failed-page exemption), JS rendering with anti-bot bypass, and error-return behavior. This is exactly the behavioral context an agent needs before invoking a crawl that consumes credits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded purpose sentence followed by labeled sections (BEHAVIOR & SAFETY, AUTHENTICATION, RATE LIMITS & BILLING, etc.). Every section conveys actionable information; there is no filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Although there is no output schema, the description explains that output is aggregated Markdown or structured error JSON, and it covers auth, billing, domain scoping, and JS rendering. With all three parameters fully schema-documented, an agent has everything needed to call this correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so url, depth, and limit are already fully documented in the schema. The description alludes to 'configured depth and page limit' but adds no syntax, range, or interaction details beyond what the schema states, so the baseline of 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('recursively crawl multiple pages of an entire website') along with scope constraints (depth, page limit, internal links). It also explicitly distinguishes itself from flycrawl_map, so an agent can tell them apart without opening either schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The 'DIFFERENCE FROM flycrawl_map' section gives explicit routing guidance between two closely related tools. However, it does not address when to prefer flycrawl_scrape, flycrawl_search, or flycrawl_extract, leaving some sibling selection to inference.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
flycrawl_extractA
Extract typed, structured JSON entities from any live webpage using natural language guidance and an optional JSON Schema.
BEHAVIOR & SAFETY: Read-only and idempotent. Makes no modifications to target servers. Safe for autonomous agents. AUTHENTICATION: Requires valid FLYCRAWL_API_KEY environment variable. RATE LIMITS & BILLING: Costs 1 credit per successful extraction. Fair billing guarantee: 0 credits are deducted if the target page fails to load, times out, or returns a 5xx error. DYNAMIC JS & HEADLESS BROWSING: Automatically executes dynamic client-side JavaScript (SPAs, React, Vue, Angular) and bypasses anti-bot systems (Cloudflare Turnstile, DataDome, Akamai) using stealth residential IP pools. DIFFERENCE FROM flycrawl_scrape: Unlike flycrawl_scrape (which returns the whole page converted to cleaned Fit-Markdown), flycrawl_extract uses an LLM to parse and transform the DOM into a validated JSON schema. ERROR HANDLING: Returns { 'isError': true, 'code': '<HTTP_STATUS>', 'error': '' } on network failure, timeout, or schema mismatch without raising unhandled exceptions.
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | Target URL to extract data from (must start with http:// or https://). | |
| prompt | Yes | Natural language instructions describing what specific data fields to extract. | |
| schema | No | Optional JSON Schema definition for the extraction structure. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden and does so: read-only/idempotent, required API key, 1-credit cost with a no-charge guarantee on failure/timeout/5xx, dynamic JS execution and anti-bot behavior, and an explicit error shape without unhandled exceptions. These are behavioral facts an agent cannot infer from the schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded purpose sentence followed by labeled blocks that make scanning cheap. It is slightly longer than necessary — the billing guarantee and anti-bot detail are dense — but each block carries distinct, decision-relevant information rather than restating the name.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema exists, and the description partially compensates by describing both success semantics (validated JSON per schema) and the failure envelope. The remaining gap is how results are wrapped relative to the supplied schema, which an agent would still have to discover empirically.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so all three parameters (url, prompt, schema) are already documented in the input schema. The description reinforces the natural-language + JSON Schema combination but adds no format, precedence, or conflict guidance (e.g., what happens when prompt and schema disagree). Baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Opens with a specific verb and resource: 'Extract typed, structured JSON entities from any live webpage,' plus the mechanism (natural-language guidance + optional JSON Schema). It also explicitly names the sibling it is not (flycrawl_scrape), so an agent can differentiate without opening either schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The 'DIFFERENCE FROM flycrawl_scrape' block gives the exact selection rule: whole-page cleaned Markdown vs LLM-parsed validated JSON. That is a genuine alternative-plus-condition. It stops short of exclusions (e.g., auth-gated pages, non-HTML targets), which keeps it below a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
flycrawl_mapA
Quickly discover and list all reachable internal URLs and sitemap hierarchy on a domain in under 2 seconds.
BEHAVIOR & SAFETY: Read-only and non-destructive. Queries sitemaps, robots.txt, and top-level link graphs. AUTHENTICATION: Requires valid FLYCRAWL_API_KEY. RATE LIMITS & BILLING: Consumes 1 credit per domain mapping operation. DYNAMIC JS RENDERING: Headless DOM parsing for dynamically injected single-page application router links. DIFFERENCE FROM flycrawl_crawl: flycrawl_map returns only a list of discovered URLs (no content bodies), making it ideal for reconnaissance before targeted scraping. ERROR HANDLING: Returns discovered URLs or structured error JSON.
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | The target website domain to map (e.g. 'https://stripe.com'). | |
| limit | No | Maximum URLs to return (default: 1000). | |
| search | No | Optional keyword to filter discovered URLs. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations present, the description carries the full burden and does so: it declares read-only/non-destructive behavior, the data sources queried (sitemaps, robots.txt, link graphs), API-key auth, cost of 1 credit per operation, JS-rendering behavior, and error semantics. It even discloses the return shape (URLs only, no content bodies).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The labeled sections are front-loaded and scannable, and each line adds information rather than restating the schema. The 'under 2 seconds' performance claim is slightly promotional and not load-bearing.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 3-parameter read tool with no output schema, the description covers purpose, safety, auth, cost, return content, and error behavior — everything an agent needs to decide to call it and interpret results.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so url, limit, and search are already fully documented in the schema. The description adds no parameter-level detail (e.g., what 'search' filters against or how limit truncates), so the baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource: 'discover and list all reachable internal URLs and sitemap hierarchy on a domain.' It also explicitly contrasts itself with flycrawl_crawl, so an agent can distinguish it from the closest sibling without opening either schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides a clear use case ('reconnaissance before targeted scraping') and names the alternative it is not (flycrawl_crawl, which returns content bodies). It does not address the other siblings (flycrawl_search, flycrawl_scrape, flycrawl_extract), so the routing guidance is strong but not exhaustive.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
flycrawl_scrapeA
Scrape a single webpage and convert it into clean, noise-free, LLM-ready Fit-Markdown, JSON, or links with 90%+ token reduction.
BEHAVIOR & SAFETY: Read-only and idempotent. Makes no modifications to target servers. AUTHENTICATION: Requires valid FLYCRAWL_API_KEY (claim 100 free credits at https://flycrawl.net). RATE LIMITS & BILLING: Consumes 1 credit per successful page scrape. Fair billing guarantee: 0 credits are deducted if the target site returns 4xx/5xx errors or network timeouts. DYNAMIC JS RENDERING: Automatically executes dynamic client-side JavaScript (SPAs, React, Vue, Angular) and bypasses anti-bot systems (Cloudflare Turnstile, DataDome) using stealth residential IP pools. DIFFERENCE FROM flycrawl_extract: flycrawl_scrape extracts the complete readable text and layout of the page as Markdown. Use flycrawl_extract when you need specific JSON fields conforming to a schema. ERROR HANDLING: Returns structured error message with status code on failure without raising unhandled exceptions.
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | The complete URL of the webpage to scrape (must start with http:// or https://). | |
| format | No | Target output format ('markdown', 'json', 'links', 'summary', 'html'). Default: markdown. | markdown |
| maxAge | No | Cache TTL in seconds (default 86400 = 24h). Cached requests are instant and cost 0 credits. | |
| onlyMainContent | No | Whether to strip navigation, headers, footers, and ads (default: true). |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden and does so: read-only/idempotent, API key requirement, 1 credit per successful scrape with a no-charge guarantee on 4xx/5xx and timeouts, cache TTL cost behavior, JS rendering and anti-bot handling, and structured error returns. This is unusually rich behavioral context for a tool with zero annotation coverage.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Section labels (BEHAVIOR & SAFETY, AUTHENTICATION, RATE LIMITS, DIFFERENCE FROM) make it scannable and front-loaded with the purpose. It is longer than strictly necessary, and the format enumeration at the top is incomplete relative to the schema, but each block carries actionable information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With four parameters, no output schema, and no annotations, the description covers auth, cost, rendering, error format, and sibling differentiation adequately. The only shortfall is the incomplete output-format list, leaving the agent to discover summary and html from the schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3; the description adds value by explaining the billing implication of caching (cached requests cost 0 credits), which the maxAge schema does not state. It lists markdown/json/links but omits the summary and html enum values, a small gap rather than a contradiction.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (scrape) and resource (single webpage) plus the concrete output artifacts (Markdown, JSON, links). It explicitly names the sibling flycrawl_extract and the boundary between them, so an agent can distinguish the two without opening either schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives an explicit routing rule: use flycrawl_extract for schema-conforming JSON fields, this tool for full readable page text. It lacks guidance on when to prefer flycrawl_search, flycrawl_crawl, or flycrawl_map for multi-page work, but the primary alternative is clearly framed.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
flycrawl_searchA
Search the live web and automatically scrape the top search results into clean, LLM-ready Markdown in a single call.
BEHAVIOR & SAFETY: Read-only live web search. AUTHENTICATION: Requires valid FLYCRAWL_API_KEY. RATE LIMITS & BILLING: Consumes 1 credit per search request (includes top scraped result summaries). SEARCH QUALITY & FRESHNESS: Queries live web index with real-time freshness, strips search advertisements and sponsored links, and extracts substantive article content. DIFFERENCE FROM flycrawl_scrape: flycrawl_search finds relevant pages by query keyword; flycrawl_scrape requires a specific target URL. ERROR HANDLING: Returns Markdown summary of results or structured error JSON.
| Name | Required | Description | Default |
|---|---|---|---|
| limit | No | Maximum number of search results to return (default: 5). | |
| query | Yes | The search query terms to execute on the live web. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden and does: it declares read-only live behavior, the FLYCRAWL_API_KEY requirement, 1-credit-per-request billing, ad/sponsored-link stripping, and the error return shape. That is authorization, cost, and failure-mode disclosure an agent needs before calling a billable external tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded lead sentence followed by clearly labeled sections, so scanning is fast. The labeled blocks are slightly ceremonial for a two-parameter tool, but every line conveys distinct information with no filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema exists, and the description compensates by stating the return is Markdown summaries or structured error JSON. Combined with auth, cost, and freshness notes, nothing essential for a correct call is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so both 'query' and 'limit' are already documented in the schema. The description adds only indirect hints ('top search results', 'result summaries') and no format or syntax guidance beyond that, so the baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb+resource (search the live web) plus the distinguishing behavior (auto-scraping top results into Markdown in one call). The explicit 'DIFFERENCE FROM flycrawl_scrape' line lets an agent separate this from its closest sibling without opening either schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly contrasts keyword-driven search with URL-driven scrape, which is precisely the selection decision an agent must make among these siblings. No exclusions are stated for the remaining siblings, but the primary ambiguity is resolved directly.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
5 tool updates
v1.0.1- First observed
flycrawl_crawl - First observed
flycrawl_extract - First observed
flycrawl_map - First observed
flycrawl_scrape - First observed
flycrawl_search
TDQS
Scored across 5 tools
Each tool explicitly documents how it differs from its closest neighbor (search vs scrape, scrape vs extract, crawl vs map). The five tools map to clearly distinct operations: query-based discovery, single-page scraping, recursive crawling, URL mapping, and schema-based extraction.
All tools follow a consistent flycrawl_ prefix plus a single clear verb (search, scrape, crawl, map, extract). Predictable and unambiguous.
Five tools is well-scoped for a web-crawling toolkit, and each earns its place with a distinct capability. No filler tools or obviously missing siblings.
The surface covers the full crawl lifecycle: discovery (search, map), retrieval (scrape, crawl), and structured extraction. Minor possible gaps like batch multi-URL scraping or cached-result retrieval exist but core workflows are fully supported.
Maintenance
Related MCP Connectors
Web scraping, search, crawling, structured extraction and research for AI agents.
Web scraping for AI agents. Converts URLs to clean, LLM-ready Markdown with anti-bot bypass.
Unblocking and fresh web data for agents: URL to Markdown, YouTube, Maps, Amazon, jobs. Pay per call
Web scraping for agents. Point it at a URL and it returns the page as clean markdown, JavaScript-rendered pages included. Point it at a site and it maps the URLs or crawls the section you need in the background, a few pages at a time so results fit in the conversation. Search the web and read full pages, extract fields with a JSON schema you define (validated, never invented), read a store's catalogue or a blog's posts from the platform's own feed, and check whether a page has changed. Failed requests cost nothing. The free plan includes 1,500 credits a month.
Related MCP Servers
- AlicenseNot gradedqualityDmaintenanceEnables web scraping, crawling, structured data extraction, and browser automation through multiple AI agents including OpenAI's CUA, Anthropic's Claude Computer Use, and Browser Use.6 npmMIT
- FlicenseNot gradedqualityDmaintenanceEnables scraping single pages or crawling entire websites, converting content to markdown and optionally extracting structured data with Claude.-
- AlicenseNot gradedqualityAmaintenanceEnables AI agents to crawl and scrape websites, converting HTML to clean Markdown and structured metadata with support for JavaScript rendering, bot evasion, and SSRF protection.195MIT

Spicrawl MCP Serverofficial
AlicenseAqualityBmaintenanceEnables AI agents in Claude Code, Claude Desktop, Cursor, VS Code, Windsurf, Gemini CLI and Codex to scrape web pages into Markdown, text, HTML, JSON or PDF, extract structured data with selectors or autoparse, capture screenshots and PDFs the model can see, drive pages with typed browser actions, keep logged-in persistent browser sessions, and batch-scrape thousands of URLs as async jobs. It also exposes usage, request-log and documentation tools so agents can check costs, diagnose failures and look up parameters without leaving the chat.25Apache 2.0