Skip to main content
Glama
FlyCrawl MCP Server

FlyCrawl MCP Server

Official

🐝 FlyCrawl Model Context Protocol (MCP) Server

npm version License: MIT TDQS Certified

Official Model Context Protocol (MCP) server for FlyCrawl — the high-performance Web-to-Markdown engine and AI crawling API. Directly connects Claude Desktop, Cursor AI, Windsurf, and autonomous LLM agents to the live web with zero token bloat and automatic anti-bot bypass.


⚡ Quick Start

1. Cursor IDE Setup

In Cursor: Settings -> Features -> MCP Servers -> + Add New MCP Server:

  • Name: flycrawl

  • Type: command

  • Command: npx -y flycrawl-mcp

  • Environment Variables: FLYCRAWL_API_KEY=fc_live_YOUR_API_KEY

2. Claude Desktop Setup

Add this block to your claude_desktop_config.json (%APPDATA%\Claude\claude_desktop_config.json on Windows or ~/Library/Application Support/Claude/claude_desktop_config.json on macOS):

{
  "mcpServers": {
    "flycrawl": {
      "command": "npx",
      "args": ["-y", "flycrawl-mcp"],
      "env": {
        "FLYCRAWL_API_KEY": "fc_live_YOUR_API_KEY"
      }
    }
  }
}

Related MCP server: icecrawl MCP server

🛠️ TDQS 5/5 Behavioral Disclosures & Tool Spec

Every FlyCrawl tool is certified with explicit behavioral disclosures, rate limits, side-effect boundaries, dynamic JS rendering specs, and zero-risk billing guarantees.

Tool Name

Behavior & Idempotency

Dynamic JS / Anti-bot

Auth Required

Billing & Fair Guarantee

Scope & Difference

flycrawl_scrape

Read-only, idempotent

Auto Playwright rendering + stealth residential proxies

FLYCRAWL_API_KEY

1 credit/page. 0 credits on 4xx/5xx failures.

Converts full page body to clean Fit-Markdown (90% token saving).

flycrawl_crawl

Read-only web traversal

Dynamic JS per page + domain boundary check

FLYCRAWL_API_KEY

1 credit/page. 0 credits on failed pages.

Traverses sub-pages recursively up to configured depth/limit.

flycrawl_map

Read-only reconnaissance

Headless sitemap + SPA route parsing

FLYCRAWL_API_KEY

1 credit per domain mapping

Discovers URL structure in < 2s without downloading full page bodies.

flycrawl_search

Read-only web search

Real-time index without search engine ads

FLYCRAWL_API_KEY

1 credit per search query

Searches live web and returns scraped top results in 1 atomic call.

flycrawl_extract

Read-only, idempotent

Dynamic JS rendering + schema validation

FLYCRAWL_API_KEY

1 credit/call. 0 credits on failure.

Extracts validated JSON conforming to your prompt and JSON schema.


🛡️ Fair Billing Guarantee

  • 0-Credit Deduction on Failures: If an upstream target returns 404 Not Found, 403 Forbidden, 500 Server Error, or times out, exactly 0 credits are billed.

  • Structured Error Schema: Failures return standard JSON indicating { isError: true, status: <CODE>, billed_credits: 0, retryable: boolean }.

  • Free Credits: Claim 100 free credits on sign up at https://flycrawl.net.


📦 Canonical Package Ecosystem

  • TypeScript / Node.js SDK: npm install flycrawl

  • MCP Server (npx): npx -y flycrawl-mcp

  • Python SDK: pip install flycrawl

  • Official Documentation: https://flycrawl.net/docs


📄 License

MIT © FlyCrawl Core Team.

Available Tools

5 tools
flycrawl_crawlA

Recursively crawl multiple pages of an entire website following internal links up to a configured depth and page limit.

BEHAVIOR & SAFETY: Read-only web traversal. Strict domain boundary enforcement; will not traverse outside the target domain. AUTHENTICATION: Requires valid FLYCRAWL_API_KEY. RATE LIMITS & BILLING: Deducts 1 credit per successfully crawled page. 0 credits billed for failed pages. DYNAMIC JS RENDERING: Renders dynamic JavaScript on each crawled page with automated anti-bot bypass. DIFFERENCE FROM flycrawl_map: flycrawl_map only discovers URLs without downloading page contents; flycrawl_crawl fetches and converts full page bodies to Markdown. ERROR HANDLING: Returns aggregated Markdown or structured error JSON on network or authentication failure.

NameRequiredDescriptionDefault
urlYesStarting root URL of the website to crawl.
depthNoMaximum traversal depth from root (0-3). Default is 1.
limitNoMaximum pages to crawl (1-50). Default is 10.

TDQS

A4.5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and does so thoroughly: read-only traversal, strict domain-boundary enforcement, API key requirement, credit billing (including failed-page exemption), JS rendering with anti-bot bypass, and error-return behavior. This is exactly the behavioral context an agent needs before invoking a crawl that consumes credits.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded purpose sentence followed by labeled sections (BEHAVIOR & SAFETY, AUTHENTICATION, RATE LIMITS & BILLING, etc.). Every section conveys actionable information; there is no filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Although there is no output schema, the description explains that output is aggregated Markdown or structured error JSON, and it covers auth, billing, domain scoping, and JS rendering. With all three parameters fully schema-documented, an agent has everything needed to call this correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so url, depth, and limit are already fully documented in the schema. The description alludes to 'configured depth and page limit' but adds no syntax, range, or interaction details beyond what the schema states, so the baseline of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('recursively crawl multiple pages of an entire website') along with scope constraints (depth, page limit, internal links). It also explicitly distinguishes itself from flycrawl_map, so an agent can tell them apart without opening either schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The 'DIFFERENCE FROM flycrawl_map' section gives explicit routing guidance between two closely related tools. However, it does not address when to prefer flycrawl_scrape, flycrawl_search, or flycrawl_extract, leaving some sibling selection to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

flycrawl_extractA

Extract typed, structured JSON entities from any live webpage using natural language guidance and an optional JSON Schema.

BEHAVIOR & SAFETY: Read-only and idempotent. Makes no modifications to target servers. Safe for autonomous agents. AUTHENTICATION: Requires valid FLYCRAWL_API_KEY environment variable. RATE LIMITS & BILLING: Costs 1 credit per successful extraction. Fair billing guarantee: 0 credits are deducted if the target page fails to load, times out, or returns a 5xx error. DYNAMIC JS & HEADLESS BROWSING: Automatically executes dynamic client-side JavaScript (SPAs, React, Vue, Angular) and bypasses anti-bot systems (Cloudflare Turnstile, DataDome, Akamai) using stealth residential IP pools. DIFFERENCE FROM flycrawl_scrape: Unlike flycrawl_scrape (which returns the whole page converted to cleaned Fit-Markdown), flycrawl_extract uses an LLM to parse and transform the DOM into a validated JSON schema. ERROR HANDLING: Returns { 'isError': true, 'code': '<HTTP_STATUS>', 'error': '' } on network failure, timeout, or schema mismatch without raising unhandled exceptions.

NameRequiredDescriptionDefault
urlYesTarget URL to extract data from (must start with http:// or https://).
promptYesNatural language instructions describing what specific data fields to extract.
schemaNoOptional JSON Schema definition for the extraction structure.

TDQS

A4.3/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden and does so: read-only/idempotent, required API key, 1-credit cost with a no-charge guarantee on failure/timeout/5xx, dynamic JS execution and anti-bot behavior, and an explicit error shape without unhandled exceptions. These are behavioral facts an agent cannot infer from the schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded purpose sentence followed by labeled blocks that make scanning cheap. It is slightly longer than necessary — the billing guarantee and anti-bot detail are dense — but each block carries distinct, decision-relevant information rather than restating the name.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No output schema exists, and the description partially compensates by describing both success semantics (validated JSON per schema) and the failure envelope. The remaining gap is how results are wrapped relative to the supplied schema, which an agent would still have to discover empirically.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so all three parameters (url, prompt, schema) are already documented in the input schema. The description reinforces the natural-language + JSON Schema combination but adds no format, precedence, or conflict guidance (e.g., what happens when prompt and schema disagree). Baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Opens with a specific verb and resource: 'Extract typed, structured JSON entities from any live webpage,' plus the mechanism (natural-language guidance + optional JSON Schema). It also explicitly names the sibling it is not (flycrawl_scrape), so an agent can differentiate without opening either schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The 'DIFFERENCE FROM flycrawl_scrape' block gives the exact selection rule: whole-page cleaned Markdown vs LLM-parsed validated JSON. That is a genuine alternative-plus-condition. It stops short of exclusions (e.g., auth-gated pages, non-HTML targets), which keeps it below a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

flycrawl_mapA

Quickly discover and list all reachable internal URLs and sitemap hierarchy on a domain in under 2 seconds.

BEHAVIOR & SAFETY: Read-only and non-destructive. Queries sitemaps, robots.txt, and top-level link graphs. AUTHENTICATION: Requires valid FLYCRAWL_API_KEY. RATE LIMITS & BILLING: Consumes 1 credit per domain mapping operation. DYNAMIC JS RENDERING: Headless DOM parsing for dynamically injected single-page application router links. DIFFERENCE FROM flycrawl_crawl: flycrawl_map returns only a list of discovered URLs (no content bodies), making it ideal for reconnaissance before targeted scraping. ERROR HANDLING: Returns discovered URLs or structured error JSON.

NameRequiredDescriptionDefault
urlYesThe target website domain to map (e.g. 'https://stripe.com').
limitNoMaximum URLs to return (default: 1000).
searchNoOptional keyword to filter discovered URLs.

TDQS

A4.4/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations present, the description carries the full burden and does so: it declares read-only/non-destructive behavior, the data sources queried (sitemaps, robots.txt, link graphs), API-key auth, cost of 1 credit per operation, JS-rendering behavior, and error semantics. It even discloses the return shape (URLs only, no content bodies).

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The labeled sections are front-loaded and scannable, and each line adds information rather than restating the schema. The 'under 2 seconds' performance claim is slightly promotional and not load-bearing.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 3-parameter read tool with no output schema, the description covers purpose, safety, auth, cost, return content, and error behavior — everything an agent needs to decide to call it and interpret results.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so url, limit, and search are already fully documented in the schema. The description adds no parameter-level detail (e.g., what 'search' filters against or how limit truncates), so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource: 'discover and list all reachable internal URLs and sitemap hierarchy on a domain.' It also explicitly contrasts itself with flycrawl_crawl, so an agent can distinguish it from the closest sibling without opening either schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides a clear use case ('reconnaissance before targeted scraping') and names the alternative it is not (flycrawl_crawl, which returns content bodies). It does not address the other siblings (flycrawl_search, flycrawl_scrape, flycrawl_extract), so the routing guidance is strong but not exhaustive.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

flycrawl_scrapeA

Scrape a single webpage and convert it into clean, noise-free, LLM-ready Fit-Markdown, JSON, or links with 90%+ token reduction.

BEHAVIOR & SAFETY: Read-only and idempotent. Makes no modifications to target servers. AUTHENTICATION: Requires valid FLYCRAWL_API_KEY (claim 100 free credits at https://flycrawl.net). RATE LIMITS & BILLING: Consumes 1 credit per successful page scrape. Fair billing guarantee: 0 credits are deducted if the target site returns 4xx/5xx errors or network timeouts. DYNAMIC JS RENDERING: Automatically executes dynamic client-side JavaScript (SPAs, React, Vue, Angular) and bypasses anti-bot systems (Cloudflare Turnstile, DataDome) using stealth residential IP pools. DIFFERENCE FROM flycrawl_extract: flycrawl_scrape extracts the complete readable text and layout of the page as Markdown. Use flycrawl_extract when you need specific JSON fields conforming to a schema. ERROR HANDLING: Returns structured error message with status code on failure without raising unhandled exceptions.

NameRequiredDescriptionDefault
urlYesThe complete URL of the webpage to scrape (must start with http:// or https://).
formatNoTarget output format ('markdown', 'json', 'links', 'summary', 'html'). Default: markdown.markdown
maxAgeNoCache TTL in seconds (default 86400 = 24h). Cached requests are instant and cost 0 credits.
onlyMainContentNoWhether to strip navigation, headers, footers, and ads (default: true).

TDQS

A4.5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden and does so: read-only/idempotent, API key requirement, 1 credit per successful scrape with a no-charge guarantee on 4xx/5xx and timeouts, cache TTL cost behavior, JS rendering and anti-bot handling, and structured error returns. This is unusually rich behavioral context for a tool with zero annotation coverage.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Section labels (BEHAVIOR & SAFETY, AUTHENTICATION, RATE LIMITS, DIFFERENCE FROM) make it scannable and front-loaded with the purpose. It is longer than strictly necessary, and the format enumeration at the top is incomplete relative to the schema, but each block carries actionable information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With four parameters, no output schema, and no annotations, the description covers auth, cost, rendering, error format, and sibling differentiation adequately. The only shortfall is the incomplete output-format list, leaving the agent to discover summary and html from the schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3; the description adds value by explaining the billing implication of caching (cached requests cost 0 credits), which the maxAge schema does not state. It lists markdown/json/links but omits the summary and html enum values, a small gap rather than a contradiction.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (scrape) and resource (single webpage) plus the concrete output artifacts (Markdown, JSON, links). It explicitly names the sibling flycrawl_extract and the boundary between them, so an agent can distinguish the two without opening either schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives an explicit routing rule: use flycrawl_extract for schema-conforming JSON fields, this tool for full readable page text. It lacks guidance on when to prefer flycrawl_search, flycrawl_crawl, or flycrawl_map for multi-page work, but the primary alternative is clearly framed.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 5 tool updatesv1.0.1
    • First observedflycrawl_crawl
    • First observedflycrawl_extract
    • First observedflycrawl_map
    • First observedflycrawl_scrape
    • First observedflycrawl_search

TDQS

A4.5/5.0

Scored across 5 tools

Disambiguation5/5

Each tool explicitly documents how it differs from its closest neighbor (search vs scrape, scrape vs extract, crawl vs map). The five tools map to clearly distinct operations: query-based discovery, single-page scraping, recursive crawling, URL mapping, and schema-based extraction.

Naming Consistency5/5

All tools follow a consistent flycrawl_ prefix plus a single clear verb (search, scrape, crawl, map, extract). Predictable and unambiguous.

Tool Count5/5

Five tools is well-scoped for a web-crawling toolkit, and each earns its place with a distinct capability. No filler tools or obviously missing siblings.

Completeness4/5

The surface covers the full crawl lifecycle: discovery (search, map), retrieval (scrape, crawl), and structured extraction. Minor possible gaps like batch multi-URL scraping or cached-result retrieval exist but core workflows are fully supported.

Maintenance

ActivityMaintained
ResponsivenessNo recent community issues

Related MCP Connectors

Related MCP Servers

  • F
    license
    Not graded
    quality
    D
    maintenance
    Enables scraping single pages or crawling entire websites, converting content to markdown and optionally extracting structured data with Claude.
    -
  • A
    license
    Not graded
    quality
    A
    maintenance
    Enables AI agents to crawl and scrape websites, converting HTML to clean Markdown and structured metadata with support for JavaScript rendering, bot evasion, and SSRF protection.
    195
    MIT
  • A
    license
    A
    quality
    B
    maintenance
    Enables AI agents in Claude Code, Claude Desktop, Cursor, VS Code, Windsurf, Gemini CLI and Codex to scrape web pages into Markdown, text, HTML, JSON or PDF, extract structured data with selectors or autoparse, capture screenshots and PDFs the model can see, drive pages with typed browser actions, keep logged-in persistent browser sessions, and batch-scrape thousands of URLs as async jobs. It also exposes usage, request-log and documentation tools so agents can check costs, diagnose failures and look up parameters without leaving the chat.
    25
    Apache 2.0