Spider MCP Server

The fastest web crawling, scraping, and browser automation server for AI agents. Gives Claude direct access to the web through 22 tools — crawl sites at 100K+ pages/sec, extract structured data with AI, and control remote browsers with built-in anti-bot bypass.

Why Spider

• Speed — Crawl 100K+ pages per second. Smart request routing picks HTTP or headless Chrome automatically. Streaming responses start delivering data immediately.

• Cost — Pay-per-use credits with no subscription required for core tools. Check your balance anytime with spider_get_credits. AI tools available with an AI subscription.

• Reliability — Anti-bot bypass with fingerprinting and proxy rotation. Browser fleet with automatic fallback across Chrome, Firefox, and more. Built-in retry and stealth escalation.

Related MCP server: The Web MCP

Quick Start

Claude Code

claude mcp add spider -- npx -y spider-cloud-mcp

Set your API key:

export SPIDER_API_KEY="your-api-key"

Claude Desktop

Add to claude_desktop_config.json:

{
  "mcpServers": {
    "spider": {
      "command": "npx",
      "args": ["-y", "spider-cloud-mcp"],
      "env": {
        "SPIDER_API_KEY": "your-api-key"
      }
    }
  }
}

Cursor

Add to .cursor/mcp.json:

{
  "mcpServers": {
    "spider": {
      "command": "npx",
      "args": ["-y", "spider-cloud-mcp"],
      "env": {
        "SPIDER_API_KEY": "your-api-key"
      }
    }
  }
}

Windsurf

Add to ~/.codeium/windsurf/mcp_config.json:

{
  "mcpServers": {
    "spider": {
      "command": "npx",
      "args": ["-y", "spider-cloud-mcp"],
      "env": {
        "SPIDER_API_KEY": "your-api-key"
      }
    }
  }
}

Configuration

Tools

Core Tools (8)

These work on pay-per-use credits with no subscription required.

AI Tools (5)

Natural language web interaction. Describe what you want in plain English.

Requires an AI subscription.

Browser Automation Tools (9)

Direct browser control via spider-browser. Browsers run in Spider's cloud with smart retry, browser switching, anti-bot protection, proxy rotation, and automatic stealth escalation. Supports CDP and BiDi protocols.

Browser sessions auto-close after 5 minutes of inactivity. Always call spider_browser_close when done.

Examples

Research and RAG

"Crawl the React documentation and summarize the hooks API"

Uses spider_crawl to fetch 50+ pages in seconds and return clean markdown ready for context.

spider_crawl: {
  url: "https://react.dev/reference/react",
  limit: 50,
  return_format: "markdown",
  filter_output_main_only: true
}

Structured Data Extraction

"Get all product names and prices from this e-commerce page"

Uses spider_ai_scrape to extract structured JSON with zero CSS selectors.

spider_ai_scrape: {
  url: "https://example-store.com/products",
  prompt: "Extract every product name, price, and availability status as JSON"
}

Multi-Step Browser Automation

"Log into the dashboard, go to reports, and screenshot the monthly summary"

Uses spider_browser_* tools to drive a remote browser with full anti-bot protection.

1. spider_browser_open: { browser: "auto" }
2. spider_browser_navigate: { url: "https://app.example.com/login" }
3. spider_browser_fill: { selector: "input[name='email']", value: "user@example.com" }
4. spider_browser_fill: { selector: "input[name='password']", value: "..." }
5. spider_browser_click: { selector: "button[type='submit']" }
6. spider_browser_wait_for: { navigation: true }
7. spider_browser_navigate: { url: "https://app.example.com/reports/monthly" }
8. spider_browser_screenshot: {}
9. spider_browser_close: {}

Competitive Intelligence

"Search for recent AI startup funding rounds and get the details"

Uses spider_search with time filtering, then spider_scrape for details.

spider_search: {
  search: "AI startup Series A funding 2025",
  num: 10,
  fetch_page_content: true,
  return_format: "markdown",
  tbs: "qdr:m"
}

API Reference

All tools map directly to the Spider API. Core tools accept the same parameters as their API counterparts:

• Crawl/Scrape: url, return_format, request, readability, root_selector, proxy_enabled, cache, and many more

• Search: search, num, fetch_page_content, country, language, tbs

• AI tools: Add a prompt parameter describing what you want in natural language

• Browser tools: Use session_id from spider_browser_open for all operations

Full parameter reference: spider.cloud/docs/api

Links

• Website

• Documentation

• API Reference

• Get API Key

• Credit Pricing

• AI Subscription

• Discord

• Support

License

MIT