Skip to main content
Glama
Sign In
spider-cloud-mcp

spider-cloud-mcp

by spider-rs
OverviewSchemaSourceRelated ServersDiscussionsAdmin
TypeScript
Remote

spider-cloud-mcp

MCP server for the Spider web crawling and scraping API. Crawl, scrape, search, and extract web data for AI agents, RAG pipelines, and LLMs.

Setup

1. Get your API key

Sign up at spider.cloud and get your API key from the API Keys page.

2. Configure your MCP client

Claude Desktop

Add to your claude_desktop_config.json:

{
  "mcpServers": {
    "spider": {
      "command": "npx",
      "args": ["-y", "spider-cloud-mcp"],
      "env": {
        "SPIDER_API_KEY": "your-api-key"
      }
    }
  }
}

Claude Code

claude mcp add spider -- npx -y spider-cloud-mcp

Then set your API key in the environment or .env file:

SPIDER_API_KEY=your-api-key

Cursor

Add to your Cursor MCP settings:

{
  "mcpServers": {
    "spider": {
      "command": "npx",
      "args": ["-y", "spider-cloud-mcp"],
      "env": {
        "SPIDER_API_KEY": "your-api-key"
      }
    }
  }
}

Related MCP server: crawl4ai-mcp

Tools

Core Tools

Tool

Description

spider_crawl

Crawl a website and extract content from multiple pages

spider_scrape

Scrape a single page (no crawling)

spider_search

Search the web with optional page content fetching

spider_links

Extract all links from a page

spider_screenshot

Capture page screenshots

spider_transform

Transform HTML to markdown/text/other formats

spider_get_credits

Check your credit balance

AI Tools (Subscription Required)

These tools require an active AI subscription plan.

Tool

Description

spider_ai_crawl

AI-guided crawling with natural language prompts

spider_ai_scrape

Extract structured data using plain English

spider_ai_search

AI-enhanced semantic web search

spider_ai_browser

Automate browser interactions with natural language

spider_ai_links

Intelligent link extraction and filtering

Examples

Crawl a website to markdown

Use spider_crawl to crawl https://example.com with limit 10 and return_format "markdown"

Search the web

Use spider_search to search for "latest AI research papers" with fetch_page_content true and num 5

AI-powered extraction

Use spider_ai_scrape on https://news.ycombinator.com with prompt "Extract all article titles, URLs, points, and comment counts as structured JSON"

Check credits

Use spider_get_credits to check my balance

API Reference

Full API documentation: spider.cloud/docs/api

License

MIT

Available Tools

13 tools
spider_ai_browserA

AI-powered browser automation using natural language. Describe actions like 'click login, fill email, submit form' and Spider automates the browser. REQUIRES an active AI subscription plan (https://spider.cloud/ai/pricing).

Inputs: url (required), prompt (required), cookies, proxy_enabled, +1 more.

spider_ai_crawlA

AI-guided crawling using natural language prompts. Describe what content to find and Spider's AI will guide the crawl. REQUIRES an active AI subscription plan (https://spider.cloud/ai/pricing).

Inputs: url (required), prompt (required), limit, cookies, +3 more.

spider_ai_linksA

AI-powered intelligent link extraction and filtering. Describe what links you want and Spider uses AI to find and categorize them. REQUIRES an active AI subscription plan (https://spider.cloud/ai/pricing).

Inputs: url (required), prompt (required), limit, request, +1 more.

spider_ai_scrapeA

AI-powered structured data extraction using plain English. Describe what data you want and get structured JSON back — no CSS selectors needed. REQUIRES an active AI subscription plan (https://spider.cloud/ai/pricing).

Inputs: url (required), prompt (required), cookies, request, +2 more.

spider_ai_searchB

AI-enhanced semantic web search. Uses intent understanding and relevance ranking to find the most relevant results. REQUIRES an active AI subscription plan (https://spider.cloud/ai/pricing).

Inputs: search (required), num, tbs, prompt, +4 more.

spider_crawlB

Crawl a website and extract content from multiple pages. Returns page content in the specified format (markdown, HTML, text, etc.). Powered by Spider - the fastest web crawler at 100K+ pages/sec.

Inputs: url (required), tld, cron, cache, +59 more.

spider_get_creditsA

Check your available Spider API credit balance.

Inputs: no named parameters.

spider_linksA

Extract all links from a page without fetching content. Fast way to discover URLs on a site.

Inputs: url (required), limit, request, return_format.

spider_scrapeB

Scrape a single page and extract its content. No crawling — just fetches and processes one URL. Supports all output formats and screenshot capture.

Inputs: url (required), tld, cron, cache, +62 more.

spider_screenshotC

Capture screenshots of web pages. Returns base64-encoded images or binary data.

Inputs: url (required), binary, locale, cookies, +15 more.

spider_searchC

Search the web and optionally crawl results. Returns search results with optional full page content.

Inputs: search (required), num, tbs, url, +13 more.

spider_transformA

Transform HTML content to markdown, text, or other formats. No network requests — processes HTML you provide directly.

Inputs: data (required), clean, clean_full, readability, +1 more.

spider_unblockerB

Access blocked or protected content with advanced anti-bot bypass. Uses enhanced fingerprinting and proxy rotation. Adds 10-40 extra credits per successful unblock.

Inputs: url (required), tld, cron, cache, +62 more.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    Deploy Server
    One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

    TDQS

    B3.4/5.0

    Scored across 13 tools

    Disambiguation4/5

    Most tools have distinct purposes (crawl vs. scrape vs. screenshot), but the AI variants (spider_ai_crawl, spider_ai_links) overlap conceptually with their non-AI counterparts, requiring careful description reading to select the right one.

    Naming Consistency3/5

    All tools share the spider_ prefix, but the second word mixes verbs (crawl, scrape, search, transform), nouns (links, unblocker), and verb phrases (get_credits), and the AI group includes both verbs and nouns (ai_browser). This inconsistency makes the API less predictable.

    Tool Count4/5

    13 tools is slightly above the typical sweet spot but still reasonable for the server's scope, which includes crawling, scraping, linking, screenshots, transformation, and multiple AI-powered variants.

    Completeness4/5

    The tool surface covers crawling, scraping, link extraction, screenshots, transformation, and credit checking thoroughly. The AI tools extend the surface with additional capabilities. Minor gaps might include batch processing or bulk URL handling, but core workflows are well served.

    Resources

    Related MCP Connectors

    Related MCP Servers

    • crawl4ai-mcp

      stgmt
      A
      license
      Not graded
      quality
      D
      maintenance
      MCP server integrating Crawl4AI for universal web crawling and data extraction. Enables AI agents to crawl, extract markdown/HTML, take screenshots, generate PDFs, and execute JavaScript on web pages.
      32 npm
      7
      MIT

    • crawl4ai-mcp-server

      sadiuysal
      A
      license
      Not graded
      quality
      D
      maintenance
      A lightweight MCP server that exposes Crawl4AI web scraping and crawling capabilities as tools for AI agents, enabling single-page scraping and multi-page crawling with adaptive stopping.
      118
      MIT

    • crawl4ai-mcp-server

      amienbou121
      -
      license
      Not graded
      quality
      C
      maintenance
      Self-hosted MCP server that provides web scraping and crawling tools, integrating seamlessly with AI frameworks like OpenAI Agents SDK, Cursor, and Claude Code.
      4
      -