scrapedatshi-mcp

MCP (Model Context Protocol) server for the scrapedatshi RAG pipeline API.

Use scrapedatshi's scraping, crawling, extraction, and vector DB sync tools directly from Claude Desktop — no code required.

What you can do

Just talk to Claude naturally:

• "Scrape https://docs.example.com and give me the chunks"

• "Crawl https://example.com/products and extract the title and price from every page"

• "Sync https://docs.example.com to my Pinecone index using OpenAI embeddings"

• "What embedding providers does scrapedatshi support?"

Related MCP server: Hyperbrowser MCP Server

Tools exposed

Prerequisites

1. scrapedatshi account — Sign up at scrapedatshi.com

2. Add credits — Billing portal

3. Get your API key — starts with sds_...

4. Claude Desktop — Download here

5. Python 3.10+ — python.org

Installation

Option A — Install from PyPI (recommended, works with uvx)

pip install scrapedatshi-mcp

Or use uv for isolated installs:

uv tool install scrapedatshi-mcp

Option B — Install from source (local development)

git clone https://github.com/mxchris18/scrapedatshi-mcp.git
cd scrapedatshi-mcp
pip install -e .

Claude Desktop configuration

Open your Claude Desktop config file:

• macOS: ~/Library/Application Support/Claude/claude_desktop_config.json

• Windows: %APPDATA%\Claude\claude_desktop_config.json

If installed via PyPI / pip (using uvx)

{
  "mcpServers": {
    "scrapedatshi": {
      "command": "uvx",
      "args": ["scrapedatshi-mcp"],
      "env": {
        "SCRAPEDATSHI_API_KEY": "sds_your_key_here"
      }
    }
  }
}

If installed via pip (using python)

{
  "mcpServers": {
    "scrapedatshi": {
      "command": "python",
      "args": ["-m", "scrapedatshi_mcp.server"],
      "env": {
        "SCRAPEDATSHI_API_KEY": "sds_your_key_here"
      }
    }
  }
}

If cloned from source (absolute path)

{
  "mcpServers": {
    "scrapedatshi": {
      "command": "python",
      "args": ["/absolute/path/to/scrapedatshi-mcp/scrapedatshi_mcp/server.py"],
      "env": {
        "SCRAPEDATSHI_API_KEY": "sds_your_key_here"
      }
    }
  }
}

Restart Claude Desktop after saving the config.

Secure key configuration (BYOK)

You bring your own LLM, embedding, and vector DB keys. The server resolves keys in this priority order:

1. Argument passed in the tool call — explicit override

2. Environment variable in the MCP config — preferred secure path (keys never appear in chat)

3. Clear error message if neither is found

Add your provider keys to the env block in claude_desktop_config.json:

{
  "mcpServers": {
    "scrapedatshi": {
      "command": "uvx",
      "args": ["scrapedatshi-mcp"],
      "env": {
        "SCRAPEDATSHI_API_KEY": "sds_your_key_here",

        "OPENAI_API_KEY": "sk-...",
        "ANTHROPIC_API_KEY": "sk-ant-...",
        "GEMINI_API_KEY": "AIza...",

        "COHERE_API_KEY": "...",
        "MISTRAL_API_KEY": "...",
        "VOYAGE_API_KEY": "...",

        "PINECONE_API_KEY": "pc-...",
        "QDRANT_API_KEY": "...",
        "WEAVIATE_API_KEY": "..."
      }
    }
  }
}

Once set, Claude will automatically use these keys without asking you to type them in chat.

Supported environment variables

Example conversations

Scrape a single page

You: Scrape https://docs.example.com/getting-started and show me the chunks.

Claude calls scrape_url and returns the chunked content with token counts and credit usage.

Crawl a documentation site

You: Crawl https://docs.example.com — just the first 5 pages.

Claude calls crawl_site with max_pages=5 and returns all chunks from all pages.

Extract structured data from a product page

You: Extract the product name, price, and whether it's in stock from https://example.com/products/widget-pro

Claude calls extract_data with a schema it constructs from your request, using your OpenAI key from the env config.

Extract data from an entire product catalogue

You: Crawl https://example.com/products and extract the title and price from every product page. Limit to 10 pages.

Claude calls extract_crawl with max_pages=10 and returns per-page extraction results.

Sync a page to your vector DB

You: Sync https://docs.example.com to my Pinecone index. The index host is https://my-index-abc123.svc.pinecone.io. Use OpenAI text-embedding-3-small.

Claude calls sync_to_vectordb. If OPENAI_API_KEY and PINECONE_API_KEY are set in your env config, no keys need to be typed in chat.

Discover what's supported

You: What embedding providers does scrapedatshi support?

Claude calls list_embedding_providers and returns a formatted list with model notes.

You: What fields do I need to configure for Qdrant?

Claude calls list_vector_db_providers and returns the required and optional fields for each provider.

Supported providers

Embedding providers

Vector databases

LLM providers (for extraction + contextual retrieval)

Billing

• Credits are deducted from your scrapedatshi account after each successful API call

• Failed requests are not charged

• Every tool response includes credits_used and credits_remaining

• LLM, embedding, and vector DB costs are billed directly by your chosen providers — scrapedatshi only charges for scraping and orchestration

• Top up at scrapedatshi.com/portal/billing

Safety limits

To prevent runaway credit usage and client timeouts:

• crawl_site: defaults to 10 pages, maximum 200

• extract_crawl: defaults to 5 pages, maximum 50 per call

Claude will always confirm page limits with you before calling multi-page tools.

License

MIT — see LICENSE