Skip to main content
Glama
Sign In
HeartLoveLung

Scrapegraphai_MCP

by HeartLoveLung
OverviewSchemaSourceRelated ServersDiscussionsAdmin
Python
Local

Scrapegraphai MCP

A local stdio MCP server for structured web scraping with ScrapeGraphAI and an OpenAI-compatible model API.

Tools

  • list_models: List models from the configured OpenAI-compatible API.

  • scrape_url: Scrape a URL with ScrapeGraphAI SmartScraperGraph and return extracted results.

Related MCP server: crawl4ai-mcp-server

Requirements

  • Python 3.12+

  • Playwright Chromium

  • An OpenAI-compatible API key and base URL

Install

git clone https://github.com/HeartLoveLung/Scrapegraphai_MCP.git
cd Scrapegraphai_MCP

python -m venv .venv
.\.venv\Scripts\pip install -r requirements.txt
.\.venv\Scripts\playwright install chromium

You can also install dependencies manually:

.\.venv\Scripts\pip install scrapegraphai langchain-openai httpx requests python-dotenv playwright
.\.venv\Scripts\playwright install chromium

Environment

Copy the example file:

Copy-Item .env.example .env

Edit .env:

SCRAPEGRAPHAI_API_KEY=sk-your-api-key
SCRAPEGRAPHAI_BASE_URL=https://gpt.qt.cool/v1
SCRAPEGRAPHAI_MODEL=deepseek-v4-flash
SCRAPEGRAPHAI_VERIFY_SSL=false
SCRAPEGRAPHAI_MODEL_TOKENS=50000

Do not commit .env.

MCP Client Config

Copy mcp_config.example.json, replace the paths with your local repository path, and add it to an MCP-compatible client.

Windows example:

{
  "mcpServers": {
    "scrapegraphai-local": {
      "command": "C:\\path\\to\\Scrapegraphai_MCP\\.venv\\Scripts\\python.exe",
      "args": [
        "C:\\path\\to\\Scrapegraphai_MCP\\tools\\scrapegraphai_mcp_server.py"
      ],
      "env": {
        "SCRAPEGRAPHAI_API_KEY": "sk-your-api-key",
        "SCRAPEGRAPHAI_BASE_URL": "https://gpt.qt.cool/v1",
        "SCRAPEGRAPHAI_MODEL": "deepseek-v4-flash",
        "SCRAPEGRAPHAI_VERIFY_SSL": "false",
        "SCRAPEGRAPHAI_MODEL_TOKENS": "50000"
      }
    }
  }
}

You can also omit env from the MCP config and use a local .env file in the repository root.

Example Tool Calls

list_models

{
  "name": "list_models",
  "arguments": {
    "timeout": 10
  }
}

scrape_url

{
  "name": "scrape_url",
  "arguments": {
    "url": "https://gpt.qt.cool/checkin",
    "prompt": "Extract the main visible content and return JSON.",
    "model": "deepseek-v4-flash",
    "timeout": 90,
    "headless": true,
    "html_mode": true
  }
}

Local Smoke Test

Check that the server imports and exposes tools:

.\.venv\Scripts\python.exe -c "import importlib.util; spec=importlib.util.spec_from_file_location('m', r'tools\scrapegraphai_mcp_server.py'); m=importlib.util.module_from_spec(spec); spec.loader.exec_module(m); print(sorted(m.TOOLS.keys()))"

Expected output:

['list_models', 'scrape_url']

Notes

  • scrape_url starts a Playwright browser. Some sandboxed environments must allow Chromium process launch.

  • If the model API returns 503 model_unavailable, switch to another available model or retry later.

  • Replace all example API keys with your own key.

This server cannot be deployed

One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

Maintenance

ActivityInactive
ResponsivenessNo issues

Resources

Related MCP Connectors

Related MCP Servers

  • ISIS MCP

    alucardeht
    A
    license
    B
    quality
    D
    maintenance
    A local web scraping MCP server with RAG capabilities that provides intelligent web search, content extraction, and screenshot tools without requiring API keys.
    4
    11 npm
    MIT

  • crawl4ai-mcp-server

    sadiuysal
    A
    license
    Not graded
    quality
    F
    maintenance
    A lightweight MCP server that exposes Crawl4AI web scraping and crawling capabilities as tools for AI agents, enabling single-page scraping and multi-page crawling with adaptive stopping.
    117
    MIT

  • crawl4ai-mcp-server

    amienbou121
    -
    license
    Not graded
    quality
    C
    maintenance
    Self-hosted MCP server that provides web scraping and crawling tools, integrating seamlessly with AI frameworks like OpenAI Agents SDK, Cursor, and Claude Code.
    4
    -