Skip to main content
Glama
@aegis-ai/mcp-scraper

@aegis-ai/mcp-scraper

@aegis-ai/mcp-scraper

MCP server for intelligent web scraping. Supports static pages, JavaScript-rendered SPAs (via Playwright), and natural language data extraction.

Install

# Add to Claude Desktop config (~/.claude/claude_desktop_config.json):
{
  "mcpServers": {
    "scraper": {
      "command": "bunx",
      "args": ["@aegis-ai/mcp-scraper"]
    }
  }
}

# Or run directly:
bunx @aegis-ai/mcp-scraper

Related MCP server: Crawl4AI MCP Server

Tools

scrape

Fetch a web page and return clean text content.

{
  "url": "https://news.ycombinator.com",
  "js": false,
  "selector": ".titleline",
  "headers": { "Cookie": "session=abc" }
}

extract

Scrape a page and extract structured data using natural language.

{
  "url": "https://example.com/products",
  "prompt": "Get all product names, prices, and ratings",
  "js": true
}

batch_scrape

Scrape multiple URLs with rate limiting (1s between requests).

{
  "urls": ["https://example.com/page1", "https://example.com/page2"],
  "js": false
}

Features

  • Static page fetching (fast, no browser needed)

  • JavaScript rendering via Playwright (for SPAs)

  • CSS selector targeting

  • Custom headers support

  • Automatic HTML stripping

  • Content truncation to avoid context overflow

  • Rate-limited batch scraping

License

MIT - AEGIS AI Cooperative

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    Related MCP Connectors

    Related MCP Servers

    • A
      license
      Not graded
      quality
      D
      maintenance
      Enables web scraping and document processing with JavaScript execution, anti-detection measures, batch processing, and structured data extraction. Supports multiple formats including markdown, HTML, screenshots, and handles PDFs with OCR capabilities.
      44 PyPI
      4
      MIT
    • F
      license
      Not graded
      quality
      D
      maintenance
      Enables advanced web crawling and content extraction with JavaScript support, AI-powered analysis, PDF/Office document processing, YouTube transcript extraction, Google search integration, and multi-format data export capabilities.
      2
      -
    • A
      license
      A
      quality
      A
      maintenance
      Enables web scraping, structured data extraction, and screenshot capture with automatic anti-bot bypass, supporting JavaScript rendering, proxy rotation, and tiered pricing.
      26
      2,618 npm
      2
      MIT