Skip to main content
Glama
123,413 servers. Updated
20 Best Web Scraping MCP Servers: compared and ranked, November 2026Ranked from 2,147 matching servers on stars, growth, downloads and maintenance. Updated .

Matching MCP tools:

Matching MCP Connectors:

"A web crawler for collecting and indexing website content" matching MCP servers:

GET /v1/servers – MCP directory API reference
  • A
    license
    Not graded
    quality
    C
    maintenance
    Provides lightweight web crawling and scraping capabilities, enabling users to fetch pages as clean Markdown, recursively crawl domains, extract metadata, and perform search-and-crawl operations.
    13 npm
    MIT
  • F
    license
    Not graded
    quality
    D
    maintenance
    An intelligent web crawling server that uses Cloudflare's headless browser to render dynamic pages and Workers AI to extract relevant links based on natural language queries. It enables AI assistants to search and filter website content while providing secure access through GitHub OAuth authentication.
    3
    -
  • A
    license
    A
    quality
    D
    maintenance
    Web-crawler MCP server that fetches web pages with static parsing or headless browser fallback, enabling Claude to fetch, extract links, crawl sites, and select content via CSS selectors.
    4
    ISC
  • A
    license
    Not graded
    quality
    C
    maintenance
    Enables AI assistants to check Google Search Console and Bing Webmaster Tools indexing status, diagnose why pages are not indexed, inspect URLs, review search performance, submit URLs to Bing and IndexNow, and resubmit sitemaps.
    MIT
  • A
    license
    Not graded
    quality
    C
    maintenance
    Enables AI assistants such as Claude, ChatGPT, Cursor, and Claude Code to run 25+ specialized SEO checks on any URL, generate XML sitemaps and robots.txt files, extract brand design elements, and audit technical SEO, content quality, mobile responsiveness, and security.
    MIT
  • A
    license
    A
    quality
    Not graded
    maintenance
    Enables AI agents to extract clean, structured web content (articles, tables, links, visual layouts) optimized for LLM token efficiency, with fast response times and optional JavaScript support.
    5
    72 npm
    MIT
  • A
    license
    Not graded
    quality
    B
    maintenance
    Fetches web pages politely (it obeys robots.txt and sends one request at a time to each site), keeps them in a local store, and lets the client search them and verify quotes. Nine tools for Claude and other MCP clients.
    MIT
  • A
    license
    Not graded
    quality
    C
    maintenance
    Enables AI agents to read any web page as clean Markdown and inspect its metadata, structure and technology stack, plus check redirects, TLS certificates, security headers, robots.txt, sitemaps, feeds, domains, SEO, broken links, email authentication (SPF/DMARC/DKIM) and SSO (SAML/OIDC). It also provides developer utilities such as cron, regex, color, Unicode and time-zone tools along with Hyperliquid perpetuals market data, all hosted remotely on Apify with pay-per-call pricing.
    MIT
  • A
    license
    Not graded
    quality
    D
    maintenance
    A specialized web crawler and scraper for the US Patent and Trademark Office (USPTO) website, built with Crawl4AI and MCP integration, enabling patent and trademark search, extraction, and status checking.
    4
    MIT
  • A
    license
    A
    quality
    B
    maintenance
    Enables crawling sites from start URLs and retrieving internal and external links, file download links, HTML tables as rows, and lists, with controls for crawl depth, page limits, sitemaps, robots.txt, and proxy mode.
    1
    728 npm
    MIT
  • F
    license
    A
    quality
    B
    maintenance
    Enables comprehensive website SEO analysis including crawling, on-page audits, site structure visualization, and content topic classification with taxonomy mapping.
    7
    1
    -
  • A
    license
    A
    quality
    C
    maintenance
    Enables reading and sending messages in Google Messages for web through a paired Chrome tab using chromefile, with tools for status, listing conversations, reading message history, and sending texts.
    4
    MIT
  • A
    license
    A
    quality
    B
    maintenance
    Extract content from URLs, documents, videos, and audio files using intelligent auto-engine selection. Supports web pages, PDFs, Word docs, YouTube transcripts, and more with structured JSON responses.
    2
    185
    MIT
  • A
    license
    A
    quality
    F
    maintenance
    A comprehensive website crawler and SEO analyzer that stores site data in a local SQLite database for AI-driven auditing. It enables users to detect technical SEO issues, broken links, and security vulnerabilities through natural language queries or terminal commands.
    4
    61 npm
    16
    Apache 2.0
  • A
    license
    Not graded
    quality
    D
    maintenance
    An MCP server for crawling websites and extracting comprehensive data including images, SEO metadata, security headers, and business intelligence. It features twelve distinct extraction modes to perform detailed audits for e-commerce sites and general web analysis.
    6
    MIT
  • A
    license
    Not graded
    quality
    C
    maintenance
    Lets a content team edit a GitHub-hosted website's Markdown pages and blog posts through Claude, with each change validated and committed to a staging branch. Editors can preview, publish live, or discard changes, while code checks block edits that would break the site.
    13 npm
    MIT
  • A
    license
    Not graded
    quality
    B
    maintenance
    Enables AI agents to call tools through a security gateway that intercepts requests, enforces deterministic policy and risk checks, blocks or escalates high-risk actions for human approval, and keeps tool credentials and audit logs protected.
    1
    MIT
  • A
    license
    Not graded
    quality
    B
    maintenance
    Enables users to mint website tasks into tested, deterministic MCP tools and replay them without a model, using Playwright browser tools for public sites and page tools that run in the user’s own Chrome tab for signed-in sites.
    356 npm
    MIT
  • A
    license
    Not graded
    quality
    C
    maintenance
    Enables AI agents to analyze a piece of content across 11 dimensions and receive ranked, scored recommendations among six cross-media output formats (PPT, infographic, comic, short video script, newsletter, reusable skill), then generate dispatch prompts for the chosen format. It also exposes the format catalog and profile schema as read-only tools, supporting batch routing of an entire content library.
    1
    MIT
  • A
    license
    Not graded
    quality
    B
    maintenance
    Enables LLM clients to index Java, Python, TypeScript, and JavaScript projects into a persistent property graph and query callers, callees, execution paths, virtual dispatch, aspect advice, framework wiring, annotations, and external dependencies through MCP tools.
    1,428 npm
    MIT
  • A
    license
    A
    quality
    C
    maintenance
    AutomateLab Content Distribution publishes content across DEV.to, Hashnode, GitHub Discussions, Reddit and Bluesky, with browser workflows for LinkedIn, Medium and X. Tracks publishing state and adapts content per platform. Product and documentation: https://automatelab.tech/products/mcp/content-distribution-mcp/
    8
    74 npm
    4
    MIT
  • A
    license
    A
    quality
    D
    maintenance
    Provides AI agents with structured short-form content mechanics including hooks, script structures, retention strategies, and CTAs, along with auditing tools to avoid common posting failures.
    7
    48 npm
    1
    MIT
  • A
    license
    A
    quality
    C
    maintenance
    Fast, token-efficient web content extraction tool that converts websites to clean Markdown for AI agents, featuring smart caching, content extraction with Mozilla Readability, and polite crawling capabilities.
    1
    1,450 npm
    163
    MIT
  • A
    license
    A
    quality
    C
    maintenance
    Enables auditing any public website, returning a scored plain-English report that flags issues costing customers, covering speed, phone experience, search visibility, contact options, writing quality, and modernity.
    1
    MIT
  • F
    license
    A
    quality
    D
    maintenance
    Enables interaction with Prismic through tools for reading documents, managing media assets, and performing content migrations or updates. Supports the Prismic Content, Asset, Migration, and Custom Types APIs with secure write operations and multi-transport support.
    18
    -
  • A
    license
    A
    quality
    B
    maintenance
    Enables users to look up a Douyin creator by name, public ID, or profile URL and receive the latest video link, like/comment/favorite/share counts, a full timestamped transcript with an AI-reviewed correction and change log, and a content overview. Collection, media download, and local speech transcription are orchestrated as stdio MCP tools so Codex, WorkBuddy, and other compatible clients can run creator research and scheduled checks without extra API keys.
    7
    MIT
  • A
    license
    A
    quality
    C
    maintenance
    Enables local discovery of research topics, interviews users to uncover their unique perspective, and turns insights into content ideas and short-form Reel scripts.
    10
    MIT
  • A
    license
    A
    quality
    D
    maintenance
    Enables site-wide crawling of internal links with BFS traversal, time/title/URL filtering, and page operations like summarization and link clicking, with results persisted to SQLite.
    6
    MIT
  • A
    license
    A
    quality
    B
    maintenance
    This server performs a comprehensive analysis of a website by fetching its HTML and scoring it across five categories: performance, SEO, design, functionality, and technology. It enables users to identify technical strengths and weaknesses in a webpage's structure and health.
    2
    MIT
  • A
    license
    A
    quality
    C
    maintenance
    Enables MCP clients like Claude to query Chinese A-share market data, including K-line charts with forward/backward adjustment, limit-up and limit-down pools, per-stock fund flows, index quotes, and stock news, with automatic symbol normalization and retry handling for throttled data sources.
    6
    MIT
  • A
    license
    A
    quality
    C
    maintenance
    Gives AI agents ten website and domain research tools, including full domain reports, readable page extraction, DNS lookups, feed and sitemap parsing, link and metadata extraction, site checks, and redirect tracing. Calls are paid per use in USDC on Base via x402 from a configured wallet, so no account, API key, or subscription is required.
    10
    84 npm
    MIT
  • A
    license
    B
    quality
    C
    maintenance
    Provides A-share (Chinese stock market) quantitative analysis through tools for stock screening, northbound capital flow tracking, dragon-tiger list analysis, margin trading, sector analysis, technical indicators, IPO info, and limit-up/down statistics using akshare data.
    11
    2
    MIT
  • A
    license
    B
    quality
    D
    maintenance
    An MCP server that enables agents to safely interact with CMS content, with deny-by-default write policies, type/field allowlists, and structured audit trails.
    5
    MIT
  • A
    license
    B
    quality
    D
    maintenance
    A server that leverages Cloudflare Browser Rendering to extract and process web content for use as context in LLMs, offering tools for fetching pages, searching documentation, extracting structured content, and summarizing content.
    4
    12
    MIT
  • A
    license
    B
    quality
    C
    maintenance
    Enables AI agents to read and edit a Django site's content over its API, creating tools at startup from whatever resources the site registers so no code changes are needed. Works with any site running django-agent-content, with HTTPS-only API key authentication and no delete operations.
    1
    100 PyPI
    3
    MIT