Skip to main content
Glama
123,438 servers. Updated
20 Best Web Scraping MCP Servers: compared and ranked, November 2026Ranked from 2,147 matching servers on stars, growth, downloads and maintenance. Updated .

Matching MCP tools:

Matching MCP Connectors:

"A web crawler for collecting and indexing website content" matching MCP servers:

GET /v1/servers – MCP directory API reference
  • A
    license
    A
    quality
    D
    maintenance
    Web-crawler MCP server that fetches web pages with static parsing or headless browser fallback, enabling Claude to fetch, extract links, crawl sites, and select content via CSS selectors.
    4
    ISC
  • A
    license
    A
    quality
    Not graded
    maintenance
    Enables AI agents to extract clean, structured web content (articles, tables, links, visual layouts) optimized for LLM token efficiency, with fast response times and optional JavaScript support.
    5
    72 npm
    MIT
  • A
    license
    A
    quality
    B
    maintenance
    Enables crawling sites from start URLs and retrieving internal and external links, file download links, HTML tables as rows, and lists, with controls for crawl depth, page limits, sitemaps, robots.txt, and proxy mode.
    1
    728 npm
    MIT
  • A
    license
    A
    quality
    C
    maintenance
    Enables auditing any public website, returning a scored plain-English report that flags issues costing customers, covering speed, phone experience, search visibility, contact options, writing quality, and modernity.
    1
    MIT
  • F
    license
    A
    quality
    B
    maintenance
    Enables comprehensive website SEO analysis including crawling, on-page audits, site structure visualization, and content topic classification with taxonomy mapping.
    7
    1
    -
  • A
    license
    A
    quality
    C
    maintenance
    Enables reading and sending messages in Google Messages for web through a paired Chrome tab using chromefile, with tools for status, listing conversations, reading message history, and sending texts.
    4
    MIT
  • A
    license
    A
    quality
    B
    maintenance
    Extract content from URLs, documents, videos, and audio files using intelligent auto-engine selection. Supports web pages, PDFs, Word docs, YouTube transcripts, and more with structured JSON responses.
    2
    185
    MIT
  • A
    license
    A
    quality
    F
    maintenance
    A comprehensive website crawler and SEO analyzer that stores site data in a local SQLite database for AI-driven auditing. It enables users to detect technical SEO issues, broken links, and security vulnerabilities through natural language queries or terminal commands.
    4
    61 npm
    16
    Apache 2.0
  • A
    license
    A
    quality
    C
    maintenance
    AutomateLab Content Distribution publishes content across DEV.to, Hashnode, GitHub Discussions, Reddit and Bluesky, with browser workflows for LinkedIn, Medium and X. Tracks publishing state and adapts content per platform. Product and documentation: https://automatelab.tech/products/mcp/content-distribution-mcp/
    8
    74 npm
    4
    MIT
  • A
    license
    A
    quality
    D
    maintenance
    Provides AI agents with structured short-form content mechanics including hooks, script structures, retention strategies, and CTAs, along with auditing tools to avoid common posting failures.
    7
    48 npm
    1
    MIT
  • A
    license
    A
    quality
    C
    maintenance
    Enables editorial content QA over the Model Context Protocol with four local tools that analyze readability, AI-sounding language, SEO on-page factors, and produce full reports where every finding includes an actionable fix.
    4
    1
    MIT
  • F
    license
    A
    quality
    D
    maintenance
    Enables interaction with Prismic through tools for reading documents, managing media assets, and performing content migrations or updates. Supports the Prismic Content, Asset, Migration, and Custom Types APIs with secure write operations and multi-transport support.
    18
    -
  • A
    license
    A
    quality
    B
    maintenance
    Enables users to look up a Douyin creator by name, public ID, or profile URL and receive the latest video link, like/comment/favorite/share counts, a full timestamped transcript with an AI-reviewed correction and change log, and a content overview. Collection, media download, and local speech transcription are orchestrated as stdio MCP tools so Codex, WorkBuddy, and other compatible clients can run creator research and scheduled checks without extra API keys.
    7
    MIT
  • A
    license
    A
    quality
    C
    maintenance
    Enables local discovery of research topics, interviews users to uncover their unique perspective, and turns insights into content ideas and short-form Reel scripts.
    10
    MIT
  • A
    license
    A
    quality
    D
    maintenance
    Enables site-wide crawling of internal links with BFS traversal, time/title/URL filtering, and page operations like summarization and link clicking, with results persisted to SQLite.
    6
    MIT
  • A
    license
    A
    quality
    B
    maintenance
    This server performs a comprehensive analysis of a website by fetching its HTML and scoring it across five categories: performance, SEO, design, functionality, and technology. It enables users to identify technical strengths and weaknesses in a webpage's structure and health.
    2
    MIT
  • A
    license
    A
    quality
    C
    maintenance
    Enables MCP clients like Claude to query Chinese A-share market data, including K-line charts with forward/backward adjustment, limit-up and limit-down pools, per-stock fund flows, index quotes, and stock news, with automatic symbol normalization and retry handling for throttled data sources.
    6
    MIT
  • A
    license
    A
    quality
    C
    maintenance
    Gives AI agents ten website and domain research tools, including full domain reports, readable page extraction, DNS lookups, feed and sitemap parsing, link and metadata extraction, site checks, and redirect tracing. Calls are paid per use in USDC on Base via x402 from a configured wallet, so no account, API key, or subscription is required.
    10
    84 npm
    MIT
  • A
    license
    B
    quality
    C
    maintenance
    Provides A-share (Chinese stock market) quantitative analysis through tools for stock screening, northbound capital flow tracking, dragon-tiger list analysis, margin trading, sector analysis, technical indicators, IPO info, and limit-up/down statistics using akshare data.
    11
    2
    MIT
  • A
    license
    B
    quality
    D
    maintenance
    An MCP server that enables agents to safely interact with CMS content, with deny-by-default write policies, type/field allowlists, and structured audit trails.
    5
    MIT
  • A
    license
    B
    quality
    C
    maintenance
    Enables AI agents to read and edit a Django site's content over its API, creating tools at startup from whatever resources the site registers so no code changes are needed. Works with any site running django-agent-content, with HTTPS-only API key authentication and no delete operations.
    1
    100 PyPI
    3
    MIT
  • A
    license
    B
    quality
    B
    maintenance
    Enables any MCP-capable agent to run a local content pipeline: capture an idea, attach sources, produce drafted newsletter and social copy, then fact-check citations, split X threads, and render newsletter HTML plus ready-to-post social packs. Keeps voice, sources, drafts, and calendar state in a local project folder with approval-gated sending and no LLM calls of its own.
    33
    MIT
  • A
    license
    C
    quality
    D
    maintenance
    MCP server for managing web3 content, fetching blockchain data, and analyzing content for web3 topics.
    25
    MIT
  • A
    license
    C
    quality
    C
    maintenance
    提供A股实时行情、历史K线、财务报表、技术指标等全方位数据查询的MCP服务器,支持盘中即时更新及批量查询。
    67
    29
    MIT
  • A
    license
    A
    quality
    C
    maintenance
    A web-fetch MCP server for LLM agents that fetches pages with an escalation ladder through different engines, raising FetchBlocked instead of returning blocked content.
    2
    3
    Apache 2.0
  • A
    license
    A
    quality
    A
    maintenance
    Enables AI agents to check a list of websites by running an Apify Actor through one tool, returning per-site tech stack, on-page SEO issues and score, robots.txt rules for AI crawlers, sitemap, llms.txt presence, and an AI-readiness score. Supports local stdio operation with an Apify token for tool calls.
    8
    MIT
  • A
    license
    A
    quality
    D
    maintenance
    MCP server exposing content contract validators (SEO.md, PRODUCTHUNT.md, LINKEDIN.md) as tools so AI agents can verify generated content against concrete rules without maintaining linters. It provides linting and JSON-LD export tools with config and content passed as parameters, not file paths.
    4
    MIT
  • A
    license
    A
    quality
    B
    maintenance
    Classifies company sites as B2B or B2C and as SaaS, online store, agency, manufacturer, wholesaler, distributor, or marketplace, while matching your own buyer phrases and returning the quoted text and page as evidence. Runs deterministically on Apify, giving one row per domain with grace-free status rows for unreachable, blocked, or empty sites.
    1
    128 npm
    MIT
  • A
    license
    A
    quality
    B
    maintenance
    Enables agents to search the live web and fetch URLs as clean Markdown, returning ranked search results and paginated page content through two lightweight tools.
    2
    23 npm
    MIT
  • A
    license
    A
    quality
    B
    maintenance
    Enables web search and web fetch operations using Ollama's hosted APIs, allowing MCP clients to search the web and retrieve page content.
    2
    34 PyPI
    MIT
  • A
    license
    A
    quality
    C
    maintenance
    Enables reading Slack channels, messages, and threads using the web interface via a persistent browser session, designed for guest users without API access. Does not send messages.
    5
    MIT
  • A
    license
    A
    quality
    B
    maintenance
    Provides AI-visibility scoring and site auditing capabilities for websites, enabling agents to check how sites appear in AI engines like ChatGPT and Perplexity, run full SEO/security audits, and monitor changes over time.
    20
    715 npm
    1
    MIT
  • A
    license
    A
    quality
    B
    maintenance
    Provides LLMs with real-time web search and content extraction capabilities, including text/news search, full-text URL reading, and targeted technical documentation search.
    3
    25
    MIT
  • A
    license
    A
    quality
    A
    maintenance
    An MCP server that enables AI agents to pause and request human approval or information via Slack, Telegram, or macOS dialogs before proceeding with actions.
    2
    15
    Apache 2.0
  • A
    license
    A
    quality
    B
    maintenance
    Enables reading web pages and PDFs as clean Markdown, extracting main content and letting agents query for relevant passages, outline sections, search in-page, and list links while respecting robots.txt and blocking private addresses.
    4
    1
    MIT
  • A
    license
    A
    quality
    B
    maintenance
    Enables extraction of structured item lists from public web pages using the user's own OpenAI, Anthropic, Gemini, or Groq key through an Apify actor, returning one row per item with validation flags.
    1
    698 npm
    MIT
  • A
    license
    A
    quality
    D
    maintenance
    An MCP server for searching and downloading books from Library Genesis, supporting EPUB, MOBI, PDF, and more through natural language queries.
    3
    8
    MIT
  • F
    license
    A
    quality
    B
    maintenance
    Enables live-web research by providing stateless browsing, link extraction, and page search capabilities via Playwright, allowing agents to fetch and analyze web content without session management.
    3
    -
  • F
    license
    A
    quality
    B
    maintenance
    Odysseus Web MCP is a standalone Model Context Protocol (MCP) server for safe public-web search and URL fetching.
    2
    -
  • A
    license
    A
    quality
    B
    maintenance
    MCP server that lets AI editors execute JavaScript, read console output, and simulate clicks/input on static web pages. Supports multiple browser pages and can be deployed remotely via HTTP transport.
    13
    6 npm
    1
    Apache 2.0