"Web crawler for sequential URL content retrieval" matching MCP servers:

• Web Crawler MCP Server

  dhannusch

  -

  security

  F

  license

  -

  quality

  An intelligent web crawling server that uses Cloudflare's headless browser to render dynamic pages and Workers AI to extract relevant links based on natural language queries. It enables AI assistants to search and filter website content while providing secure access through GitHub OAuth authentication.

  Last updated 2025-08-06

  3

• Web URL Reader MCP Server

  clsferguson

  -

  security

  F

  license

  -

  quality

  Enables fetching and reading content from web URLs through a configurable proxy prefix. Uses curl to retrieve webpage content and return it as text for analysis or processing.

  Last updated 2025-10-17

• SEO Crawler MCP

  houtini-ai

  A

  security

  A

  license

  -

  quality

  A comprehensive website crawler and SEO analyzer that stores site data in a local SQLite database for AI-driven auditing. It enables users to detect technical SEO issues, broken links, and security vulnerabilities through natural language queries or terminal commands.

  Last updated 2026-03-22

  4

  10

  11

  Apache 2.0

• Site Crawler MCP

  AndacGuven

  -

  security

  A

  license

  -

  quality

  An MCP server for crawling websites and extracting comprehensive data including images, SEO metadata, security headers, and business intelligence. It features twelve distinct extraction modes to perform detailed audits for e-commerce sites and general web analysis.

  Last updated 2025-07-23

  5

  MIT

• sequential-thinking-mcp

  philogicae

  A

  security

  A

  license

  -

  quality

  Simple sequential thinking MCP in python

  Last updated 2025-11-28

  1

  4

  MIT

• content-core

  lfnovo

  A

  security

  A

  license

  -

  quality

  Extract content from URLs, documents, videos, and audio files using intelligent auto-engine selection. Supports web pages, PDFs, Word docs, YouTube transcripts, and more with structured JSON responses.

  Last updated 2026-04-14

  1

  147

  MIT

• Documentation Retrieval & Web

  AIwithhassan

  A

  security

  F

  license

  -

  quality

  Enables retrieval and cleaning of official documentation content for popular AI/Python libraries (uv, langchain, openai, llama-index) through web scraping and LLM-powered content extraction. Uses Serper API for search and Groq API to clean HTML into readable text with source attribution.

  Last updated 2025-10-06

  1

  1

• Sequential Questioning MCP

  bitgeese

  -

  security

  F

  license

  -

  quality

  A specialized server that enables LLMs to gather specific information through sequential questioning, implementing the MCP standard for seamless integration with LLM clients.

  Last updated 2025-04-28

  1

• MCP-RSS-Crawler

  mshk

  -

  security

  F

  license

  -

  quality

  An MCP server that fetches RSS feeds and shares them with LLMs, allowing AI assistants to access and present the latest news and articles from configured feeds.

  Last updated 2025-03-31

  18

• mcp-shoten-url

  hatimitu76

  -

  security

  A

  license

  -

  quality

  Enables creation of shortened URLs using the x.gd URL shortening service through its API.

  Last updated 2025-06-29

  3

  MIT

• MCP URL Fetcher

  nathanonn

  A

  security

  F

  license

  -

  quality

  A Model Context Protocol server that enables LLMs to fetch and process web content in multiple formats (HTML, JSON, Markdown, text) with automatic format detection.

  Last updated 2025-03-30

  5

  5

• URL Fetcher MCP Server

  lucoo01

  -

  security

  F

  license

  -

  quality

  Enables AI assistants to fetch and analyze web content from URLs through MCP protocol. Supports batch processing, content extraction, summarization, and metadata extraction with intelligent filtering of ads and navigation elements.

  Last updated 2025-10-11

• URL-Context-MCP MCP

  nanameru

  A

  security

  F

  license

  -

  quality

  The URL-Context-MCP MCP Server provides a tool to analyze and summarize the content of URLs using Google Gemini's URL Context capability via the Gemini API. Now also supports optional grounding with Google Search alongside URL Context. The server is designed to follow prompt-only orchestration: con

  Last updated 2025-08-21

  2

  5

  7

• Retrieval-Augmented Thinking

  stat-guy

  A

  security

  A

  license

  -

  quality

  Enhances AI model capabilities with structured, retrieval-augmented thinking processes that enable dynamic thought chains, parallel exploration paths, and recursive refinement cycles for improved reasoning.

  Last updated 2025-08-13

  22

  MIT

• COAIA Sequential Thinking

  miadisabelle

  -

  security

  A

  license

  -

  quality

  Facilitates structured creative thinking through sequential thought processing, helping users shift from reactive problem-solving to proactive outcome creation using structural tension analysis and stage-based thinking workflows.

  Last updated 2025-11-11

  1

  MIT

• Sequential-Thinking

  xiaobenyang-com

  A

  security

  A

  license

  -

  quality

  A MCP server that implements sequential thinking protocols, provides structured problem-solving methods, decomposes complex problems into manageable steps, and supports iterative optimization and alternative reasoning paths.

  Last updated 2026-01-09

  1

  2

  Apache 2.0

• URL Fetch MCP

  aelaguiz

  A

  security

  A

  license

  -

  quality

  A Model Context Protocol (MCP) server that enables Claude or other LLMs to fetch content from URLs, supporting HTML, JSON, text, and images with configurable request parameters.

  Last updated 2025-03-19

  3

  2

  MIT

• content-optimizer-mcp

  sharozdawa

  A

  security

  A

  license

  A

  quality

  SERP-based content scoring and optimization with 7 SEO categories.

  Last updated 2026-03-22

  4

  6

  MIT

• URL Text Fetcher MCP Server

  alexandru2882

  A

  security

  F

  license

  -

  quality

  Enables fetching visible text content and extracting all links from web pages through URL requests. Designed specifically for LM Studio integration to provide web scraping capabilities.

  Last updated 2025-08-12

  2

  1

• Sequential Thinking MCP Server

  arben-adm

  A

  security

  A

  license

  -

  quality

  This server facilitates structured problem-solving by breaking down complex issues into sequential steps, supporting revisions, and enabling multiple solution paths through full MCP integration.

  Last updated 2026-03-06

  873

  MIT

• Web Content MCP Server

  amotivv

  A

  security

  A

  license

  -

  quality

  A server that leverages Cloudflare Browser Rendering to extract and process web content for use as context in LLMs, offering tools for fetching pages, searching documentation, extracting structured content, and summarizing content.

  Last updated 2025-03-12

  4

  10

  MIT

• url-download-mcp

  dmitryglhf

  A

  security

  A

  license

  -

  quality

  A Model Context Protocol (MCP) server that enables AI assistants to download files from URLs to the local filesystem.

  Last updated 2025-10-24

  2

  1

  MIT

• Prismic Content MCP

  rahulpowar

  A

  security

  F

  license

  A

  quality

  Enables interaction with Prismic through tools for reading documents, managing media assets, and performing content migrations or updates. Supports the Prismic Content, Asset, Migration, and Custom Types APIs with secure write operations and multi-transport support.

  Last updated 2026-03-05

  18

• Strapi Content MCP

  PaulBratslavsky

  -

  security

  A

  license

  -

  quality

  Enables AI assistants to interact directly with Strapi v5 CMS content through full CRUD operations, media uploads, and content type exploration using Strapi's Document Service API.

  Last updated 2026-01-05

  160

  1

  MIT

• mcp-web-calc

  tickernelz

  A

  security

  A

  license

  -

  quality

  An MCP server that enables web searching, URL content extraction, and summarization without requiring API keys. It also provides advanced mathematical evaluation and multi-language Wikipedia summary retrieval tools.

  Last updated 2026-03-11

  5

  271

  3

  MIT

• MCP Content Credentials Server

  noga7

  A

  security

  A

  license

  -

  quality

  Enables reading and verifying C2PA Content Credentials from images and videos through both embedded manifests and invisible watermarks. Supports direct filesystem access, URL checking, and provides structured provenance data including creator identity, AI generation info, and content authenticity.

  Last updated 2025-12-12

  2

  MIT

• mcp-web

  TsFreddie

  -

  security

  A

  license

  -

  quality

  Enables AI agents to search DuckDuckGo and fetch web content with human-assisted CAPTCHA handling, designed for local use where humans can intervene when bot detection occurs.

  Last updated 2025-11-01

  7

  MIT

• Enhanced Sequential Thinking

  new69toss

  -

  security

  -

  license

  -

  quality

  Enables structured problem-solving through sequential thinking stages with persistent storage and analysis. Helps break down complex problems into manageable cognitive steps while tracking progress and generating summaries of the entire thought process.

  Last updated 2025-09-05

  MIT

• MedAdapt Content Server

  ryoureddy

  -

  security

  A

  license

  -

  quality

  A specialized Model Context Protocol server that enhances AI-assisted medical learning by connecting Claude Desktop to PubMed, NCBI Bookshelf, and user documents for searching, retrieving, and analyzing medical education content.

  Last updated 2025-03-28

  6

  MIT

• content-distribution-mcp

  gomessoaresemmanuel-cpu

  A

  security

  A

  license

  A

  quality

  Multi-platform content distribution — draft posts, repurpose content, generate carousels, schedule, analyze performance, create threads. 7 MCP tools.

  Last updated 2026-03-26

  7

  16

  MIT

• Content Fetcher MCP

  EbonyLouis

  A

  security

  F

  license

  -

  quality

  Fetches and tracks content from YouTube channels, RSS feeds, and GitHub releases with persistence to identify new items across sessions.

  Last updated 2025-11-20

  6

  3

  1

• WordPress Content MCP

  Akungapaul

  -

  security

  F

  license

  -

  quality

  Enables management of WordPress posts and pages including creating, updating, deleting, duplicating, scheduling, and bulk editing content through the WordPress REST API.

  Last updated 2025-11-28

• Web Search MCP

  guhcostan

  A

  security

  A

  license

  -

  quality

  Enables web searching via DuckDuckGo and extracting readable content from any URL using Mozilla Readability, providing web context similar to Cursor's built-in functionality.

  Last updated 2026-03-07

  2

  244

  3

  MIT

• URL Shortener MCP

  Talismanic

  A

  security

  A

  license

  -

  quality

  Provides a simple tool to shorten URLs using the CleanURI API, designed to run as a FastMCP server that can be integrated with agent or tool-based systems.

  Last updated 2025-07-06

  1

  4

  Apache 2.0

• Sequential Thinking MCP Server

  LangGPT

  A

  security

  F

  license

  -

  quality

  An MCP server that enables Claude to break down complex problems into manageable steps with support for revision and branching, facilitating dynamic and reflective problem-solving through a structured thinking process.

  Last updated 2025-07-13

  1

  3

• Sequential Thinking MCP Server

  camilovelezr

  -

  security

  A

  license

  -

  quality

  Enables AI assistants to work through complex problems step-by-step with dynamic thought processes, allowing for revision of previous steps, exploration of alternative approaches, and flexible planning as understanding deepens.

  Last updated 2025-07-02

  6

  MIT

• GitHub URL MCP Server

  kongyo2

  A

  security

  A

  license

  -

  quality

  An MCP (Model Context Protocol) server for handling GitHub URLs with validation and parsing capabilities. This server provides tools to convert between GitHub repository information and URLs, with intelligent validation to distinguish between public repositories, private repositories, and non-exist

  Last updated 2025-08-06

  2

  1

  MIT

• Web-Scout

  UpperMoon0

  -

  security

  F

  license

  -

  quality

  Enables AI-powered web searches using DuckDuckGo with automatic summarization via Google's Gemini AI. Supports both summary and detailed analysis modes for search results.

  Last updated 2026-02-11

• Sequential-Thinking-Tools

  xiaobenyang-com

  -

  security

  A

  license

  -

  quality

  A sequential thinking tool based on the MCP protocol, used to break down complex problems into manageable steps and provide intelligent tool recommendations for each step.

  Last updated 2026-01-09

  2

  Apache 2.0

• MCP Web Tools Server

  JoaoPedroLanca

  A

  security

  F

  license

  -

  quality

  Provides web connectivity tools for searching the web via DuckDuckGo or SerpAPI, fetching URL content, and extracting readable text from web pages.

  Last updated 2025-11-02

  3

• Sequential Thinking MCP Server

  badbboy

  -

  security

  A

  license

  -

  quality

  Facilitates structured, progressive thinking through defined cognitive stages (Problem Definition, Research, Analysis, Synthesis, Conclusion), enabling users to break down complex problems, track thought progression, and generate summaries of their thinking process.

  Last updated 2025-07-02

  MIT

• MCP Enhanced Data Retrieval

  kalpalathika

  -

  security

  F

  license

  -

  quality

  Enables AI applications to access and contextualize organizational knowledge sources including GitHub repositories and internal documentation through standardized MCP protocol integration. Features OAuth 2.1 authentication, vector-based semantic search, and optimized context chunking for enterprise development workflows.

  Last updated 2025-10-21

• Content Creation MCP Server

  preangelleo

  -

  security

  -

  license

  -

  quality

  Enables AI-powered Ghost CMS blog management through Claude Desktop with content generation, image creation, and complete post management. Integrates Google Gemini for content generation and Flux/Imagen for feature images with GitHub OAuth authentication.

  Last updated 2025-08-28

  1

  MIT

• Content Manager MCP Server

  Talljack

  -

  security

  F

  license

  -

  quality

  Enables comprehensive content management through Markdown processing, HTML rendering, intelligent fuzzy search, and document analysis. Supports frontmatter parsing, tag-based filtering, table of contents generation, and directory statistics for efficient content organization and discovery.

  Last updated 2025-09-19

  5

  3

• Content Plan Builder

  smcdonnell7

  A

  security

  F

  license

  -

  quality

  An MCP server that uses AI to generate structured Asana project plans from diverse sources like text, PDFs, and meeting transcripts. It enables users to preview, generate, and directly sync detailed task lists, subtasks, and dependencies into Asana.

  Last updated 2026-02-06

  1

• Web Search MCP Server

  mrkrsl

  -

  security

  A

  license

  -

  quality

  Enables comprehensive web searching and content extraction using multiple search engines (Bing, Brave, DuckDuckGo) without API keys. Provides tools for full web searches with content extraction, quick search summaries, and single webpage content retrieval.

  Last updated 2025-08-08

  718

  MIT

• Sequential Thinking Multi

  FradSer

  A

  security

  F

  license

  -

  quality

  An advanced MCP server that implements sophisticated sequential thinking using a coordinated team of specialized AI agents (Planner, Researcher, Analyzer, Critic, Synthesizer) to deeply analyze problems and provide high-quality, structured reasoning.

  Last updated 2026-04-14

  1

  296

• web-scrapper-stdio

  JustAzul

  -

  security

  A

  license

  -

  quality

  A headless web scraping server that extracts main content from web pages into Markdown, text, or HTML for AI and automation integration. It features per-domain rate limiting and robust error handling using Playwright and BeautifulSoup.

  Last updated 2026-04-12

  MIT

• Web Search MCP Server

  vishalkg

  -

  security

  F

  license

  -

  quality

  Enables web search across multiple search engines (DuckDuckGo, Bing, Startpage) with parallel execution and result deduplication. Also provides web page content extraction capabilities.

  Last updated 2026-04-09

  2

• AI Content Detector

  BACH-AI-Tools

  A

  security

  A

  license

  -

  quality

  Enables detection of AI-generated content from ChatGPT, GPT-4, and Gemini models through the BACH AI Content Detector API.

  Last updated 2025-12-07

  1

  MIT