A
licenseNot graded
qualityB
maintenanceProvides lightweight web crawling and scraping capabilities, enabling users to fetch pages as clean Markdown, recursively crawl domains, extract metadata, and perform search-and-crawl operations.
3
MIT
76,047 servers. Updated
Search Relevance ↓
Recent Usage ↓
Date Added ↓
Date Updated ↓
Weekly Downloads ↓
GitHub Stars ↓
Recent GitHub Stars ↓
Matching MCP tools:
Matching MCP Connectors:
com.create-web-page
app.apick
com.jojapi
io.github.foomworks
"Understanding Web Crawlers" matching MCP servers:
- AlicenseNot gradedqualityDmaintenanceEnables web search via DuckDuckGo and web content fetching through the Model Context Protocol, allowing AI assistants to search and retrieve web page content.15MIT
- FlicenseNot gradedqualityCmaintenanceA minimal MCP server that crawls web pages using crawl4ai and returns content as markdown, deployable to Cloud Run.
- FlicenseNot gradedqualityDmaintenanceAn intelligent web crawling server that uses Cloudflare's headless browser to render dynamic pages and Workers AI to extract relevant links based on natural language queries. It enables AI assistants to search and filter website content while providing secure access through GitHub OAuth authentication.3
- FlicenseAqualityBmaintenanceA local MCP server that gives Claude Code web search capabilities using DuckDuckGo, enabling discovery of URLs from natural language queries.11
- AlicenseAqualityAmaintenanceA web-fetch MCP server for LLM agents that fetches pages with an escalation ladder through different engines, raising FetchBlocked instead of returning blocked content.21Apache 2.0
- AlicenseAqualityBmaintenanceA Model Context Protocol server that provides real-time web search capabilities to AI assistants through pluggable search providers, currently integrated with the Brave Search API.515MIT
- AlicenseAqualityAmaintenanceA fast, dependency-light MCP server that converts web pages into clean Markdown, structured metadata, and classified links for language models.339MIT
- FlicenseAqualityCmaintenanceEnables MCP-compatible agents to perform web searches via the agent-web-search engine, returning ranked results with sources.1
- AlicenseAqualityAmaintenanceZero-dependency MCP server that wraps OpenAI Codex's standalone search endpoint into codex_web_search and codex_web_research tools for Claude Code, enabling real-time web search and multi-step deep research independent of the underlying model.21MIT
- AlicenseAqualityAmaintenanceEnables web search and web fetch operations using Ollama's hosted APIs, allowing MCP clients to search the web and retrieve page content.2MIT
- FlicenseAqualityDmaintenanceProvides local LLMs with web search and page fetching capabilities via MCP, with a focus on OWASP security best practices.2
- AlicenseAqualityBmaintenanceThe MCP for the Web Speed Agent SDK that enables post-auth agents.173GPL 3.0
- AlicenseAqualityDmaintenanceEnables private web search and webpage content extraction using a local SearxNG instance, prioritizing user privacy and autonomy.22MIT
- AlicenseAqualityAmaintenanceA DOM-first, vision-second MCP server for webpage analysis, combining structural DOM extraction with MiniMax vision to provide accurate, context-rich page understanding for AI coding assistants.4Apache 2.0
- AlicenseAqualityBmaintenanceA self-hosted MCP server for web search with multi-engine support, combining search, URL fetching, and deep research along with proxy routing and deduplication.3MIT
- AlicenseAqualityAmaintenanceAn MCP server that enables LLMs to drive a Chromium browser for web scraping, collecting structured data, and exporting to CSV.1530MIT
- FlicenseAqualityBmaintenanceMCP server that persists web fetch and search results as timestamped JSONL, enabling long-term archival and cross-domain search via unified-history-mcp.5
- AlicenseAqualityCmaintenanceMCP server providing free, IP-direct tools for web search, crawling, anti-detect browsing, and structured data from Wikipedia, Wikidata, GitHub, and YouTube.261MIT
- AlicenseAqualityCmaintenanceMCP server for web page fetching (converting to Markdown/text with automatic fallback between Tavily and Firecrawl) and web search via Tavily.2MIT