A
license-
qualityD
maintenanceMCP server that fetches raw HTML content from a given URL to provide web context to LLMs.
1
The Unlicense
73,464 servers. Updated
Search Relevance ↓
Recent Usage ↓
Date Added ↓
Date Updated ↓
Weekly Downloads ↓
GitHub Stars ↓
Recent GitHub Stars ↓
Matching MCP tools:
Matching MCP Connectors:
io.github.pipeworx-io
io.github.lazymac2x
io.github.eltociear
io.github.varvararatta
"Web crawler for sequential URL content retrieval" matching MCP servers:
- FlicenseBqualityDmaintenanceA Model Context Protocol server that analyzes webpage design structures, providing detailed layout, navigation, content, form, image, and link information from a given URL.16
- Flicense-qualityCmaintenanceA minimal MCP server that crawls web pages using crawl4ai and returns content as markdown, deployable to Cloud Run.
- Alicense-qualityCmaintenanceProvides lightweight web crawling and scraping capabilities, enabling users to fetch pages as clean Markdown, recursively crawl domains, extract metadata, and perform search-and-crawl operations.6MIT
- Alicense-qualityAmaintenanceAn open-source MCP server providing AI agents with neural web search via Exa and tiered web fetch (Exa, local browser, Firecrawl) as a drop-in replacement for built-in web tools, preserving provenance and guarding against SSRF.1MIT
- Flicense-qualityDmaintenanceAn intelligent web crawling server that uses Cloudflare's headless browser to render dynamic pages and Workers AI to extract relevant links based on natural language queries. It enables AI assistants to search and filter website content while providing secure access through GitHub OAuth authentication.3
- Flicense-qualityDmaintenanceEnables fetching and reading content from web URLs through a configurable proxy prefix. Uses curl to retrieve webpage content and return it as text for analysis or processing.
- AlicenseAqualityCmaintenanceEnables AI agents to extract clean, structured web content (articles, tables, links, visual layouts) optimized for LLM token efficiency, with fast response times and optional JavaScript support.581MIT
- FlicenseAqualityCmaintenanceWeb-crawler MCP server that fetches web pages with static parsing or headless browser fallback, enabling Claude to fetch, extract links, crawl sites, and select content via CSS selectors.4
- AlicenseAqualityCmaintenanceThis server crawls documentation sites and provides search and page-retrieval tools through MCP, enabling users to query indexed documentation content.4MIT
- Alicense-qualityAmaintenanceAn MCP server providing web search, image search, and page scraping tools to LLMs without requiring API keys.2MIT
- Alicense-qualityDmaintenanceMCP server for intelligent web crawling using a ReAct agent. Adaptively explores websites and returns structured analysis.Apache 2.0
- Flicense-qualityDmaintenanceEnables AI assistants to search and collect data from Xiaohongshu/REDnote, including notes, comments, and user info, with dual-layer anti-detection and persistent login.11
- Flicense-qualityBmaintenanceMCP server that gives AI agents access to current web information through web_search and fetch_url tools, using Tavily, Brave, and DuckDuckGo providers with automatic fallback.
- Flicense-qualityDmaintenanceA specialized web crawler and scraper for the US Patent and Trademark Office (USPTO) website, built with Crawl4AI and MCP integration, enabling patent and trademark search, extraction, and status checking.4
- AlicenseAqualityBmaintenanceAn MCP server that fetches web pages and extracts clean, AI-usable context from them, enabling tools for link discovery, content search, and integrated fetch-and-search operations.571MIT
- AlicenseAqualityBmaintenanceExtract content from URLs, documents, videos, and audio files using intelligent auto-engine selection. Supports web pages, PDFs, Word docs, YouTube transcripts, and more with structured JSON responses.2168MIT
- AlicenseAqualityDmaintenanceEnables extracting visible text and links from webpages for content analysis.2MIT
- FlicenseAqualityDmaintenanceEnables retrieval and cleaning of official documentation content for popular AI/Python libraries (uv, langchain, openai, llama-index) through web scraping and LLM-powered content extraction. Uses Serper API for search and Groq API to clean HTML into readable text with source attribution.12
- AlicenseAqualityFmaintenanceA comprehensive website crawler and SEO analyzer that stores site data in a local SQLite database for AI-driven auditing. It enables users to detect technical SEO issues, broken links, and security vulnerabilities through natural language queries or terminal commands.45416Apache 2.0