Fetch web pages and APIs with a single HTTP request. Supports static pages, JSON APIs, and server-rendered HTML, plus unblocker for challenging targets.
MIT
308,279 tools. Last updated 2026-07-18 05:10
"A server for crawling web pages and finding subpages" matching MCP tools:
- Start multi-page web crawling to extract structured data with AI or convert content to markdown from a starting URL.MIT
- Retrieve pages and nested subpages from a ClickUp doc, ordered by tree position. Optionally include full markdown content for detailed access.Apache 2.0
- Search the web to find relevant web pages on any topic. Returns titles, URLs, snippets, and relevance scores.MIT
- Extract structured data from web pages by supplying URLs and a prompt, with options for web search and retries.MIT
- Capture web page screenshots for verifying updates, with automatic tiling for full pages and optimized processing for CLI tools.MIT
Matching MCP Servers
- AlicenseBqualityDmaintenanceEnables deployment of HTML content, folders, and full-stack projects to EdgeOne Pages to generate publicly accessible URLs. It utilizes EdgeOne Pages Functions and KV storage for high-performance edge delivery of web applications.Last updated24MIT
- Flicense-qualityDmaintenanceExposes two MCP tools (discover and execute) that enable agents to query an OpenAPI schema via natural language and execute matched API operations.Last updated
Matching MCP Connectors
tldr-pages community simplified man pages (cached 24h)
URL to clean markdown for LLMs: a polite, robots.txt-respecting web reader. Free, no API key
- Create a new page in Voog CMS. Specify title, slug, and language; optionally set parent for subpages or node for multilingual translations.MIT
- Add web pages to your knowledge base by crawling a URL. Returns a task ID to check progress using crawl_status.MIT
- Extract sanitized HTML from web pages to analyze structure, identify form fields, and plan automation selectors for web crawling operations.MIT
- Analyze complex research questions by crawling multiple web sources and generating comprehensive LLM-based analysis.MIT
- Submit URLs to Google Indexing API for fast crawling and indexing. Use to notify Google about new, updated, or deleted web pages.MIT
- Extract and analyze hyperlinks from web pages, organizing URLs, anchor text, and contextual information into a structured format. Supports site mapping, SEO analysis, broken link checking, and targeted crawling preparation. Handles relative and absolute URLs with optional base URL and output limits.MIT
- Retrieve all pages within a specified namespace hierarchy, including subpages, to discover and access related content.MIT
- Fetch a web page and extract all hyperlinks with anchor text, classified as internal or external. Ideal for crawling and agent navigation.MIT
- Search full-text of archived Portuguese web pages. Find pages that contained specific terms, with optional filtering by date range, site, or MIME type.MIT
- Retrieve a list of Wiki.js pages for crawling or indexing, with options to filter by locale, limit results, and sort by fields like title or update date.MIT
- Analyze a website's robots.txt file to determine crawl permissions and ensure compliance with ethical web scraping practices. Provides insights into allowed and disallowed paths for crawling.MIT
- Stop crawling a specific document to prevent indexing or updates. Provide the document name to disable its crawling.MIT
- Conduct comprehensive web research by crawling, searching, and analyzing content to gather detailed information on any topic.MIT
- Conduct deep web research on any query using crawling, search, and AI analysis to gather comprehensive information from multiple sources.MIT