Scrape a public URL to create a searchable knowledge base. Extracts content from web pages (including JavaScript-rendered), chunks it, and indexes for RAG.
MIT
604,399 tools. Updated 2026-09-23 19:38
"Tools for simulating browser behavior to scrape web pages" matching MCP tools:
- Batch scrape up to 100 URLs in parallel into LLM-ready markdown, with background job support and options for JavaScript-heavy pages.-
- Search web, news, and image sources, then optionally scrape each result URL for markdown, HTML, links, or screenshots to support research and information gathering.Server Side Public , v 1
- Crawl a website to discover and scrape multiple pages, following links up to a set depth. Filter URLs with regex and use saved or custom scraping configs.MIT
- Scrape any public URL and return clean, LLM-ready markdown. Supports JavaScript-rendered pages for complete data extraction.MIT
- Scrape a single URL using request, browser, or auto mode. Get HTML, markdown, screenshot, or PDF output, with optional AI extraction for structured data.MIT
Matching MCP Servers
- AlicenseNot gradedqualityBmaintenanceEnables scraping and fetching websites with protection handling like Cloudflare and captchas, via an MCP interface.11 npm1Apache 2.0
- AlicenseBqualityDmaintenanceA comprehensive web scraping server that transforms web content into clean, agent-ready Markdown with automatic citations and efficient caching. It features a robust suite of tools for metadata extraction, sentiment analysis, SEO auditing, and security scanning while strictly adhering to robots.txt policies.482 npm45MIT
Matching MCP Connectors
x402-gated web extraction gateway. Tools: extract, extract_batch.
Decision Layer for AI Agents — 58+ tools, Advisor, MCP. Free key: POST /v1/register {}.
- Trigger an immediate web scrape for a specific Gluedly page ID to get current snapshot data on demand.MIT
- Scrape web pages into Markdown, JSON, or CSV, with JavaScript rendering and schema-based structured data extraction for complex sites.MIT
- Turn natural language descriptions into structured scrape configs. Specify what to scrape, get a ready-to-use configuration for web scraping.MIT
- Extract structured data from web pages including markdown content, links, tables, images, and metadata. Supports JavaScript-rendered pages with optional CSS selector waiting for dynamic content.MIT
- Retrieve URLs, titles, and snippets from web search to selectively scrape or filter results yourself.MIT
- Scrape a single web page and extract content in markdown, text, or HTML. Strip navigation and ads, include links, and choose between HTTP or browser rendering.MIT
- Given a URL, extracts the page title, readable markdown, and plain text. Helps AI agents read web content, scrape data, and feed RAG pipelines.MIT
- Browse, scrape, and capture web pages with proxy rotation, JavaScript rendering, and anti-bot bypass.MIT
- Bypass Cloudflare, Datadome, and other bot-detection blocks by rendering pages in a real browser with randomized fingerprints and human-like behavior. Use after a normal scrape returns 403, 429, CAPTCHA, or an empty shell.MIT
- Capture web page screenshots for verifying updates, with automatic tiling for full pages and optimized processing for CLI tools.MIT
- Search the live web and read top result pages as clean markdown in one call. Ask a question to get ranked results and full page content from fresh sources.MIT
- Retrieve stored sessions with cookies for authenticated web scraping. Use the returned session IDs to scrape login-protected pages like Amazon orders or LinkedIn profiles.MIT
- Discover all indexed URLs on a website. Use the search parameter to find specific pages when scrape returns empty or incomplete results.MIT
- Open a new browser session to audit web pages, including those behind logins or multi-step journeys. Returns a session ID for navigation and interaction tools.MIT
- Trigger hover events on web elements to reveal dropdown menus, tooltips, and hidden UI content by simulating realistic browser mouse behavior.MIT