Extract and scrape webpage content using auto, simple, Scrapy, or Selenium methods. Define extraction rules or wait for specific elements to retrieve targeted data.
MIT
524,765 tools. Updated 2026-09-06 18:04
"Scrapy" matching MCP tools:
- Generate web scrapers using Playwright, Scrapy, or httpx with anti-bot configuration and data extraction logic to pull structured content from websites.MIT
- Fetch raw HTML content from any URL with optional JavaScript rendering for dynamic websites and Single Page Applications.MIT
- Extract webpage content for analysis by fetching clean markdown, HTML, or structured JSON. Handles JavaScript-rendered pages and bypasses anti-bot protection.MIT
- Fetch a URL and return its content as clean, LLM-ready Markdown. Automatically renders JavaScript and bypasses bot detection for any single page.MIT
- Fetch a single web page through a residential proxy and return clean Markdown, escalating to a real browser when bot-protected. Supports structured or AI extraction for targeted data.MIT
Matching MCP Servers
- AlicenseAqualityDmaintenanceA headless web-scraping MCP server built on Scrapy, providing tools for polite fetching, CSS/XPath extraction, link/table extraction, sitemap and robots.txt reading, and bounded asynchronous crawls.10MIT
- AlicenseNot gradedqualityCmaintenanceA general-purpose MCP server for crawling and extracting structured data from any website. Supports tools for crawling, single-page extraction, search-and-crawl, and schema extraction.2MIT
Matching MCP Connectors
Pay-per-call web scraping for AI agents via x402 on Base USDC. Six tools, no signup.
Pay-per-call web scraping for AI agents — no signup, no API keys, just USDC micropayments via the x402 protocol on Base. Six tools: scrape webpages, extract structured data, capture screenshots, parse metadata, manage browser sessions, and run workflows. Runs on Cloudflare Workers with Browser Rendering. Free tier: 10 calls per wallet per 30 days.
- Scrape any URL through Sessemi, bypassing Cloudflare, DataDome, and Akamai anti-bot protections. Returns page content as HTML or JSON.MIT
- Scrapes a single web page and returns its content as structured JSON with markdown, HTML, links, or metadata.Apache 2.0
- Fetch a public URL and convert it to clean, readable markdown with extraction details. Handles HTML, JavaScript-rendered pages, and PDFs, while verifying redirects and upstream errors.Apache 2.0
- Generate a ready-to-use NodeMaven proxy URL with geo-targeted username for curl, Playwright, or Scrapy. Encodes country, session, and IP filters into the connection string.MIT
- Scrape any public webpage by providing a target URL and API key. Automatically handles rotating proxies, CAPTCHA bypass, and optional JavaScript rendering to return the raw HTML.MIT
- Search live web or news index and return ranked results with URLs, titles, and snippets. Customize query, type, and limit for precise findings.Apache 2.0
- Check Scrapyard's version, breaker state, adapter registry, and cache counters to diagnose search failures or fallback adapter usage.Apache 2.0
- Unlock Android device screens without secure locks by waking the display and sending unlock commands through ADB for devices without PIN, password, or pattern protection.MIT
- Extract structured fields from any URL by describing them in plain language, with caching, pagination, and optional LLM-based selector discovery.MIT
- Given a URL, extracts the page title, readable markdown, and plain text. Helps AI agents read web content, scrape data, and feed RAG pipelines.MIT
- Start a bounded breadth-first crawl from a starting URL and receive a job ID immediately, which you can use to poll crawl status and retrieve results asynchronously.MIT
- Retrieve full text from multiple domain-deduplicated web sources in one query, including extraction metadata for provenance.Apache 2.0
- Pulls markdown, links, metadata, branding, screenshots, or structured JSON from a single URL in one call, eliminating multi-request fetch fan-out.MIT
- Execute shell commands on Android devices via ADB to automate tasks, manage files, or control system functions. Use caution as commands can perform any device operation.MIT
- Set clipboard content on Android devices via ADB for text transfer during automation tasks, with awareness of Android security restrictions.MIT