"Information about web crawlers or scraping techniques" matching MCP connectors:
GET /v1/connectors – MCP directory API referenceMatching Connector Tools:
Hundreds of scraping & data APIs through one key. USD pay-per-request, normalized schemas, failover.
Pay-per-call agent superpowers: media/video gen, product demos, research, GTM, scraping, compute.
Direct access to 60+ scraping and search tools. Extract structured data from Google (Search, Maps, Trends), Amazon, Airbnb, Social Media, and any web page directly into your AI agent.
Convert documents and web pages to clean Markdown: PDF, DOCX, XLSX, EPUB, scanned files, any URL.
Stealth scraping API for AI agents. Clean Markdown from any URL. x402 crypto payments.
Micro-settled web content extraction and JSON schema validation utilities for AI agents and MCP clients, backed by the Bristlecone Logic API engine.
Check whether AI agents can read any website: 18 deterministic checks, each with its fix.
SSRF-safe web intelligence: 4 outcomes, 43 paid x402 tools, 3 free tools and a URL preview.
All HasData scraping tools in one MCP server: Google, TikTok, Instagram, maps, e-commerce and more.
An MCP server that gives any LLM or agent clean YouTube transcripts on demand: a single video, a whole channel, or a playlist, plus AI cleanup of auto-generated captions. API-key auth, credit-based, same backend as the public v1 API. Get a free API key with 25 free credits at youtubetranscriptdownload.com/account.
Hundreds of scraping and data APIs through one gateway — one key, USD pay-per-request, normalized schemas, automatic failover.
Docs: https://docs.keenable.ai/mcp-server Keenable is a free, remote MCP server that gives agents access to the web index. Search the web with ranked results and date/site filters, then fetch any indexed page as clean markdown. Works out of the box with no account or API key.
Do things on live websites: prices, availability, quotes, bookings, anything behind a form or login.
CatchAll is a web search API built for comprehensive event retrieval — not ranked results, but all matching records.
Product search and price comparison for AI agents. Search 100M+ products across thousands of retailers by text or image, compare live offers and prices, and turn any merchant URL into structured product data. No API key required to start; add one for unlimited use and affiliate commission.
Pay-per-call web scraping for AI agents via x402 on Base USDC. Six tools, no signup.
Pay-per-call web scraping for AI agents — no signup, no API keys, just USDC micropayments via the x402 protocol on Base. Six tools: scrape webpages, extract structured data, capture screenshots, parse metadata, manage browser sessions, and run workflows. Runs on Cloudflare Workers with Browser Rendering. Free tier: 10 calls per wallet per 30 days.
Reliable web access for AI agents: smart HTTP, rotating proxies, and full-browser rendering.
Web tools for agents: fetch URL as markdown (free MCP) + x402 scrape, links, AI JSON, snapshot.
One MCP server for 180+ live web-data APIs returning clean JSON from sites that block scrapers.
