"Information about web crawlers or related technology" matching MCP connectors:
GET /v1/connectors – MCP directory API referenceMatching Connector Tools:
Let your AI agent see any public web page: one screenshot tool, returns the page as an image.
Fetch the transcript of any YouTube video (plain text or timestamped segments, in any available caption language), search YouTube for videos and channels, list a channel's uploads, Shorts, playlists and community posts, search inside one channel, and list every video in a playlist. Hosted Streamable HTTP server from getyoutubetranscript.com; sign in with OAuth or an API key. Credit-based: one credit per successful call.
Fetch public web pages as clean, token-budgeted text for AI agents.
One MCP endpoint for 414 data APIs across 90 platforms, including TikTok, Instagram, YouTube, Amazon, Xiaohongshu (RedNote) and Douyin, plus 17 web search tools and CAPTCHA solving for 52 types. Pay per call from $0.56 per 1,000 calls; failed or empty calls are not charged.
Live web data for AI agents: structured JSON from 248 websites through one API key.
5 tools: PDF, Word, Excel, PowerPoint and web pages to Markdown; invoice fields as JSON.
Run saved PDF/office file flows, convert HWP/HWPX to Markdown or DOCX, and call data/text APIs.
Web research across search, Reddit, YouTube, local businesses, reviews, ad libraries and more. Every answer carries line-numbered citations; up to 1,000 rows per call without loading them into context.
Search the web, fetch any URL and map a site as clean, LLM-ready Markdown.
SnapRender gives AI agents eyes on the web through a hosted MCP server. Send a public URL and get the page back as an image, a PDF or clean markdown, with no browser to run or maintain. Capture the viewport or the full page as PNG, JPEG, WebP or PDF, at desktop size or with a phone, tablet or laptop preset, in light or dark mode. Ads and cookie banners are removed by default, and you can hide elements or click one before the capture. extract_content reads the same page as markdown, text, article, links or metadata, so an agent can see a page and read it in one session. Made for monitoring and client work: batch up to 50 URLs in one job, check the cache for free before spending a capture, share signed links that render without an API key, and get webhooks when a batch finishes or a scheduled capture spots a change. Track competitors' pricing and landing pages, or send clients current screenshots of their sites. Connect with OAuth (Google, GitHub or email) or an API key. Works with Claude, ChatGPT, Cursor, VS Code and any MCP client. Free plan: 200 screenshots a month, no credit card. Paid plans from $9 a month for 2,000 up to $199 a month for 200,000. Home: https://snap-render.com Docs: https://snap-render.com/ai-agents
Give your agent eyes: screenshots of any web page it can see, pages as markdown, and PDFs.
Let Claude, Cursor or Cline fetch pages and run Google searches from a real Singapore mobile IP (Singtel/M1). Hosted; free 24-hour trial key.
Evidence-backed technical SEO website check. EUR 9.90 via MPP or buyer-approved Stripe Checkout.
Remote MCP server for ProxyLane residential and ISP proxies. AI agents can check the account balance, create residential proxy lists by country, city or ISP, test connections, check IP reputation, and quote ISP proxy orders. OAuth sign-in, works with Claude, ChatGPT and Cursor.
Search Eventbrite events by city, category or keyword, with new-event alerts, by Magnova
Buzzsearch mcp is the social intelligence crawler that gives your agent access to user comments on TikToke, YouTube, Reddit, and Facebook. Know exactly what words your customers are saying and what they are complaining about.
30 tools: web pages, JS apps and PDFs as text, feeds, sitemaps, search. 25 free calls.
Web search and clean-text page fetch for AI agents, with SSRF protection.
Website audits for AI agents: one-call AI visibility report, schema, WCAG, PageSpeed, AI crawlers.
Find every upcoming event on Lu.ma by city, calendar or category, by Magnova
Find local businesses with no website, or with a broken or out-of-date one, in six countries.
Search open jobs from company career sites by the country you can apply from, remote or onsite.
Live web search results and web pages as Markdown; page fetches use a residential network.
One MCP URL for web search, page reading, pay-per-result data tools and a browser agent.
Seven services: account, HTML, Markdown, screenshots, brand, JSON extraction, and web search.
Personal web presence audit: declared personal presence record checked per call.
Four web/API/Base workflows, 11 paid MCP/REST tools; x402 USDC Base/Solana, MPP Base, prepaid.
Convert noisy HTML into clean Markdown, JSON tables & JSON-LD with token metrics via x402 USDC.
All public upAPI operations as MCP tools: web scraping, search, screenshots, PDF, OCR and more.
Verified web fetch for agents: detects blocks and empty JS pages, falls back, bills only on success.
Free AI visibility check: can AI crawlers read a site, and do ChatGPT, Gemini, Perplexity cite it?
Free compare_page_observations for two held JSON pages. No fetch, wallet, or payment.
Public web data as clean JSON for AI agents: Google, Amazon, TikTok, Instagram, YouTube, LinkedIn, Google Maps, Reddit, Zillow and hundreds more websites. One API key, pay per call, failed calls are free.
Pay-per-call web reading for AI agents over x402 — no account, no API key, billed only when content is delivered, with signed provenance on every paid response. Start free with POST /v1/probe.
Direct schema-verified web scraper ($0.015 USDC on Base), pre-flight dead-end URL health check ($0.001 USDC), and ABVS-1.0 two-bot verification relay.
Turn any web page URL or HTML into clean Markdown or plain text for an LLM to read.
Converts noisy HTML pages into clean Markdown, structured JSON tables, and JSON-LD metadata with token-savings telemetry. Supports x402 USDC on Base.
Full-page screenshots, PDFs and Markdown of any site, even bot-protected ones. Built for teams that capture third-party websites for brand protection, compliance evidence, media monitoring and competitor intelligence, with delivery by webhook or to your own S3 bucket. Also clean text, rendered HTML, and Urlbox projects, webhooks, past renders and usage. Sign in with OAuth or send an API key as a Bearer token.
Preserve public web pages as signed, timestamped evidence packages, then verify them, compare versions and monitor pages for changes.
Browser-backed web access, automation and structured data extraction for AI agents.
13 pay-per-result tools: jobs, funding, contacts, teams, phones, tech stacks, DNS, PDFs, reviews
Check which AI bots a website or a pasted robots.txt lets in, grouped by purpose.
Transcripts of YouTube, TikTok, Instagram Reels and podcasts for AI assistants. Text, SRT or VTT.
Scrape any website, extract structured data, and collect web content at scale with AI agents
Convert any web page into clean, LLM-ready Markdown. Strips ads, navigation and clutter; keeps headings, tables, code and links.
Web search for agents: page to Markdown, screenshots, news search, metadata, sitemaps and RDAP.
PriceInIndia is a price-comparison assistant for products sold in India. Ask about any product — phones, laptops, cameras, appliances and more — and it searches live listings from Amazon, Flipkart, Croma, Reliance Digital and 50+ other Indian retailers, then shows current prices and the best available offer as interactive product cards with direct links to buy from the retailer. You can also browse product categories to explore. Prices are in INR and come from live listings, so they can change quickly. PriceInIndia is for shopping research only — it does not sell products or process payments.
Web/PDF extraction and API comparison. x402 payments or prepaid comparison credit. Free discovery.
Production web scraping and extraction engine with markdown extraction, DOM pruning, and anti-bot mitigation.
Monitor web pages and read item-level updates: the field that moved, with its old and new value.
Fetch any public web page through managed proxies, with optional JS rendering and extraction rules.
Remote MCP server for Google SERP, SEO research and web scraping. Live Google results, keyword research, rank checks and page scraping for AI agents. Free plan with 250 credits, no card; failed requests never charged.
Send a prompt to ChatGPT anonymously and get the answer with its web sources, as JSON.
Scrape product catalogs from any e-shop and match the same products across shops, without EAN or GTIN. Hosted remote MCP with OAuth. Pay per result from prepaid credit.
Live crypto, DeFi, stock and web data plus datasets for agents. Pay per call with x402. No API key.
Hosted MCP: CSV/XLSX to JSON rows and text-PDF pages; private uploads or URLs, no API key.
Web scraping with real-browser rendering, browser automation recipes, full-page screenshots, Google AI Overview and ChatGPT answers, and Booking.com data through one hosted MCP server. Auth via OAuth 2.1 or API key.
Read any web page as clean markdown or text with title, description and links
Check public web pages for common WCAG 2.2 accessibility issues from your MCP client. The free scan_url tool needs no account and returns a score, severity breakdown, WCAG references, affected elements and fix guidance. Automated checks do not establish conformance; manual review is still required. Optional authenticated tools support monitoring your own sites. Documentation: https://allyproof.com/docs/mcp
See which ecommerce brands just started, scaled or stopped Meta and Google ads. Look up any store’s ad activity, platform and tracking pixels.