"Understanding Web Crawlers" matching MCP connectors:
Matching Connector Tools:
Agent402: 500+ pay-per-call tools for agents over x402 + MPP or free via PoW. Agentic Finance
Agent intake desk for property renovation. Read only, CORS open, fail closed, receipts recompute.
SSRF-safe web intelligence: 4 outcomes, 43 paid x402 tools, 3 free tools and a URL preview.
Read any page or file as clean text, plus live news, weather, market holidays and timezone lookups for agents. Pay-per-call in USDC on Base via x402 — no signup, no API keys.
Scan/purge hidden text & score web domains: TLS, DNS, headers, speed. Free scans, $0.01 actions.
Give AI assistants secure access to your organization's structured business data. Search records, create and update records, retrieve schema information, and manage workflow states using natural language. You need two values for every request: x-api-key — your Web Data Forms API Key x-group-id — your Web Data Forms Group ID You can find these in your Web Data Forms accounts group->information page. Preferred method: request header When possible, pass the credentials as HTTP headers: x-api-key: <your-api-key> x-group-id: <your-group-id> This is the preferred option because it keeps credentials out of the URL and is more secure. Fallback method: query parameters If your MCP client does not support custom headers, the server also accepts the credentials as URL query parameters. Example: https://mcp.webdataforms.com?x-api-key=abc123&x-group-id=xyz456 Detailed information here: https://github.com/Web-Data-Forms/mcp-server-docs/blob/main/README.md
Russian-language data endpoints for AI agents: EGRUL/INN company lookup, Cyrillic web search, RU page to Markdown. Pay-per-call via x402 (HTTP 402) in USDC on Base — no API keys, no accounts.
51Drive real Android & iOS devices and web browsers from natural language for mobile + web QA. 145+ tools across device control, app management, automation sessions, browser automation, and flow recording / replay. Bearer-auth — get a token at robotactions.com → Profile → API Tokens.
AppDeploy turns app ideas described in AI chat into live full-stack web applications
CatchAll is a web search API built for comprehensive event retrieval — not ranked results, but all matching records.
Docs: https://docs.keenable.ai/mcp-server Keenable is a free, remote MCP server that gives agents access to the web index. Search the web with ranked results and date/site filters, then fetch any indexed page as clean markdown. Works out of the box with no account or API key.
Pay-per-call web scraping for AI agents via x402 on Base USDC. Six tools, no signup.
Search real-world UI & UX design references for mobile apps, web apps, and websites with Mobbin.
Official MCP server providing AI assistants with direct access to Stimulsoft Reports & Dashboards developer documentation. Enables semantic search across FAQ, Programming Manual, Server Manual, User Manual, and Server/Cloud API references across all Stimulsoft platforms (.NET, WPF, Avalonia, WEB, Blazor, Angular, React, JS, PHP, Java, Python).
Pay-per-call web scraping for AI agents — no signup, no API keys, just USDC micropayments via the x402 protocol on Base. Six tools: scrape webpages, extract structured data, capture screenshots, parse metadata, manage browser sessions, and run workflows. Runs on Cloudflare Workers with Browser Rendering. Free tier: 10 calls per wallet per 30 days.
Universal pay-per-call tool API for AI agents. Web search, price feeds, hive memory, LLM routing, code sandbox. Free tier (10 calls/day, no signup). x402 payments on Base + Solana.
Give your agent web search and authoritative datasets: S&P Global, FRED, OECD, SimilarWeb & more.
Fast, intelligent web search and web crawling. New mcp tool: Exa-code is a context tool for coding
**ColdState Knowledge Search MCP Server** https://github.com/daniel-coldstate/coldstate-mcp Semantic search over 64.6M knowledge entries — the structured alternative to web search APIs and web scraping for LLM agents. No crawling, no rate limits, sub-3s responses. Cloud-hosted at services.coldstate.ai
Ultra-fast web scraper and deep discussion crawler delivering clean Markdown for AI Agents
11