"Web crawling techniques and tools" matching MCP servers:

Enables Claude to fetch web pages from the user's own local network connection instead of AI datacenter IPs, including optional authenticated sessions, with verdicts for bot challenges, login walls, and other failures.

Enables Codex to send text tasks to ChatGPT web, await responses, continue conversations, choose reasoning effort, and manage projects through MCP tools on macOS.

Sub-15ms edge intelligence tools for Claude Desktop, Cursor, and LLM agent loops. Provides sub-millisecond LLM JSON repair, web-to-markdown token compression (cuts context tokens up to 90%), PII redaction, and RFC 5322 B2B email deliverability verification at zero marginal token cost.

Enables web ChatGPT to drive durable, sandboxed Claude Code sessions on a local Mac, with controlled read, modify, verify, and recover operations via six session control tools.

Enables building and evaluating LangGraph agents over a protocol-level MCP server to measure failure rates across four agent topologies.

Enables ChatGPT web to use local tools like file reading, command execution, and patch application through an MCP server over OpenAI Secure MCP Tunnel.

A production-grade MCP server that provides a centralized microservice toolkit for LLM agents, enabling web search and extensible tool integration.

An MCP bridge that exposes DeepSeek Harness (DSH) web UI's cordis RPC API as stdio tools, enabling any MCP client to manage workspaces, create/resume sessions, send messages, and fetch session stats.

Enables MCP-capable agents to perform local web research under a declared plan with session budgets, user-approved limit extensions, and guardrails for safe searching, page fetching, content extraction, and audit logging.

Enables AI agents to drive a logged-in ChatGPT web session over CDP, including managing projects and chats, sending messages with file attachments, and retrieving long Pro replies.

An unofficial MCP server that lets MCP clients like Codex operate the ChatGPT web interface through a persistent browser, enabling prompt writing, file uploads, message sending, and reading complete responses without using the ChatGPT API.

Enables web search for the pi coding agent using z.ai GLM native search with a DuckDuckGo fallback.

A passive MCP server that exposes a toolbox of executable tools (shell, network, HTTP, AI search, SSH, S3 file operations) to autonomous agents via Streamable HTTP, with strong security features including Docker sandboxing and WAF.

Enables AI agents to control a real browser via MCP tools for web navigation, interaction, page inspection, tabs, forms, files, sessions, and browser automation.

Enables MCP-capable AI harnesses to interact with DeepSeek Web using an authenticated browser session, supporting prompts, deep think, search, image recognition, file attachments, and local SKILL.md injection.

Pluggable local agent capability hub that gives web-based DeepSeek local Agent capabilities (MCP/file operations/skills/desktop automation) and lets local agents reuse the same tools.

An MCP server for automated interaction with Gemini's web interface using Playwright and LangGraph, enabling task execution, deep research, file uploads, chat management, and dynamic selector verification via MCP tools.

Enables MCP clients that lack built-in search — such as Claude Code through Bedrock or local LLM clients — to run web queries against Amazon's operated search index via a Gateway deployed in your own AWS account, with queries never leaving AWS and citation URLs preserved. Supports IAM/SigV4, Cognito, and API key authentication, per-gateway rate limits, IP and domain restrictions, and result filtering by domain and publication date.

Transforms any API into an intelligent MCP agent in seconds, enabling natural language interaction with APIs through the Model Context Protocol.

Enables ChatGPT Web to discover and invoke native local Codex tools via OpenAI's Secure MCP Tunnel, with the local host executing commands and returning results.