"Web crawling techniques and tools" matching MCP servers:

Enables web searches through DuckDuckGo without an API key and extracts readable main content from web pages.

Enables private web search and webpage content extraction using a local SearxNG instance, prioritizing user privacy and autonomy.

Enables AI-assisted user feedback collection and management via a web interface, with MCP integration for automated replies and tool execution.

An MCP server that enables AI assistants to perform web searches on Perplexity.ai using browser automation instead of an official API. It supports persistent authenticated sessions and returns search results along with cited sources directly to the client.

Search and resolve MCP server packages on the mpak registry, enabling agents to discover servers, inspect tools, and check credential requirements before installation.

Provides an LLM client with self-hosted web scraping capabilities including scraping, crawling, mapping, searching, structured data extraction, browser interaction, and deep research, without reliance on paid proxy or CAPTCHA services.

Provides a web search tool with citations by running Claude Code CLI in an isolated Docker container, enabling search even in environments without direct Anthropic web search support.

Provides web automation utilities, including taking Camofox browser tab screenshots and saving them directly to local files.

Fetches a URL and returns the main content as clean Markdown, using plain HTTP when possible and headless Chromium for JavaScript-rendered or bot-protected pages.

Connect AI assistants to your Gramps Web family tree. Search and edit genealogy records, explore ancestors, descendants, relationships and timelines, and read document scans with vision-capable models. Includes guided research workflows, source citations, read-only mode, and privacy-aware media previews. Install with Docker or a Claude Desktop bundle.

Bridges the editing capabilities of an already-open DingTalk document in the browser to any MCP host via standard stdio, forwarding tools to the page's internal MCP server over a local WebSocket. Supports read/write operations through suggested edits, pairing-based security, and zero runtime dependencies.

A lean FastMCP server that wraps xAI's web_search and x_search tools for MCP hosts, enabling live web and X search with citations.

MCP server for AI-powered web research — search (Google, Brave, Serper, SearXNG), scrape any page, extract PDFs/DOCX/YouTube transcripts, academic & patent search. Single Go binary. Works with Claude, Cursor, Copilot, and any MCP client.

Enables AI assistants to search Perplexity.ai through browser automation without an API key, returning answers with cited sources.

A lightweight MCP server providing common utility tools such as unit conversion, password generation, date calculation, hashing, Base64 encoding/decoding, UUID generation, text case conversion, and text statistics, with no external dependencies.

Enables inspecting local OpenAPI specification files by listing and retrieving spec details with filters and $ref resolution.

Enables natural-language interaction with Ladybug Tools for building performance simulation, 3D modeling, and data visualization, allowing agents to perform complex environmental analysis tasks without CAD software.

repo-release-tools mcp-server automates the software release process by managing version bumps, changelog generation, and releases. It streamlines repository maintenance by handling repetitive release tasks directly through your AI assistant.

MCP server for querying Google Gemini Web from Claude Code and other MCP clients, using cookie-based authentication. Enables creating, continuing, listing, viewing, and deleting Gemini chat conversations with model and thinking mode control.

Exposes SQL Server and Snowflake schema metadata to AI coding agents, enabling schema search, join path discovery, and stored procedure metadata retrieval without live queries.