"Information about web crawlers or related technology" matching MCP servers:

MCP server for AI-powered web research — search (Google, Brave, Serper, SearXNG), scrape any page, extract PDFs/DOCX/YouTube transcripts, academic & patent search. Single Go binary. Works with Claude, Cursor, Copilot, and any MCP client.

Records Chrome network requests and responses for LLM knowledge bases, and exposes query and browser control capabilities via MCP tools like list_sessions, navigate, and evaluate.

MCP server that provides web search via LLM API gateways with server-side search, returning cited answers and multi-provider fallback.

Provides read-only access to Chinese A-share stock market data such as quotes, K-lines, financials, capital flows, news, and research reports via a stdio MCP server for DeepSeek Harness and other MCP clients.

Gives AI assistants real-time access to web data with the ability to scrape any website, bypass anti-bot systems, extract structured data, and capture screenshots through enterprise-grade infrastructure.

Enables users to interact with container registries through the ORAS CLI, providing information about container images, platforms, and signatures via natural language queries.

Mu is an MCP server and web app that provides agents and humans access to real-world services like news, web search, mail, markets, weather, video, places, images, files, calendar, and contacts. It offers a wide range of tools and can be self-hosted or used live at micro.mu.

A high-performance, asynchronous MCP server that provides comprehensive Google search and web content scraping capabilities through the Firecrawl API, designed for LLMs to retrieve external information efficiently.

Enables AI agents to search and retrieve academic literature from Web of Science and CNKI, with support for institutional login/SSO. Can be deployed locally on Windows or as a remote Linux server.

A secure and pluggable MCP server to run terminal commands on your local machine or cloud server — remotely, safely, and with LLMs or agentic clients.

CNN's Fear & Greed index for the US stock market: current score and rating, the seven component indicators, and about a year of daily history. Go standard library only, no API key required.

Baryon connects MCP-compatible AI assistants to Proton Mail through a local bridge, enabling email search, reading, attachment handling, and draft creation while enforcing deliberate permission limits. It runs locally and cannot send, delete, or modify mail.

MCP server for a self-hostable personal planner — exposes memo, todo, calendar event, and cross-resource full-text search tools so Claude, Cursor, or any MCP client can manage your schedule and notes in natural language.

A local AST and type-aware Go repository context indexer that uses CHA to map symbols, call graphs, dependencies, and architecture patterns. Built for AI coding agents that need to navigate and reason about large Go codebases without reading raw files.

An MCP server providing web search, image search, and page scraping tools to LLMs without requiring API keys.

Scans source code for possible Korean compliance risks (PIPA, Network Act, Credit Information Act, E-Commerce Act) and maps findings to specific legal provisions. Runs locally, exposes MCP tools for scanning and law lookup, and does not constitute legal advice. That's hedged (matches the project's "possible risk / not legal advice" rule), keyword-rich for search, and names all four laws.

A Linux system monitoring MCP server that provides real-time information on CPU, memory, disk, network, processes, Docker, security, and more via MCP tools.

Local-first MCP proxy with BM25 tool discovery, quarantine security, Docker isolation, OAuth support, activity logging, and web UI. Routes multiple upstream MCP servers through a single endpoint.

A deterministic symbol oracle over a locally built index of the repository: whether a symbol exists, what a file declares, and what changed structurally since the last snapshot. It also ships a PreToolUse guard that stops an Edit/Write referencing a function, constant or type the repo does not declare, before the write lands rather than after the build fails. No LLM, no API keys, no network, no l

Local CI/CD engine for AI agents. ActionD listens to LGH git events and runs plugin jobs (Go/Python/Java/web toolchains) in isolated checkouts, with structured results, root-cause failure diagnosis, approval gates, and auto-rollback. The MCP server exposes 23 tools, including dev_cycle_run (commit → CI → structured verdict in one call) and actiond_diagnose for failed-job analysis.