"Web crawling techniques and tools" matching MCP servers:

Enables deterministic calculations for AI API and subscription costs, meeting time zones, recurring schedules, and cron next-run previews, returning normalized inputs, units, assumptions, dated sources, and links to results.

Odysseus Web MCP is a standalone Model Context Protocol (MCP) server for safe public-web search and URL fetching.

Enables converting any URL or website into clean, LLM-ready Markdown, supporting single-page scraping and full-site crawling for AI agents.

Enables AI agents to search, filter, and monitor curated AI model API free credits and low-cost token promotions through eight tools, four resources, and three prompts — including finding deals expiring soon, checking eligibility and blocked conditions, and producing daily briefs.

Enables AI agents to query multi-cloud costs, inventory, waste, and commitments across AWS, GCP, Cloudflare, and OVH through read-only MCP tools.

MCP tool server that gives any AI agent the ability to search, scrape, and analyze content across the internet.

Enables AI agents to perform common utility tasks through a streamable HTTP MCP endpoint, including JSON inspection, regex testing, cron parsing, hashing, base64 encoding/decoding, URL analysis, color conversion, text diffing, CSV parsing, JWT decode/verification, Markdown-to-HTML rendering, and UUID/token generation without API keys.

Whatsapp-Web MCP

Connect AI assistants to your Gramps Web family tree. Search and edit genealogy records, explore ancestors, descendants, relationships and timelines, and read document scans with vision-capable models. Includes guided research workflows, source citations, read-only mode, and privacy-aware media previews. Install with Docker or a Claude Desktop bundle.

MCP server for AI-powered web research — search (Google, Brave, Serper, SearXNG), scrape any page, extract PDFs/DOCX/YouTube transcripts, academic & patent search. Single Go binary. Works with Claude, Cursor, Copilot, and any MCP client.

Enables reading web pages and PDFs as clean Markdown, extracting main content and letting agents query for relevant passages, outline sections, search in-page, and list links while respecting robots.txt and blocking private addresses.

Provides a unified web search tool for AI agents by running multiple search providers concurrently and returning normalized results. It integrates with Claude Code, Codex CLI, OpenCode, and Hermes.

MCP server for file operations with non-UTF-8/Unicode encoding: Cyrillic, CP1251, CP1252, ISO-8859, KOI8 auto-detection

Enables fetching any public web page through managed proxies, with optional JavaScript rendering, returning markdown, text, HTML, or structured JSON.

Enables AI coding assistants to validate HTML/CSS markup using W3C APIs, perform technical SEO audits, check broken links, and validate JSON-LD schemas directly in local workspaces.

Provides read-only, governance-audited operations for OLVM and oVirt 4.5 engines, enabling inventory, health, capacity, and diagnosis queries over MCP and CLI with audit logging and budget guards.

Enables validating JSON/JSONC, getting/setting values with jq-like paths, pretty-printing or minifying, validating against Ajv JSON Schema, and diffing local files or inline strings without network calls.

Governed AI-ops for Ceph, providing root-cause health analysis and guarded destructive operations via a built-in governance harness with risk tiers, audit, and undo recording.

repo-release-tools mcp-server automates the software release process by managing version bumps, changelog generation, and releases. It streamlines repository maintenance by handling repetitive release tasks directly through your AI assistant.

Enables MCP-capable agents to perform reliable web search and content fetching through a single server that automatically fails over across multiple providers. It handles keyed or keyless authentication automatically and returns clean Markdown or text results.