"A search for information about deep crawl web crawling and site auditing tools" matching MCP servers:

MCP server and CLI for querying Common Crawl via AWS Athena, enabling users to check a domain's presence in crawls, analyze dynamics, compare domains, and inspect page text.

Provides keyless access to USDA SSURGO soils, NWI wetlands, EPA environmental databases, and USGS seismic design values for parcel-based site screening.

Provides access to the Reuters Business and Financial News API to retrieve articles, trending news, and market data. It enables searching and filtering financial content by date, author, category, and keywords.

MCP server that provides tools to query Korean financial products (savings, deposits, pensions, mortgages, loans) via the FSS Open API, enabling search by company or product type.

This MCP server enables users to search for scientific papers on arXiv and retrieve detailed metadata for specific papers. It provides tools to perform search queries and fetch in-depth information using paper IDs.

Enables searching Reddit posts, fetching subreddit content, and retrieving post comments without requiring an API key. It uses public JSON endpoints to provide seamless access to Reddit data for LLM-based applications.

Enables searching, downloading, and exporting academic papers from 20+ scholarly sources including arXiv, PubMed, and Semantic Scholar. Supports multi-source concurrent search, citation network tracing, and export to CSV, RIS, and BibTeX.

MCP server that silently monitors state changes in structured data sources (e.g., government open data) and provides a pull-based channel for AI agents to retrieve real mutations since their last cursor.

MCP server that finds open-source, commercially-usable icons and returns direct SVG download links, searching over 300,000 icons with license filtering.

Enables searching and retrieving Korean Supreme Court precedents from the law.go.kr Open API, with support for keyword search and detailed judgment text.

Enables searching scholarly literature across CrossRef, ERIC, Semantic Scholar, and OpenAlex, and retrieving open-access PDFs via Unpaywall.

Real-time search of Japanese government subsidies and grants (official jGrants data): deadlines, amounts, eligibility, filterable by prefecture.

Enables to search academic papers in Japanese and English, inspect availability of full text, and download public PDFs locally.

A standalone MCP server that searches multiple scholarly metadata providers (PubMed, Europe PMC, bioRxiv, Crossref, OpenAlex, Semantic Scholar, arXiv), deduplicates results, and returns ranked literature with optional open-access full-text retrieval from Europe PMC.

Enables AI assistants to search for images on Wikimedia Commons, returning structured metadata and a composite thumbnail grid for visual comparison.

Search Japan's National Diet records (1947-present): ministerial statements, committee Q&A, and policy debates with speaker, party, and citation URLs.

An MCP server that provides access to NS (Dutch Railways) travel information.

High-value scientific tools for AI agents — literature search (PubMed, arXiv, Semantic Scholar), chemical compound lookup (PubChem, ChEMBL), patent prior art search (USPTO, EPO), GPU spot prices across 4 providers, and real-time earth science data (USGS, NASA, OpenAQ). Per-call billing via API key. Keys issued instantly at https://mcp-site.com/keys/request

Enables searching and downloading academic papers from 14 platforms including arXiv, PubMed, Google Scholar, Web of Science, Springer, and Sci-Hub with unified data format and intelligent rate limiting.

MCP server for answering questions about official statistics from a single publication URL, with tools to check query scope, retrieve overview, text, data files, and search.