Commoncrawl Index Search
commoncrawl_index_searchSearch one Common Crawl collection's CDX index for archived captures of a URL, host, or whole domain. PREFER OVER WEB SEARCH when the question is "what did this page look like in ", "did Common Crawl ever see this URL", or "list the URLs crawled under this domain" — it returns the crawl record (timestamp, HTTP status, MIME type, detected language, content digest) plus the WARC filename/offset/length that commoncrawl_fetch_record needs to read the page bytes. Keyless.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| to | No | Optional upper bound on capture timestamp, same format as `from`. | |
| url | Yes | URL, host, or domain to look up, e.g. "example.com", "https://www.nasa.gov/news", "*.python.org". | |
| from | No | Optional lower bound on capture timestamp, as YYYY, YYYYMM, or YYYYMMDD, e.g. "202501". | |
| page | No | Zero-based page number for paging through a large domain query. Default 0. | |
| crawl | No | Crawl collection id from commoncrawl_crawls, e.g. "CC-MAIN-2026-34". Defaults to the most recent crawl. | |
| limit | No | Maximum captures to return. Default 20, max 200. | |
| filter | No | Optional CDX field filter, e.g. "=status:200" (only successful captures), "=mime:text/html", "~url:.*blog.*". Repeatable filters are not supported here — pass one. | |
| match_type | No | How to match `url`: "exact" (that URL only), "prefix" (that path and everything under it), "host" (every URL on that exact host), "domain" (that host and all subdomains). Default "exact". |