Commoncrawl Fetch Record
commoncrawl_fetch_recordRead one archived page back out of Common Crawl by byte range — pass the filename, offset and length from a commoncrawl_index_search result and get the WARC headers, the captured HTTP response headers, and the page body as it was crawled. AUTHORITATIVE for "what did this page actually say when it was crawled", including pages that have since changed or gone offline. The body is truncated to a byte cap you control. Keyless.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| length | Yes | Compressed byte length of the record (the capture row's "length"). Max 10485760 (10 MB). | |
| offset | Yes | Byte offset of the record within that WARC file (the capture row's "offset"). | |
| filename | Yes | WARC path from a capture row, e.g. "crawl-data/CC-MAIN-2025-05/segments/1736703361969.6/warc/CC-MAIN-20250112194358-20250112224358-00433.warc.gz". | |
| max_body_bytes | No | Truncate the decoded page body to this many bytes before returning it. Default 20000, max 200000. WARC headers and HTTP headers are always returned in full. |