Scrape a page
pagesift_scrapeFetch a single known URL and return clean Markdown or structured content—HTML, links, images, or metadata—for AI and RAG workflows.
Instructions
Fetch one URL and return clean content: markdown by default, or html, rawHtml, links, images and metadata. Use it when you know the page you need. For several known URLs use pagesift_batch_scrape; to find URLs on a site use pagesift_map; to collect a whole site use pagesift_crawl. pagesift respects robots.txt and reports blocked pages as errors instead of returning their content.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | The page to scrape. | |
| proxy | No | Only basic and auto are supported; pagesift does not bypass bot protection. | |
| engine | No | pagesift extension: fetch with plain HTTP, a headless browser, or decide automatically (default). | |
| maxAge | No | Accept a cached copy at most this many milliseconds old; 0 forces a live fetch. | |
| mobile | No | ||
| actions | No | Browser actions to run before capturing the page. | |
| formats | No | Output formats. Defaults to ["markdown"]. | |
| parsers | No | ||
| profile | No | ||
| timeout | No | Timeout in milliseconds. | |
| waitFor | No | Milliseconds to wait before capturing the page. | |
| location | No | ||
| lockdown | No | ||
| redactPII | No | ||
| pdfOptions | No | ||
| excludeTags | No | ||
| ifNoneMatch | No | pagesift extension: ETag for a conditional request. | |
| includeTags | No | ||
| jsonOptions | No | Options for the json format. | |
| queryOptions | No | Options for the query format. | |
| storeInCache | No | ||
| ifModifiedSince | No | pagesift extension: HTTP date for a conditional request. | |
| onlyMainContent | No | Keep only the main content. Defaults to true. | |
| screenshotOptions | No | ||
| zeroDataRetention | No | ||
| removeBase64Images | No | ||
| skipTlsVerification | No |