Scrape URL
metalift_scrapeTurn a single URL into clean markdown, HTML, or text, handling WAF, SPA, and e-commerce pages via auto strategy and proxy options for AI-ready content.
Instructions
Scrape a single URL into markdown, HTML, or text. Default: fast static article path (1 credit, response_detail=compact). E-commerce hosts auto-route to retail/residential. Pass strategy=auto for WAF/SPA pages. Fetch metalift://agent-guide for session workflow. You are solely responsible for complying with website terms, robots.txt, copyright, and data protection laws when using scraped content.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | Page URL to scrape | |
| proxy | No | Proxy tier; default direct for static pages | |
| render | No | Render mode; default static for plain URLs | |
| cookies | No | Session cookies (name→value). Routes through Playwright browser session + proxy — not static HTTP. Prefer metalift_seed_session for reuse. | |
| formats | No | Output formats; default markdown only | |
| headers | No | Extra request headers. Pass User-Agent from the user's browser when using cookie_header so the session fingerprint matches. | |
| strategy | No | Scrape strategy: auto, article, spa, cloudflare, authenticated, listing, retail, jsonld, download, raw, or comma-separated chain. Use download for full static HTML. See /v1/strategies for protection levels and credit estimates. | |
| wait_for | No | CSS selector to wait for before scraping | |
| screenshot | No | Capture page screenshot (adds cost) | |
| timeout_ms | No | Per-attempt timeout in milliseconds | |
| cookie_header | No | Raw Cookie header from DevTools. Routes through Playwright + sticky proxy to preserve browser TLS fingerprint. Prefer metalift_seed_session with storage_state for repeat scrapes. | |
| response_detail | No | Response depth: compact (default, ~16k chars, no links), standard (full markdown + capped links), full (complete JSON). | |
| only_main_content | No | Extract main article content only (default true) |