Scrape URL
alterlab_scrapeRead-onlyIdempotent
Scrape one URL into clean, LLM-ready markdown, text, HTML, or JSON, with rendering and tier escalation handled automatically for JavaScript-heavy or blocked pages.
Instructions
Get one web page as clean, LLM-ready markdown (or text, HTML, JSON) in a single call; rendering and tier escalation are handled for you. Use when you have one URL and want its content; for an image use alterlab_screenshot, for 2-20 URLs alterlab_batch_scrape, for a whole site alterlab_crawl. Cost: from $0.0002 per page, up to $0.02 on the hardest pages; it starts at the cheapest tier that works and cache hits cost half. A 403/429, challenge page or empty JavaScript shell is retried on higher tiers automatically. Supports GET and POST via method, extraction via extraction_schema or extraction_prompt, cost caps via cost_controls, and browser actions via actions plus screenshot and generate_pdf; action_results has one entry per action with index, type, success, error, result (evaluate) and screenshot_url (screenshot actions). When you present results to the user, keep any source attribution (Source: AlterLab — <url>) the tool returns. A failure may carry next_action install_hands: reason, what Hands unlocks, install link, resume ref.
Tool call template
Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.
{
"id": 1,
"jsonrpc": "2.0",
"method": "tools/call",
"params": {
"arguments": {
"url": "<url>"
},
"name": "alterlab_scrape"
}
}Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| ocr | No | Extract text from images using OCR (+$0.001) | |
| url | Yes | URL to scrape | |
| body | No | Request body for POST requests. For GraphQL: JSON string with 'query' and optional 'variables' fields (e.g., '{"query": "{ user { id name } }"}').For REST APIs: JSON-encoded payload string. For form submissions: URL-encoded key=value pairs (e.g., 'name=Alice&email=alice@example.com'). Omit for GET requests. | |
| mode | No | Scraping mode: auto (recommended), html, js (headless browser), pdf, or ocr | auto |
| cache | No | Enable caching for this request. When true, repeat requests with identical parameters may return cached results. Only use for idempotent requests (GET pages, read-only POSTs). | |
| method | No | HTTP method for the request. Default GET (standard page scraping). Use POST for GraphQL endpoints, form submissions, and REST API calls. When using POST, provide body with the request payload. POST costs 1.5x base tier price. | GET |
| actions | No | Browser automation actions run in order after page load (max 20, requires render_js=true). Failed actions are skipped without aborting the sequence. Results are returned in action_results. Each action has a 'type': wait, wait_for, click, fill, press, scroll, infinite_scroll, screenshot, evaluate. | |
| cookies | No | Inline cookies as key-value pairs for authenticated scraping (e.g., {"session_token": "abc123"}). Use this for one-off requests; use session_id for reusable sessions. | |
| formats | No | Output formats. 'markdown' is best for LLM consumption. 'json_v2' returns a structured section tree (headings + content blocks). 'rag' returns chunked text optimized for retrieval-augmented generation. 'raw' returns the raw response body without extraction. 'content' returns body_markdown + content_hash + images + links for AI/KB pipelines. | |
| headers | No | Custom HTTP headers to send to the target URL. Hop-by-hop headers (Connection, Transfer-Encoding, Host, etc.) are not allowed. | |
| timeout | No | Request timeout in seconds (1-300) | |
| evidence | No | Include provenance/evidence snippets alongside extracted fields. Each extracted value will include the source text passage it was derived from. Requires extraction_schema or extraction_prompt. | |
| location | No | Geo-targeting parameters for localized content scraping. Controls proxy country routing, Accept-Language header, and browser locale. | |
| template | No | DEPRECATED alias for extraction_template, kept for backward compatibility. Sent to the API as extraction_template; prefer extraction_template. | |
| wait_for | No | CSS selector to wait for before extracting content (e.g., '#main-content') | |
| cache_ttl | No | Cache TTL in seconds (60–86400). Defaults to 3600 (60 min) when cache=true. Requires cache=true. | |
| fail_fast | No | Fail immediately if the page requires an expensive tier (browser/captcha) instead of auto-escalating. Shorthand for cost_controls.fail_fast. Returns an error with the required tier instead of upgrading. | |
| render_js | No | Render JavaScript using a headless browser (forces Tier 4 minimum, $0.004/req — no separate add-on charge). Required for JS-heavy sites. Set to 'auto' for smart detection (probes each page, only renders JS-heavy pages with a browser — saves 30-60% on mixed sites). | |
| use_proxy | No | Route through AlterLab's premium proxy (+$0.0002). Helps bypass geo-restrictions and anti-bot | |
| embeddings | No | Chunk the scraped text and embed it with OpenAI models (+$0.001 per page). Set enabled=true; model, chunk_size (64-8191) and chunk_overlap (0-2048) are optional. | |
| pdf_format | No | Output format for mode='pdf' extraction (API default 'markdown') | |
| screenshot | No | Capture a full-page screenshot (+$0.0002, requires render_js). The URL is returned as screenshot_url. Use alterlab_screenshot to get the image inline. | |
| session_id | No | UUID of a stored session for authenticated scraping. Use alterlab_list_sessions to find available sessions. The session's cookies will be injected into the request. | |
| wait_until | No | Page load event to wait for before extracting (networkidle, domcontentloaded or load; API default networkidle) | |
| prefer_cost | No | Optimize for lowest cost — try cheaper tiers first before escalating. Shorthand for cost_controls.prefer_cost. Mutually exclusive intent with prefer_speed. | |
| sync_policy | No | Synchronous wait policy. 'auto' (default) may return an early 202 with a job_id for predicted T3/T4 jobs; 'wait_until_deadline' keeps the connection open until the terminal result or the safe sync deadline (~85s). | |
| block_images | No | Block image downloads during browser rendering. Reduces proxy bandwidth and speeds up scrapes. Only effective with render_js=true. | |
| content_type | No | Content-Type header for the request body. Defaults to 'application/json' when body is provided. Use 'application/graphql' for raw GraphQL queries. Use 'application/x-www-form-urlencoded' for HTML form submissions. Requires body to be set. | |
| generate_pdf | No | Generate a PDF of the rendered page (+$0.0004, requires render_js) | |
| ocr_language | No | Tesseract language code for OCR (API default 'eng') | |
| prefer_speed | No | Optimize for speed — skip to a reliable tier immediately instead of escalating from Tier 1. Shorthand for cost_controls.prefer_speed. Mutually exclusive intent with prefer_cost. | |
| scroll_count | No | Number of scroll iterations when scroll_to_load is enabled (1-10, API default 3) | |
| cost_controls | No | Fine-grained cost and tier controls. Use to cap spending, pin a tier, or trade off cost vs speed. | |
| enable_scroll | No | Force scrolling for lazy-load images. true: always scroll (+5-10s), false: never scroll, omit: auto (scroll unless social media site). | |
| force_refresh | No | Bypass the cache and always fetch a fresh copy of the page. Use when you need real-time content and a cached result would be stale. | |
| proxy_country | No | ISO country code for proxy geo-targeting (e.g., 'US', 'DE'). Requires use_proxy=true | |
| use_own_proxy | No | Use your own proxy integration (bring-your-own-proxy) instead of the system proxy. BYOP requests get a 20% discount on the tier price. | |
| estimate_first | No | Run a cost estimate before scraping and include it in the response. Adds one lightweight API call (~50ms) with no credit charge. The estimated tier, cost, and confidence are prepended to the scrape result. Useful for unfamiliar or potentially expensive sites — see cost before committing. | |
| filter_content | No | Apply quality filtering to extracted content. When false (default), returns all parsed content without quality thresholds (lossless mode). When true, filters low-quality boilerplate. | |
| scroll_to_load | No | Scroll page to trigger lazy-loaded content (requires render_js). Performs explicit viewport-height scrolls to load dynamic content. Adds ~2-3s latency. | |
| section_filter | No | Filter for the json_v2 section tree. Only applies when 'json_v2' is in formats. | |
| sticky_session | No | Sticky-session handle. Scrapes that pass the same value reuse the cookie jar captured on the first call and, best-effort, the same exit IP (~10 min). Alphanumeric, '-' and '_' only. Requires a paid plan (growth tier or above). | |
| wait_condition | No | Wait condition for JS rendering: domcontentloaded, networkidle or load | |
| include_iframes | No | Inline iframe content into the main document (free, requires render_js). Same-origin iframes are read directly; cross-origin iframes are marked but not fetched. | |
| session_headers | No | Inline auth headers sent to the target (e.g. Authorization: Bearer ...). Merged with the stored session's headers when session_id is also set. | |
| extraction_model | No | Per-request LLM model override in provider-specific format (e.g. 'gpt-4o', 'claude-opus-4-5-20251101', 'llama3-70b-8192'). Overrides the model saved in your BYOK key settings for this request only. | |
| include_raw_html | No | Include raw HTML in the response alongside formatted content | |
| use_system_proxy | No | Override your default proxy integration and use AlterLab's system proxy for this request. | |
| extraction_prompt | No | Natural language extraction instruction. Describes what fields to extract from the page. Mutually exclusive with extraction_schema. Example: "Extract the product name, price, and availability". | |
| extraction_schema | No | JSON schema for structured extraction. The API extracts fields matching this schema from the scraped page using LLM (+$0.001). Result is returned in extraction_result. Example: { "title": "string", "price": "number", "in_stock": "boolean" } | |
| extraction_profile | No | Pre-built extraction schema template. auto: detect best template (API default). product: e-commerce product details. article: news/blog article fields. job_posting: job listing fields. faq: FAQ entries. recipe: recipe ingredients and instructions. event: event details. ecommerce_homepage: homepage product listings. directory_listing: directory/listing page entries. Mutually exclusive with extraction_template. | |
| flatten_shadow_dom | No | Flatten Shadow DOM roots into regular DOM for extraction (free, requires render_js). | |
| max_response_bytes | No | Soft cap on raw response body size in bytes. When the downloaded HTML exceeds this value it is truncated before extraction. Default: 5 MB (5242880). Set to 0 for no limit. Maximum: 50 MB (52428800). Useful for very large pages where you only need the beginning of the content. | |
| promote_schema_org | No | Use Schema.org JSON-LD/Microdata as the primary structured-data source when present (API default true). Promotes machine-readable metadata embedded in the page over LLM extraction. Most effective on e-commerce, recipe, and news article pages. | |
| sticky_session_ttl | No | Sticky-session cookie-jar lifetime in seconds (300-7200, API default 1800). Ignored unless sticky_session is set. | |
| extraction_provider | No | LLM provider to use for extraction. Selects the matching BYOK key registered at /dashboard/settings/llm-keys. When omitted, the most recently used registered key is used automatically. Requires extraction_schema or extraction_prompt. | |
| extraction_template | No | Shorthand alias for extraction_profile — selects the same pre-built template. Mutually exclusive with extraction_profile. | |
| proxy_integration_id | No | Specific proxy integration ID to use (requires use_own_proxy=true) | |
| remove_cookie_banners | No | Remove cookie consent banners from HTML before content extraction (free; the API default is true) | |
| skip_tls_verification | No | TLS certificate verification for the target. Omit (recommended): certificates are verified, and if the site's own certificate is invalid (expired, self-signed, wrong hostname, incomplete chain) the page is still fetched without verification. true skips verification from the start. false always requires a valid certificate, so sites with an invalid certificate fail. |