Scrape
scrapeFetch a webpage and return its text content synchronously, with optional country proxy routing or cache bypass for fresh results.
Instructions
scrape
Scrape a webpage and return the text content synchronously — the only endpoint here that doesn't return a run to poll. Unchanged from the legacy API (response uses the legacy request_status format).
Quick example
curl -X POST https://api.riveterhq.com/v1/scrape \
-H "Authorization: Bearer YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{"url": "https://example.com"}'Credit costs
With proxy: 0.30 credits
Without proxy: 0.10 credits
From cache: 0.01 credits
Proxy usage
Scraping is not guaranteed to succeed without a proxy. Some websites may block requests or require specific geographic locations. To use a proxy, include proxy_country_code with a two-character country code (e.g. 'us', 'gb', 'de').
Caching
Recently scraped pages are cached to save credits (0.01 credits on a cache hit). Set skip_cache: true to always fetch fresh content.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | The URL to scrape | |
| skip_cache | No | Set to true to bypass cache and always fetch fresh content | |
| max_credits | No | Credit ceiling. Refused with 422 credit_cap_exceeded (nothing charged) when the estimate's maximum exceeds it. | |
| estimate_cost | No | Validate and price the request without running or charging anything; the 200 response is the credit estimate (EstimateCostResult). | |
| proxy_country_code | No | Optional two-character country code for proxy (e.g. 'us', 'gb', 'de') |