Fetch one web page and return its content as clean markdown with metadata. Set js_render=true for pages that need JavaScript to render (SPAs, infinite-scroll listings, most modern storefronts). If a rendered page still comes back sparse, give it longer with wait_ms, or wait for a specific element with wait_for (a CSS selector) — that is more reliable than a fixed delay. Article-shaped pages have their navigation, cookie banners and footers stripped automatically; set main_content=false to keep the whole page. PDFs are extracted to text.

actions drives the page before reading it (implies js_render). Each is {action, selector, ...}: click, type (text), press (key), wait (ms), wait_for, scroll (times), scroll_to_bottom (max_rounds) for infinite scroll, and click_until_gone (max_clicks) for a 'load more' button. Use it for cookie gates, paginated listings and search forms.

Fetch many web pages in parallel. Returns JSON with results and errors. Set js_render=true for JavaScript-heavy pages. For more than about 20 URLs, pass save_as='' to write the pages into a result table and get back a summary instead of the full text — then use query_table to interrogate it without filling the conversation. When feeding a long URL list through in batches, pass mode='append' on every call after the first, or each batch replaces the last.

Discover the pages on a website. Prefers the site's sitemap (following sitemap indexes and robots.txt), and falls back to following links. Returns JSON with urls, count, source and truncated. Run this before fetch_pages to scrape a whole site.

Inspect a website before scraping it: detects the platform (Shopify, WordPress, Wix, ...), locates its sitemap, estimates how many pages it has, and reports whether JavaScript rendering is needed.

Scan a list of URLs for email addresses. Returns JSON with results ({url, value, context}) and errors. Pass save_as='' to store results instead of returning them inline.

Scan a list of URLs for phone numbers. Returns JSON with results ({url, value, context}) and errors. Pass save_as='' to store results instead of returning them inline.

Collect every hyperlink from a list of URLs, resolved to absolute URLs. Returns JSON with results ({url, value, context}) and errors. Pass save_as='' to store results instead of returning them inline.

Find social media profile links (LinkedIn, X, Facebook, Instagram, YouTube, TikTok, GitHub, Pinterest) across a list of URLs. Returns JSON with results ({url, platform, value}) and errors. Pass save_as='' to store results instead of returning them inline.

Extract structured product data (name, price, currency, availability, brand, sku, image, rating, review_count) from a list of product URLs. Reads the page's own JSON-LD, OpenGraph or microdata rather than guessing at selectors, so it works across most storefronts without configuration. Pass save_as='' to store the rows for querying.

Extract arbitrary fields from pages using CSS selectors — the general case the fixed extractors do not cover. fields maps output column names to selectors, e.g. {"name": "h1", "price": ".price"}. Set row_selector when a page holds a list: each match becomes a row and the field selectors resolve inside it, which turns a listing into a table. Suffix a selector with @attr to read an attribute instead of text — "a@href" gives the link, resolved to an absolute URL. Pass save_as to store the rows; add mode='append' when batching.

Search the web and get back ranked results with titles, URLs and snippets — the way in when you have a question rather than a URL. Feed the urls straight into fetch_pages or extract_by_selector. No API key and no quota.

Render a page and report the API calls it makes, with their JSON responses. A JavaScript site usually loads its data from an endpoint you can read directly — cleaner and far cheaper than parsing rendered markup, and it survives redesigns that break selectors. Use this when a page is hard to scrape, then fetch the endpoint it reveals. Set include_types to widen beyond JSON.

Fetch a page and report what changed since the last time it was checked. Returns status 'new', 'same' or 'changed', with a unified diff and line counts when it changed. This is monitoring without a scheduler: check a price, a job board or a policy page whenever you want and see only the difference.

List the pages being tracked for changes, with when each was last seen.

Stop tracking a page and forget its stored snapshot.

List a Shopify store's collections with their product counts. Use this to pick which collections to extract before calling extract_shopify_store.

Extract a Shopify store's catalogue as one row per product variant — price, sku, options, availability, vendor, image and product URL. Reads the store's public products.json, so it needs no rendering and no selectors. Pass save_as='' to store the rows (recommended: catalogues are large). max_products caps how many products are pulled.

Search for places and local businesses by name or description — 'the British Museum', 'cafes in Shoreditch'. Returns name, address, coordinates, phone, website, opening hours and category. Data comes from OpenStreetMap, so there are no star ratings or review counts; for those you would need a paid Google Places key. Pass save_as='' to store the results.

Find every business of a category within a radius of a place — 'dentists near Bath', 'cafes within 2km of Shoreditch'. This is the tool for local lead generation: it returns name, address, phone, website and opening hours for each. category accepts friendly names (cafe, dentist, hotel, solicitor, gym, hairdresser, ...) or a raw OpenStreetMap tag like 'amenity=dentist'. Data is OpenStreetMap, so there are no star ratings. Pass save_as='' to store the results.

List saved result tables with their row counts and columns. Result tables are produced by any tool called with save_as.

Inspect one result table: its columns, row count, and a small sample of rows.

Run a read-only SQL SELECT against saved result tables. This is how you analyse a large scrape without pulling it into the conversation: filter, aggregate, group and sort a table of any size and get back only the rows you asked for. Only SELECT is permitted — the query can never modify saved data. Example: SELECT vendor, COUNT(*) AS n, AVG(price) AS avg_price FROM catalogue GROUP BY vendor ORDER BY n DESC.

Pass save_as='' to persist the query's result as a new table. That is how you clean data here: SELECT DISTINCT deduplicates, aliases rename columns, a || ', ' || b AS c merges them, and WHERE drops unwanted rows — all in one step, with the original left untouched unless you deliberately target its name.

Export a result table to a file on disk as CSV or JSON, and return its path. Use this to hand data to a spreadsheet or another tool.

Delete a saved result table. This permanently removes the stored rows; the scraped pages themselves are unaffected.