navigate
Fetch and parse web pages with optional JavaScript execution, auto-extract embedded JSON, and solve browser challenges, returning structured BlockMap data for LLM agents.
Instructions
Fetch a URL with Chrome-fingerprinted HTTP using the active profile. Parses HTML, seeds the JS DOM, returns BlockMap inline. With exec_scripts: true, extracts inline AND external tags from the parsed HTML, fetches externals in parallel (8s per-fetch timeout), eval's them in document order in QuickJS (with shims for setTimeout/fetch/etc.), then settles the event loop and fires DOMContentLoaded + load. <script async> is honored: async scripts execute after the sync queue. When --policy=blocklist is set, tracker URLs are blocked at script-fetch time (see scripts.policy_blocked in the result). Returns a scripts summary with inline_count, external_count, async_count, policy_blocked, executed, errors.
Auto-extract: when the page embeds JSON-bearing tags (density.json_scripts > 0 — covers application/json, application/ld+json, text/x-magento-init, text/x-shopify-app, etc.), navigate auto-runs extract() and returns the result as the extract field. Saves a round trip on the common case where the data the JS would have rendered is already sitting in the HTML — JSON-LD article schemas on news sites, NEXT_DATA page state on Next.js apps, json_in_script product blobs on Magento/Shopify, GitHub RSC payloads, etc. Capped at 16 KB inline; over that limit extract returns a stub with strategy/confidence/size_bytes/hint and the agent should call extract() explicitly to retrieve the full payload. Pages with no embedded JSON get extract:null and pay zero extra cost.
Tool advice: navigate also returns tool_likelihoods plus tool_recommendations, derived from concrete page signals (structure/headings, selector hints, density, embedded data, network captures, challenge state, and script pathology) so agents can pick the next tool without guessing.
Auto-solve: Reddit's JS proof-of-work challenge (provider: reddit_js_challenge) is transparently solved — the challenge is detected, the GET solution URL is computed (solution = hex_value + hex_value), and the real page is returned in one navigate call. challenge:null on the result means the real page was served. Subsequent navigations in the same session carry the clearance cookie and skip the challenge entirely.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | Absolute URL to fetch | |
| exec_scripts | No | Run page <script> tags (inline + external src) after parse, settle the event loop, and fire DOMContentLoaded + load. Default false. | |
| include_ascii | No | Include human-readable ASCII blockmap grid (redundant with JSON fields). Default false. |