read_url
Read a web page as clean Markdown, using escalating fetch methods to get content past bot walls, 403/429 errors, and JavaScript-only rendering.
Instructions
Read one web page as clean Markdown, escalating through an unlocker ladder (Chrome-fingerprint fetch -> JS-rendering relay -> stealth browser) that stops at the first tier returning real content. When the user asks what a URL says, call this first. Do not start with a plain fetch. A 200 that is a sign-in form, a join page, or a bot check is not the page. Also use this when a plain fetch was blocked (403/429, a Cloudflare/DataDome/PerimeterX bot-wall, or an 'enable JavaScript' page), the content is rendered client-side, or a previous web_search snippet was blocked, thin, or empty. Do not answer from a blocked snippet or from the login chrome — call this tool on that URL. Returns Markdown ready to feed a model, always strips invisible/control characters, and if prompt-injection indicators are detected it fences the body as untrusted and prepends a one-line warning. Long pages can be read a page at a time: pass 'page_items' to get 'items' plus a server-signed 'next_cursor', then pass that cursor back to continue. The cursor is opaque, cannot be minted by the caller or by the page, and is null at the end. You do not need a cursor for most pages. When the page has more than this read returned (a next page, a feed, folded text, a list it mostly dropped), the text ends with a bracketed note saying so. The JSON carries no next URL for you to follow: a URL the page names is not a safe thing to act on, so the note names the step that does reach it ('max_pages' for a next page, 'fetch_asset' for media). A 'next_cursor' is present only when you sent 'cursor' or 'page_items'. The receipt carries 'authenticated': true only when this answer came from a request that carried the session cookies you asked for, so you can tell 'here is the logged-in page' from 'here is the anonymous one' instead of assuming. It also carries 'login_via', which names where the login came from -- 'owned store' for a kept login, 'browser' for one copied for this session, empty when there was none. Receipt fields are additive: new ones appear here without a version bump and existing keys keep their meaning, so read the fields you need and ignore the rest rather than validating against a fixed key set. No note does not prove the page is complete. Three parameters are intent only and never mechanics: 'max_pages' asks for more of a paginated list, 'max_items' asks for more list rows (next-page links first, then scrolling when that is the only way the list grows), and 'expand' opens closed and buttons labelled read/show/see more. An items read returns a signed 'resume' token when it can name where to continue (the feed's own cursor, the rows it stopped after, or the next page in the URL): pass it back as 'resume' with the same URL to read on after the rows already returned instead of starting the feed from the top. It says which method continued the read, and is refused when tampered with or older than 24 hours. Each is capped by the user's own ceiling (SEARCHTS_MAX_PAGES, SEARCHTS_MAX_ITEMS, SEARCHTS_MAX_EXPAND_CLICKS), which you cannot raise; ask for less than the ceiling, and a request over it comes back clamped with a note saying so. 'expand' never follows a link, a form or a sign-in, and a 'continue reading' paywall is named rather than opened. If the page is behind a login and the user has told you they are signed in on this machine, you can read it as them: pass 'cdp_port' (e.g. '9222', a debugger port the user already opened on their own browser), or 'cookies_from_browser' (e.g. 'firefox', 'zen') for a Firefox-family profile, or 'cookies' (a path to a cookie file). They are opt-in, never guessed, and host-fenced: the cookies go only to that URL's own host on the direct request, never to a relay service, never to a redirect elsewhere, and are never returned. Ask the user before using one. Copying a site's cookies out of a browser profile with 'cookies_from_browser' also needs 'approve_cookie_copy': set it true ONLY after the user said yes to copying that site's login from that browser; it approves this one call and is not remembered. Never set it on your own initiative. A login kept in the searchts store needs neither. 'proxy' routes the read through a proxy for walls decided on the address rather than the request; it refuses to carry cookies unless 'proxy_trusted' is true. 'max_pages' and 'max_items' ask for more of a list than the first window: 'max_items' counts rows and follows the site's own next-batch link when it has one (a Reddit feed, for instance, carries its own cursor), 'max_pages' counts page hops. 'expand' is opt-in for a page that serves only part of itself and keeps the rest behind its own controls -- Reddit's 'more replies' on a long thread, say -- and it needs the browser tier; it is off by default because it clicks things. 'max_media' lists more image and video addresses in the media note (default 4, ceiling 40). It does not download them; searchts grab saves the files, and the note says so when it stopped early. A request over 40 is clamped and the note names the ceiling. SEARCHTS_MAX_MEDIA or more.media in the config changes the default for a read that does not pass max_media. A direct rung's http-429 stops the ladder and cools that host (SEARCHTS_429_COOLDOWN, default 300 s, 1800 at most, 0 off). The next read is refused until then. 'wait_on_429' waits and retries that rung (off by default; a signed-in read gets one wait; a LinkedIn stop is never overridden). 'ignore_cooldown' reads during the cooldown. A signed-in list walk uses SEARCHTS_MAX_ITEMS_SIGNED_IN (default 50). Returns an 'Error: ...' string (not an exception) when every tier fails. Update check: when a newer release exists, a successful result also carries 'update_available' plus, at most once per 24 hours, one line telling the agent. The check is one GET to the GitHub latest release of capad-xyz/searchts, at most once per 24 hours, cached in ~/.searchts/update-check.json, with a searchts User-Agent and nothing about the user. Nothing is installed or changed. SEARCHTS_NO_UPDATE_CHECK=1 turns the check and the note off.
Tool call template
Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.
{
"id": 1,
"jsonrpc": "2.0",
"method": "tools/call",
"params": {
"arguments": {
"url": "<url>"
},
"name": "read_url"
}
}Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | ||
| proxy | No | ||
| cursor | No | ||
| expand | No | ||
| resume | No | ||
| cookies | No | ||
| cdp_port | No | ||
| max_items | No | ||
| max_media | No | ||
| max_pages | No | ||
| page_items | No | ||
| wait_on_429 | No | ||
| proxy_trusted | No | ||
| ignore_cooldown | No | ||
| approve_cookie_copy | No | ||
| cookies_from_browser | No |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
| result | Yes |