Scout a url: reach the page (as markdown), map the site's url patterns, find its listing page, and return what Scout has learned about the host. want is an optional regex the page must carry (a model number, 'price|$', a heading); without it, good = real content that is not a wall. On failure page.error says why and what to try.

Reach one page and judge whether it is good information. Tries the host's learned readers winner-first, then scored url rewrites and detail sub-pages. Returns ok, markdown (excerpt unless full=true), the trail tried, and on failure an error {code, message, next_steps}. A page read today comes from Scout's mirror (fresh=true reads it again). When the site walls Scout, or is on hold, the latest Wayback Machine copy is read instead, and archived says when it was captured. cached=true reads only Scout's copy of the site (see cache) and sends no request.

A site's real urls (robots.txt sitemaps, a polite link walk, the catalogue's root) with their url patterns. filter is a regex. A listing is reused for 12 hours; fresh=true lists the site again (only when it changed: re-listing a whole site run after run is how crawlers get blocked). archive='only' lists the site from the Wayback Machine's index without a single request to it; 'auto' does so when the site is on hold or listed nothing. Use it instead of guessing paths.

Which of a page's link families lead to good pages, and how many. Each family (links whose paths differ only in their numbers, like /chapter/1/ and /chapter/14A/) is sampled across its spread, the samples vote on what the family leads to (an index of more links, or pages), indexes are walked one level deeper, and families are ranked by yield through every level. best.recipe_hint is a draft recipe for compile. Use it on sites with no useful sitemap.

Keep every agent sharing Scout's memory off a host, or let them back. Use it when a site blocked you and a person has asked for the block to be lifted: reason says what was asked and when. min_interval (seconds) slows Scout's pace on that host for good. Scout holds a host by itself after a block page or a 429, and probes once when that ends.

Official routes and the enrollments they need: the checklist for whoever updates Scout. action=list: open requests, each with a todo that says where to sign up, what it costs, and where the key goes. action=request host info: record a site's official route. info = {route: open_api|bulk|feed|public_json|alternative (no signup) or enroll_free|enroll_paid|enroll_oauth|commercial (a person must enroll), name, signup_url, docs_url, base_url, auth: {kind: none|query|header|bearer|oauth2_client_credentials, name, token_url, headers}, cost, terms, sample (a url that returns the data; leave the key out)}. Keyed routes become an enrollment request. action=applied (signed up, waiting on approval) | done (Scout tests the key on the sample, then marks enrolled) | decline. Keys are never passed here: they go in the keys file or SCOUT_KEY_ env vars.

The moves register: rewrite rules Scout tries where its built-in moves fail, each scored (wins+1)/(tries+2) on real reaches and retired after 5 straight losses. action=list [kind, host] | propose kind scope spec [note] | retire id. kinds: url_rewrite {pattern, repl} (regex on the full url; repl is literal text plus \1 groups) · locale_prefix {prefix} and listing_root {path}, which compose into listing paths (/en-us + /products) and are ranked, never retired · listing_path {path} (a whole path) · detail_suffix {suffix}. scope: '*', a domain ('example.com' covers www.example.com) or a host regex. Propose only what you saw work.

What Scout has learned about a host: winning reader, reader order, routes (listing, detail_suffix, patterns), the moves that apply, recent failures. No host: recent failures everywhere and the whole register.

Compile what you worked out about a site into a recipe a small model can run in one call. Scout runs it on every example and saves it only if all pass; the reply carries the one-line card for the small model. name: '/'. input_name: what the caller supplies ('model', 'sku'; empty for none). examples: 2+ inputs. steps, run in order, each a one-key object: {"input": [[regex, repl], ...], "lower": true} rewrite the input (slug rules) {"map": "https://site", "filter": "/product/{input}$"} the site's own urls; first match becomes {url} {"reach": "https://site/p/{input}" | "{url}", "want": regex} read a page {"find": regex} first link on the page matching (url or text) becomes {url} {"links": "/chapter//"} a link family from families(); last step = all its links, else the first is {url} {"jsonld": {"price": "offers.price", "sku": "sku"}, "type": "Product"} from the page's schema.org data {"extract": {"field": "regex with one group", "optional?": "..."}} every field must be found, except a name ending in ? Prefer map over a guessed url template: sites are inconsistent about case and slugs.

Run a compiled recipe. Reply with the RESULT line. inputs runs it over many inputs (one RESULT line each); cached=true reads only Scout's copy of the site, sending no request (use after cache).

Cached mode, for pulling a lot from one site: list it once (filter narrows the urls) and download every page once into Scout's copy, in the background, at the host's pace. source='archive' downloads from the Wayback Machine with no request to the site; 'live' from the site; 'auto' live, with the archive where the site walls Scout. The copy is kept days; reads and recipe runs with cached=true use only it. action=status follows the job; action=stop ends it.

The compiled recipes (optionally for one host): what each takes, its score, whether its last run failed, and the one-line card a small model runs.

US state codes read whole from each legislature's own site. No state: the list of states Scout can read, with each one's citation form and source. A state (e.g. 'us-tx'): its first limit sections (max 50) as citation, heading, text and url. A whole code is tens of thousands of sections: run scout-mcp laws <state> --out file.jsonl for that.