Cancel a running or pending crawl job by job ID. The previously stored corpus and crawl history are left untouched. Has no effect on jobs already in a terminal state.

Start a background crawl for a configured site. Returns immediately with a job ID. The crawl builds in a staging area and replaces the stored corpus only when it completes, so a cancelled or failed crawl leaves the previous corpus intact. A crawl that lost a large share of its pages is kept back instead (the job fails with refused set), and the stored corpus is kept.

Returns server identity, configured sites, and recent crawl jobs in one call. Call this first to orient yourself: it consolidates what would otherwise require list_sites plus several get_job_status calls. The MCP tool list is already advertised by the protocol so it is not duplicated here.

Return added/removed/changed pages between the latest crawl and a baseline crawl: the crawl before it by default, or with since, the most recent crawl whose crawl_ended_at <= since. Hash-based verdicts from the SQLite history index. To see what a crawl changed, run crawl_site, wait for it to finish, then call diff_crawl without since.

Draft a site entry and preview one page extracted with it. Never writes the config. With url, probes a new docs site the way doc-scraper add does (robots.txt, llms.txt, sitemap, framework detection). With only site_key, starts from that configured site's entry without probing. overrides sets entry fields by their config names (content_selector, allowed_path_prefix, disallowed_path_patterns, max_depth, ...); durations are strings such as "1s". Judge the preview (code blocks kept, nav_leak, chars), draft again with overrides until it looks right, then pass entry to save_site.

Return the most recent crawl summary for a site (last_crawl with its start and end times, total_pages and mode, plus age_seconds), output/state dir presence and any running job. Use this to decide whether to query the existing crawl or run crawl_site first.

Get the status and page counts of a crawl job. Set wait_seconds to wait for the job to finish instead of polling: the call returns as soon as the job completes, fails or is cancelled, or when the wait runs out, with the status at that moment.

Fetch an HTML page live over the network and return its content as markdown. This is an on-demand fetch independent of any crawl: it does not read the stored crawl output, and it ignores the site's configured content_selector and scope. Large pages are truncated at max_bytes; follow next_offset to read the rest (each call fetches the page again).

List crawled pages for a site, paginated and sorted by URL. Returns metadata only (URL, title, depth, crawled_at, content_length). Pass any URL returned here to read_page to get its stored markdown; get_page re-fetches a URL live rather than returning the crawled copy.

List all configured sites available for crawling

Return a page's markdown from the stored crawl output, without any network access. This is the counterpart to get_page: read_page serves the crawled copy that already had the site's content_selector applied, while get_page re-fetches the URL live. Use list_pages to discover URLs, then read_page to read them. Pass a search_docs result's anchor to read only that section. Large pages are truncated at max_bytes; follow next_offset to read the rest.

Write a site entry to the config file and load it into the running server, so crawl_site, search_docs and read_page can use it at once. entry is a full entry by config field name, normally draft_site's entry with changes; fields left out are unset. Overwriting a configured site needs replace: true. Only the entry's lines change in the file. Refused while the site is crawling, and on servers started without -allow-config-writes.

Full-text search across all crawled documentation, ranked by relevance (BM25 with stemming), with zero network access. Results carry the page URL, its section heading path, and a snippet with match terms marked [like this]; follow up with read_page for the full page. Supports FTS5 syntax: quoted phrases, OR, and trailing * for prefix matching. Searches the stored index only; run crawl_site first for uncrawled sites.