doc-scraper
Server Configuration
Describes the environment variables required to run the server.
| Name | Required | Description | Default |
|---|---|---|---|
No arguments | |||
Instructions
Guidance the server publishes about itself, which clients place ahead of the tool catalog so the model reads it before choosing anything.
This server publishes no instructions, or was last inspected before Glama recorded them.
Capabilities
Features and capabilities supported by this server
Protocol revision2025-11-25
| Capability | Details |
|---|---|
| tools | {
"listChanged": true
} |
| logging | {} |
Tools
Functions exposed to the LLM to take actions
| Name | Description |
|---|---|
| cancel_crawlA | Cancel a running or pending crawl job by job ID. Has no effect on jobs already in a terminal state. |
| crawl_siteA | Start a background crawl for a configured site. Returns immediately with a job ID. |
| describe_serverA | Returns server identity, configured sites, and recent crawl jobs in one call. Call this first to orient yourself: it consolidates what would otherwise require list_sites plus several get_job_status calls. The MCP tool list is already advertised by the protocol so it is not duplicated here. |
| diff_crawlA | Return added/removed/changed pages between the latest crawl and the most recent crawl whose crawl_ended_at <= since. Hash-based verdicts from the SQLite history index. Pair with get_freshness: pass the last_crawl_ended_at value as since after running crawl_site with incremental set to true to see exactly what changed. |
| get_freshnessA | Return the most recent crawl summary for a site (last_crawl_started_at/ended_at, total_pages, mode, age_seconds) plus output/state dir presence and any running job. Use this to decide whether to query the existing crawl or run crawl_site first. |
| get_job_statusB | Get the status of a crawl job |
| get_pageA | Fetch a URL live over the network and return its content as markdown. This is an on-demand fetch independent of any crawl: it does not read the stored crawl output, and it ignores the site's configured content_selector and scope. |
| list_pagesA | List crawled pages for a site, paginated and sorted by URL. Returns metadata only (URL, title, depth, crawled_at, content_length). Pass any URL returned here to read_page to get its stored markdown; get_page re-fetches a URL live rather than returning the crawled copy. |
| list_sitesA | List all configured sites available for crawling |
| read_pageA | Return a page's markdown from the stored crawl output, without any network access. This is the counterpart to get_page: read_page serves the crawled copy that already had the site's content_selector applied, while get_page re-fetches the URL live. Use list_pages to discover URLs, then read_page to read them. Large pages are truncated at max_bytes; follow next_offset to read the rest. |
| search_docsA | Full-text search across all crawled documentation, ranked by relevance (BM25 with stemming), with zero network access. Results carry the page URL, its section heading path, and a snippet with match terms marked [like this]; follow up with read_page for the full page. Supports FTS5 syntax: quoted phrases, OR, and trailing * for prefix matching. Searches the stored index only; run crawl_site first for uncrawled sites. |
Prompts
Interactive templates invoked by user choice
| Name | Description |
|---|---|
No prompts | |
Resources
Contextual data attached and managed by the client
| Name | Description |
|---|---|
No resources | |
TDQS
Scored across 11 tools
Each tool has a distinct role, and the get_page/read_page vs list_pages/search_docs boundaries are clearly explained. The only mild overlap is get_freshness and get_job_status, and describe_server partly duplicates list_sites/job status, though it is positioned as an orientation call.
All tools use snake_case verb_noun names like list_sites, crawl_site, cancel_crawl, read_page, and search_docs. There is no mixing of styles or vague verbs; even diff_crawl follows the verb-first pattern.
11 tools is right-sized for a documentation crawling and retrieval server. Each tool maps to a meaningful operation without redundant surfaces.
The crawl-and-read workflow is fully covered: start/cancel/status, list/read/search stored pages, and diff/freshness for updates. The main gap is that site management is limited to listing pre-configured sites, with no tool to add, remove, or update site definitions, so agents cannot onboard new documentation sources without out-of-band configuration.