MCP Documentation Crawler
Server Configuration
Describes the environment variables required to run the server.
| Name | Required | Description | Default |
|---|---|---|---|
| MCP_CRAWLER_DATA_DIR | No | Directory to store the local index. | |
| MCP_CRAWLER_MAX_PAGES | No | Maximum number of pages to crawl. | |
| MCP_CRAWLER_START_URL | No | The starting URL for the crawl. | |
| MCP_CRAWLER_CONCURRENCY | No | Number of concurrent requests. | |
| MCP_CRAWLER_ALLOWED_DOMAINS | No | Comma-separated list of allowed domains. | |
| MCP_CRAWLER_URL_PATH_PREFIX | No | URL path prefix to restrict crawling. | |
| MCP_CRAWLER_REQUEST_DELAY_MS | No | Delay between requests in milliseconds. | |
| MCP_CRAWLER_MAX_CONTENT_CHARS | No | Maximum number of characters to index per page. |
Instructions
Guidance the server publishes about itself, which clients place ahead of the tool catalog so the model reads it before choosing anything.
This server publishes no instructions, or was last inspected before Glama recorded them.
Capabilities
Features and capabilities supported by this server
Protocol revision2025-11-25
| Capability | Details |
|---|---|
| tools | {} |
Tools
Functions exposed to the LLM to take actions
| Name | Description |
|---|---|
| crawl_documentationA | Crawl the configured documentation site and save a local searchable index. Use refresh=false to reuse an existing index. |
| search_documentationA | Search the locally crawled documentation index. Crawl first if no index is available. |
| get_documentation_pageB | Retrieve a complete indexed documentation page by URL. |
| crawler_statusA | Show active crawler configuration and whether a local documentation index exists. |
Prompts
Interactive templates invoked by user choice
| Name | Description |
|---|---|
No prompts | |
Resources
Contextual data attached and managed by the client
| Name | Description |
|---|---|
No resources | |
TDQS
Scored across 4 tools
Each tool has a distinct purpose: status checking, crawling, searching, and page retrieval. No overlap or ambiguity exists between them.
Most tools follow a clear verb_noun pattern (crawl_documentation, search_documentation, get_documentation_page), but crawler_status uses a noun_noun style. Still, the snake_case naming is consistent and readable.
With exactly 4 tools, the set is well-scoped for a documentation crawler. Each tool serves a necessary function and the count is appropriate for the server's purpose.
The core workflow of crawling, searching, and fetching pages is covered. A minor gap is the lack of an explicit clear/delete index operation, but refresh=false in crawl_documentation mitigates this.