Schema | MCP Documentation Crawler

MCP Documentation Crawler

Describes the environment variables required to run the server.

Name	Required	Description
`MCP_CRAWLER_DATA_DIR`	No	Directory to store the local index.
`MCP_CRAWLER_MAX_PAGES`	No	Maximum number of pages to crawl.
`MCP_CRAWLER_START_URL`	No	The starting URL for the crawl.
`MCP_CRAWLER_CONCURRENCY`	No	Number of concurrent requests.
`MCP_CRAWLER_ALLOWED_DOMAINS`	No	Comma-separated list of allowed domains.
`MCP_CRAWLER_URL_PATH_PREFIX`	No	URL path prefix to restrict crawling.
`MCP_CRAWLER_REQUEST_DELAY_MS`	No	Delay between requests in milliseconds.
`MCP_CRAWLER_MAX_CONTENT_CHARS`	No	Maximum number of characters to index per page.

Features and capabilities supported by this server

Capability	Details
`tools`	{}

Functions exposed to the LLM to take actions

Name	Description
crawl_documentationA	Crawl the configured documentation site and save a local searchable index. Use refresh=false to reuse an existing index.
search_documentationA	Search the locally crawled documentation index. Crawl first if no index is available.
get_documentation_pageB	Retrieve a complete indexed documentation page by URL.
crawler_statusA	Show active crawler configuration and whether a local documentation index exists.

Interactive templates invoked by user choice

Name	Description
No prompts

Contextual data attached and managed by the client

Name	Description
No resources

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/codecracker2020/mcp-server-crawler'

If you have feedback or need assistance with the MCP directory API, please join our Discord server