Schema | site-crawler-mcp

site-crawler-mcp

Describes the environment variables required to run the server.

Name	Required	Description	Default
`DEEPSEEK_API_KEY`	No	Your DeepSeek API key for LLM summarization (default environment variable read by `summarize_page` tool)
`PLAYWRIGHT_BROWSERS_PATH`	No	Path to an existing Playwright browsers installation, as an alternative to running `python -m playwright install chromium`

Guidance the server publishes about itself, which clients place ahead of the tool catalog so the model reads it before choosing anything.

This server publishes no instructions, or was last inspected before Glama recorded them.

Features and capabilities supported by this server

Protocol revision2025-11-25

Capability	Details
`tools`	{ "listChanged": false }
`prompts`	{ "listChanged": false }
`resources`	{ "subscribe": false, "listChanged": false }
`experimental`	{}

Functions exposed to the LLM to take actions

Name	Description
crawl_siteA	BFS 全站爬取（谷歌爬虫式）：从 start_url 遍历站内所有内链。筛选：published_after/before（ISO 时间，发布时间优先、抓取时间兜底）、 title_contains/title_exclude（标题包含）、url_pattern（glob/regex 模式）。默认遵守 robots.txt 并按 0.5s/请求限速，可关闭。结果 JSON 返回并持久化到 SQLite。
scrape_pageA	抓取单个页面，返回 markdown、标题、发布时间与链接列表，并持久化。
summarize_pageA	概括页面内容。mode: auto（LLM 优先，失败回落本地）/ llm / local。 llm_provider 如 "deepseek/deepseek-chat" 或 "openai/gpt-4o-mini"（LiteLLM 格式）； llm_api_key_env 指定 API key 的环境变量名（默认 DEEPSEEK_API_KEY）。
click_linkC	点击页面内指定链接（CSS selector 或链接文本）并抓取目标页内容。
download_fileC	下载页面文件（图片/文档等）到 E:\Reasonix-项目\crawler-output。
query_crawlsC	查询已持久化的爬取结果（SQLite），供其它项目复用。

Interactive templates invoked by user choice

Name	Description
No prompts

Contextual data attached and managed by the client

Name	Description
No resources

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/coolaigit/site-crawler-mcp'

If you have feedback or need assistance with the MCP directory API, please join our Discord server