Schema | scrapy-mcp

scrapy-mcp

Describes the environment variables required to run the server.

Name	Required	Description	Default
`SCRAPY_MCP_JOB_DIR`	No	Where crawl jobs are stored.	<tmp>/scrapy_mcp_jobs
`SCRAPY_MCP_LOG_LEVEL`	No	Scrapy log level (to stderr).	ERROR
`SCRAPY_MCP_MAX_BYTES`	No	Max characters returned per page (then truncated).	50000
`SCRAPY_MCP_USER_AGENT`	No	User-Agent header.	scrapy-mcp/<version> …
`SCRAPY_MCP_OBEY_ROBOTS`	No	Obey robots.txt.	true
`SCRAPY_MCP_RETRY_TIMES`	No	Retries on transient failures.	2
`SCRAPY_MCP_AUTOTHROTTLE`	No	Adapt delay to server latency.	true
`SCRAPY_MCP_JOB_TTL_DAYS`	No	Delete crawl jobs older than this (0 disables).	7
`SCRAPY_MCP_MAX_DEPTH_CAP`	No	Hard cap for depth.	10
`SCRAPY_MCP_MAX_PAGES_CAP`	No	Hard cap for pages.	1000
`SCRAPY_MCP_DOWNLOAD_DELAY`	No	Seconds between requests to a host.	0.5
`SCRAPY_MCP_REQUEST_TIMEOUT`	No	Wall-clock cap for a blocking single fetch (s).	60
`SCRAPY_MCP_DOWNLOAD_TIMEOUT`	No	Per-request timeout (s).	30
`SCRAPY_MCP_DEFAULT_MAX_DEPTH`	No	Crawl depth default.	2
`SCRAPY_MCP_DEFAULT_MAX_PAGES`	No	Crawl page default.	50
`SCRAPY_MCP_CONCURRENT_REQUESTS`	No	Global concurrency.	8
`SCRAPY_MCP_CONCURRENT_REQUESTS_PER_DOMAIN`	No	Per-host concurrency.	4

Server capabilities have not been inspected yet.

Functions exposed to the LLM to take actions

Name	Description
No tools

Interactive templates invoked by user choice

Name	Description
No prompts

Contextual data attached and managed by the client

Name	Description
No resources

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/eitan3/Scrapy_MCP_Scraper'

If you have feedback or need assistance with the MCP directory API, please join our Discord server