webcrawl-mcp
Server Configuration
Describes the environment variables required to run the server.
| Name | Required | Description | Default |
|---|---|---|---|
| USER_AGENT | No | HTTP User-Agent | Mozilla/5.0 (compatible; WebcrawlMCP/1.0; …) |
| POLITE_MODE | No | On a 429 with Retry-After, retry once after the indicated wait. | true |
| REQUEST_TIMEOUT | No | Seconds before request timeout | 30 |
| FIRECRAWL_API_KEY | No | If set, enables Firecrawl fallback for low-quality extractions. Leave unset for a fully free setup. | |
| FIRECRAWL_API_URL | No | Firecrawl endpoint | https://api.firecrawl.dev/v1 |
| FALLBACK_ON_TRANSPORT_ERROR | No | If true and FIRECRAWL_API_KEY is set, route bot-blocked statuses to Firecrawl instead of raising. | false |
Instructions
Guidance the server publishes about itself, which clients place ahead of the tool catalog so the model reads it before choosing anything.
This server publishes no instructions, or was last inspected before Glama recorded them.
Capabilities
Features and capabilities supported by this server
Protocol revision2025-11-25
| Capability | Details |
|---|---|
| tools | {
"listChanged": true
} |
| logging | {} |
| prompts | {
"listChanged": false
} |
| resources | {
"subscribe": false,
"listChanged": false
} |
| extensions | {
"io.modelcontextprotocol/ui": {}
} |
| experimental | {} |
Tools
Functions exposed to the LLM to take actions
| Name | Description |
|---|---|
| webcrawl_scrapeA | Fetch a URL and extract main content as markdown. |
| webcrawl_searchC | Search the web using DuckDuckGo. |
| webcrawl_mapA | Discover URLs on a website. Fetches the given URL and extracts all same-domain links. |
| webcrawl_crawlA | Crawl multiple pages starting from a URL. Uses BFS to discover and fetch pages up to max_depth links away. Respects rate limiting between requests. |
Prompts
Interactive templates invoked by user choice
| Name | Description |
|---|---|
No prompts | |
Resources
Contextual data attached and managed by the client
| Name | Description |
|---|---|
No resources | |
Latest Blog Posts
- Who's Calling? MCP Hosts Are an Identity Blind Spot (And the Spec Knows It)By Om-Shree-0709 on .mcpAgent IdentityOAuth 2.1
- Your AI Chatbot Just Exposed Your CEO's Salary to an InternBy Om-Shree-0709 on .Agent IdentityMCP SecurityOAuth Delegation
- Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)By Om-Shree-0709 on .Agentic AiPrompt InjectionWebAssembly
MCP directory API
We provide all the information about MCP servers via our MCP API.
curl -X GET 'https://glama.ai/api/mcp/v1/servers/andyliszewski/webcrawl-mcp'
If you have feedback or need assistance with the MCP directory API, please join our Discord server