Arachne MCP
Server Configuration
Describes the environment variables required to run the server.
| Name | Required | Description | Default |
|---|---|---|---|
| MCP_HOST | No | Host untuk Streamable HTTP | 127.0.0.1 |
| MCP_PORT | No | Port untuk Streamable HTTP | 8000 |
| LLM_MODEL | No | Model LLM yang digunakan | |
| LLM_API_KEY | No | API key untuk LLM | |
| SEARXNG_URL | No | URL SearXNG server | http://localhost:8080 |
| LLM_BASE_URL | No | Base URL untuk LLM endpoint | |
| MCP_TRANSPORT | No | Transport MCP (stdio atau streamable-http) | stdio |
| ARACHNE_PROXY_URL | No | Proxy HTTP/HTTPS opsional | |
| ARACHNE_MAX_REQUESTS | No | Batas maksimum request per crawl | 200 |
| BRAVE_SEARCH_API_KEY | No | API key untuk Brave Search | |
| ARACHNE_RESPECT_ROBOTS | No | Mematuhi robots.txt | true |
| ARACHNE_MAX_TOTAL_CHARS | No | Batas output agregat | 250000 |
| ARACHNE_CACHE_TTL_SECONDS | No | TTL cache | 3600 |
| ARACHNE_ALLOW_BROWSER_EVAL | No | Mengizinkan arbitrary browser JS | false |
| ARACHNE_ALLOW_IGNORE_ROBOTS | No | Mengizinkan caller memilih respect_robots=false | false |
| ARACHNE_BROWSER_CONCURRENCY | No | Jumlah context Chromium bersamaan | 2 |
| ARACHNE_BLOCK_PRIVATE_NETWORKS | No | Proteksi SSRF | true |
Capabilities
Features and capabilities supported by this server
| Capability | Details |
|---|---|
| tools | {
"listChanged": false
} |
| prompts | {
"listChanged": false
} |
| resources | {
"subscribe": false,
"listChanged": false
} |
| experimental | {} |
Tools
Functions exposed to the LLM to take actions
| Name | Description |
|---|---|
| healthA | Return enabled engines, integrations, security policy, and parser availability. |
| crawl_urlB | Fetch and extract one URL using fast HTTP, browser rendering, or automatic escalation. Supports HTML, text, JSON, XML, PDF, DOCX, XLSX, CSV and TSV. Content returned to an agent is marked as untrusted and scanned for common prompt-injection text. |
| parse_urlC | Parse a remote document or non-HTML resource into LLM-ready text/Markdown. |
| crawl_siteB | Crawl a site with strict request/error/token budgets and per-host rate limiting. |
| map_siteC | Discover site URLs from recursive sitemaps plus optional shallow crawling. |
| extract_linksC | Extract normalized crawlable links from static or JavaScript-rendered pages. |
| check_robotsC | Inspect robots.txt permissions, crawl delay, request rate, and sitemap declarations. |
| browser_interactA | Operate a browser using safe declarative actions: click, fill, press, select, wait, scroll, and goto. JavaScript evaluate is disabled unless the server owner explicitly enables it. This tool does not bypass CAPTCHA, authentication, paywalls, or access controls. |
| batch_scrapeA | Scrape up to 500 URLs concurrently with a shared output budget. |
| search_webC | Search through a configured self-hosted SearXNG or Brave Search provider. |
| extract_structuredC | Extract JSON matching a caller-supplied schema through an optional OpenAI-compatible LLM. |
| crawl_diffB | Compare a page with its previous persistent snapshot and return a unified diff. |
| chunk_contentB | Split long content into stable, overlapping, hash-addressed chunks for RAG or agents. |
Prompts
Interactive templates invoked by user choice
| Name | Description |
|---|---|
No prompts | |
Resources
Contextual data attached and managed by the client
| Name | Description |
|---|---|
No resources | |
Latest Blog Posts
- Who's Calling? MCP Hosts Are an Identity Blind Spot (And the Spec Knows It)By Om-Shree-0709 on .mcpAgent IdentityOAuth 2.1
- Your AI Chatbot Just Exposed Your CEO's Salary to an InternBy Om-Shree-0709 on .Agent IdentityMCP SecurityOAuth Delegation
- Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)By Om-Shree-0709 on .Agentic AiPrompt InjectionWebAssembly
MCP directory API
We provide all the information about MCP servers via our MCP API.
curl -X GET 'https://glama.ai/api/mcp/v1/servers/Rizky742/crawl-mcp'
If you have feedback or need assistance with the MCP directory API, please join our Discord server