searxNcrawl
Server Configuration
Describes the environment variables required to run the server.
| Name | Required | Description | Default |
|---|---|---|---|
| MCP_PORT | No | MCP proxy HTTP port | 9555 |
| SEARXNG_URL | No | Optional SearXNG URL with JSON output enabled; no default | |
| YDC_API_KEY | No | You.com API key; enables You.com search | |
| SEARXNG_PASSWORD | No | Optional SearXNG basic auth password | |
| SEARXNG_USERNAME | No | Optional SearXNG basic auth user; set with password | |
| SEARCH_RESULT_FIELDS | No | Comma-separated title,url,content,publishedDate,providers; unset means all | |
| SEARXNG_EXTRA_HEADERS | No | JSON object of SearXNG-only ASCII headers; secret-file references must exist inside the container |
Instructions
Guidance the server publishes about itself, which clients place ahead of the tool catalog so the model reads it before choosing anything.
A web crawler and search server that provides:
1. Web Crawling Tools:
- crawl: Crawl one or more URLs and get markdown content
- crawl_site: Crawl an entire website with depth/page limits
2. Web Search Tool:
- search: Search the web via the configured providers (SearXNG, You.com),
queried in parallel; results are merged and attributed per provider.
include_content=true also returns the Markdown of each result page
Output formats for crawl tools:
- markdown: Clean concatenated markdown (default)
- json: Full details including metadata and references
Capabilities
Features and capabilities supported by this server
Protocol revision2025-11-25
| Capability | Details |
|---|---|
| tools | {
"listChanged": true
} |
| logging | {} |
| prompts | {
"listChanged": true
} |
| resources | {
"subscribe": false,
"listChanged": true
} |
Tools
Functions exposed to the LLM to take actions
| Name | Description |
|---|---|
| crawlC | Crawl one or more URLs and extract content as markdown or JSON. |
| crawl_siteC | Crawl an entire website starting from a seed URL using DFS strategy. |
| searchA | Search the web via the configured search providers (SearXNG, You.com). All configured providers are queried in parallel. Their results are merged (URL dedup, round-robin, results found by several providers first) and each result lists the providers that returned it. |
Prompts
Interactive templates invoked by user choice
| Name | Description |
|---|---|
No prompts | |
Resources
Contextual data attached and managed by the client
| Name | Description |
|---|---|
No resources | |
TDQS
Scored across 3 tools
The three tools have mostly distinct roles: search queries providers, crawl fetches specific URLs, and crawl_site traverses a whole site. The crawl vs. crawl_site boundary is clarified by the DFS/seed-URL description, though some overlap remains because crawl accepts multiple URLs.
All names are lowercase snake_case and action-oriented, which is readable and predictable. However, search and crawl are bare verbs while crawl_site uses a verb_noun compound, so the convention is not perfectly uniform.
Three tools are well scoped for a search-and-crawl server: one search entry point, one targeted URL crawler, and one whole-site crawler. Each tool earns its place without obvious redundancy.
The surface covers the core domain lifecycle: searching configured providers, extracting content from specific URLs, and crawling an entire site from a seed URL. No obvious missing operation prevents agents from accomplishing search-and-crawl workflows.