searxng-mcp-scraper
Server Configuration
Describes the environment variables required to run the server.
| Name | Required | Description | Default |
|---|---|---|---|
| MCP_HOST | No | Bind host when --transport streamable-http | 127.0.0.1 |
| MCP_PORT | No | Bind port when --transport streamable-http | 8765 |
| LOG_LEVEL | No | Stderr log verbosity: DEBUG/INFO/WARNING/ERROR | INFO |
| USER_AGENT | No | User-Agent header for all requests | searxng-mcp-scraper/0.1 |
| FETCH_MAX_BYTES | No | Refuse to read more than this many bytes per fetch | 5000000 |
| FETCH_TIMEOUT_S | No | HTTP timeout (seconds) for fetch | 20 |
| MCP_CORS_ORIGINS | No | Comma-separated CORS allow-origins. Use * to allow any origin (not recommended on LAN). | http://localhost,http://localhost:*,http://127.0.0.1,http://127.0.0.1:* |
| SEARCH_TIMEOUT_S | No | HTTP timeout (seconds) for search | 30 |
| SEARXNG_BASE_URL | Yes | Base URL of your SearXNG instance, e.g. http://localhost:8888 | |
| FIRECRAWL_API_KEY | No | Optional. Empty uses the keyless free tier | |
| FIRECRAWL_BASE_URL | No | Firecrawl API base — the local keyless proxy by default | http://127.0.0.1:8788 |
| FETCH_ALLOW_PRIVATE | No | Allow fetching private/loopback URLs (SSRF guard, default on) | false |
| FETCH_MAX_REDIRECTS | No | Max HTTP redirects to follow | 5 |
| FIRECRAWL_TIMEOUT_S | No | Timeout for a fallback scrape (Firecrawl renders before responding) | 60 |
| BLOG_SCRAPE_MAX_POSTS | No | Max post pages per blog scrape | 200 |
| BLOG_SCRAPE_OUTPUT_DIR | No | Directory for scrape_blog / deep_scrape Markdown output and assets | ~/scraped-blogs |
| SEARCH_MAX_RESULTS_CAP | No | Hard cap on results per call | 50 |
| BLOG_SCRAPE_CONCURRENCY | No | Max parallel post fetches | 5 |
| DEEP_SCRAPE_CONCURRENCY | No | Max parallel document/image downloads during deep_scrape | 5 |
| SEARXNG_DEFAULT_ENGINES | No | Default engine list when a caller doesn't pass engines. Curated to the keyless engines that return results from this machine's egress IP (Brave/Google/Startpage/DuckDuckGo are rate-limited here). Set to empty to use SearXNG's own default set. | wiby,naver,seznam,marginalia,wikipedia,duckduckgo_html |
| FIRECRAWL_MIN_TEXT_CHARS | No | Static extractions shorter than this are treated as failures and retried | 500 |
| SEARXNG_DEFAULT_CATEGORY | No | Default search category | general |
| SEARXNG_DEFAULT_LANGUAGE | No | Default language code | en |
| FIRECRAWL_FALLBACK_ENABLED | No | Retry a failed/thin static fetch through Firecrawl (see below) | true |
| SEARXNG_DEFAULT_SAFESEARCH | No | Default safesearch: 0/1/2 | 0 |
| BLOG_SCRAPE_DISCOVERY_PAGES | No | Max HTML index pages to scan when no feed is found | 10 |
| DEEP_SCRAPE_MAX_IMAGE_BYTES | No | Refuse a single image above this size | 15000000 |
| DEEP_SCRAPE_MAX_DOCUMENT_BYTES | No | Refuse a single linked document above this size | 50000000 |
| DEEP_SCRAPE_MAX_IMAGES_PER_BLOG | No | Max images downloaded by one deep_scrape call | 200 |
| DEEP_SCRAPE_MAX_DOCUMENTS_PER_BLOG | No | Max linked documents downloaded/extracted by one deep_scrape call | 100 |
Instructions
Guidance the server publishes about itself, which clients place ahead of the tool catalog so the model reads it before choosing anything.
This server publishes no instructions, or was last inspected before Glama recorded them.
Capabilities
Features and capabilities supported by this server
Protocol revision2025-11-25
| Capability | Details |
|---|---|
| tools | {
"listChanged": false
} |
| prompts | {
"listChanged": false
} |
| resources | {
"subscribe": false,
"listChanged": false
} |
| experimental | {} |
Tools
Functions exposed to the LLM to take actions
| Name | Description |
|---|---|
| searchA | Search the web via the configured SearXNG instance. |
| fetchA | Fetch a URL and return its cleaned main text content. |
| scrape_blogA | Discover every post on a blog and write a single aggregated Markdown file. |
| deep_scrapeA | Discover every post on a blog, fetch each, AND download + extract every linked document (PDF/DOCX/XLSX/PPTX/CSV/JSON/XML/MD/EPUB/...). |
Prompts
Interactive templates invoked by user choice
| Name | Description |
|---|---|
No prompts | |
Resources
Contextual data attached and managed by the client
| Name | Description |
|---|---|
No resources | |
TDQS
Scored across 4 tools
The tools are mostly distinct: search performs queries, fetch retrieves a single URL, while scrape_blog and deep_scrape both focus on blog content aggregation. The overlap between scrape_blog and deep_scrape is clear (deep_scrape is an extended version), but both descriptions are detailed enough to avoid confusion in practice.
The names mix conventions: 'search' and 'fetch' are single generic verbs, 'scrape_blog' follows a verb_noun pattern, and 'deep_scrape' is an adjective_verb compound. While each name is readable and descriptive, the lack of a uniform pattern makes the set feel slightly inconsistent.
With only 4 tools, the server is well-scoped for its purpose. Each tool covers a distinct aspect of web scraping and search: querying, single-page extraction, blog aggregation, and deep scraping with document extraction. No tool feels redundant or unnecessary.
The tool surface covers the core workflows: web search, URL fetching, blog scraping, and extended scraping with document extraction. Minor gaps exist, such as a generic crawling tool for non-blog sites, but these are not critical given the server's stated focus on search and blog content.