searxng-deepdive
Allows AI agents to perform web searches through a SearXNG instance, with support for engine-specific searches, category filtering, multi-page results, and reading URL content as Markdown.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@searxng-deepdive search arxiv for recent papers on transformers"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
searxng-deepdive
An MCP server for SearXNG designed for LLM agents doing real research. Four tools with agent-friendly schemas, multi-page result fanout, lightweight URL→Markdown reading, and tool descriptions generated dynamically from the live engine pool of your SearXNG instance.
Why another mcp-searxng?
Existing packages are minimal — most expose a single search(query) tool
with no way for the model to ask for more results, target specific engines,
or constrain by category. The richer ones bake static descriptions, so the
LLM never learns what's actually enabled on this instance. None of them
treat agent-tool-selection ergonomics as a design priority.
searxng-deepdive opens those knobs up:
Feature | This | npm | PyPI |
Engine targeting | ✅ via | ❌ | ❌ |
Category targeting | ✅ via | ❌ | ❌ |
Multi-page fanout in one call | ✅ via | ❌ (one page per call) | ❌ |
Pagination | ✅ via | ✅ | ❌ |
Compact response trim | ✅ via | ❌ | ❌ |
Dynamic descriptions per instance | ✅ live engine list injected | ❌ static | ❌ static |
Validation with cross-tool hints | ✅ engine-vs-category, case-insensitive | ❌ | ❌ |
Zero-result hints | ✅ time_range / unresponsive engines / single-engine | ❌ | ❌ |
URL reader (HTML→Markdown) | ✅ with TOC scan + section extraction | ✅ basic | ❌ |
Test suite | ✅ 102 unit + integration | minimal | ❌ |
Related MCP server: searxng-mcp
Quickstart
Install via npx -y from any MCP client:
{
"mcpServers": {
"searxng": {
"command": "npx",
"args": ["-y", "searxng-deepdive"],
"env": { "SEARXNG_URL": "http://127.0.0.1:7979/" }
}
}
}SEARXNG_URL should point at your running SearXNG instance. Need one?
The companion repo SearXNG-Compose
ships a plug-and-play Docker stack tuned for LLM consumption.
Requirements: Node.js 22 or newer.
Tools
The server registers four tools. The LLM picks among them based on the descriptions below, augmented at startup with the live engine and category list from your instance.
search(query, [...])
Broad web search across the full enabled engine pool. Use when you don't have a specific source preference. Returns merged, deduplicated results across however many engines respond.
search_on_engines(query, engines, [...])
Search using only the specified engines (e.g. ["arxiv", "pubmed", "semantic scholar"]).
The tool description registered with the MCP client includes the actual
list of engines enabled on your instance — agents don't have to guess
names. Validation rejects invalid names with a "did you mean" hint when
they look like categories instead of engines.
search_by_category(query, categories, [...])
Search within specific categories — runs every engine tagged with each.
Description includes the live category list and which engines belong to
each. Same validation: invalid category names produce a clear error
that points at search_on_engines when the offending value is actually
an engine name.
web_url_read(url, [readHeadings, section, paragraphRange, startChar, maxLength])
Fetch a URL and convert its HTML to clean Markdown. Lightweight HTTP + HTML→Markdown (no headless browser) — handles ~80% of the static-HTML web (Wikipedia, docs sites, blogs, news, GitHub READMEs).
Token-efficient extraction modes (priority order, first set wins):
readHeadings: true— return only the heading list as a hierarchical TOCsection: "Installation"— return content under matching headingparagraphRange: "3-7"— 1-indexed paragraph slicestartChar+maxLength— character window pagination
Recommended workflow for long pages: TOC scan first (readHeadings), then
targeted read (section). Far more token-efficient than fetching the full
page up front.
If readHeadings comes back with no entries (Reddit threads, comment
sections, blog posts that use bold paragraphs instead of <h*> tags),
the page is structurally flat — fall through to paragraphRange for
sequential sampling, or just fetch without an extraction mode.
web_url_read also accepts JSON, YAML, and TOML content-types directly
(spec files, package manifests, registry API responses, CI workflow
YAML), so research agents can read these without the HTML-only stub.
For JS-rendered SPAs and bot-protected sites this tool returns minimal/empty content — fall back to a Chromium-backed reader (e.g. Crawl4AI) for those.
Common parameters across all search tools
pageno— 1-indexed starting page (default 1)pages— multi-page fanout in one call (1–5, default 1)time_range—day/week/month/year(warning: not all engines support this; some return empty when set)language— BCP-47 code orallsafe_search— 0 / 1 / 2format—compact(default) orfull
Configuration
Env var | Default | Meaning |
|
| Base URL of the SearXNG instance |
Development
git clone <this repo>
cd searxng-deepdive # or wherever you cloned to
npm install
npm run build # tsc
npm test # vitest
SEARXNG_URL=http://127.0.0.1:7979 npm run probe # exercise the SearXNG client
SEARXNG_URL=http://127.0.0.1:7979 npm run dev # start the MCP stdio serverPointing an MCP client at the source during development
Use tsx to run from src/ directly so you don't need to rebuild on every edit:
{
"mcpServers": {
"searxng": {
"command": "npx",
"args": ["-y", "tsx", "/absolute/path/to/searxng-deepdive/src/index.ts"],
"env": { "SEARXNG_URL": "http://127.0.0.1:7979/" }
}
}
}MCP clients cache the subprocess. When you edit code, the running server keeps the old behavior until the subprocess is killed and respawned. Quit the host (LM Studio, Claude Desktop, etc.) fully and reopen — closing the chat window alone usually isn't enough. Symptom of not doing this: a fix you just shipped doesn't appear to take effect.
Testing
npm testTest coverage spans seven files:
normalize-name — case-insensitive name handling
validators — engine/category validation with cross-reference hints
zero-result-hint — every hint trigger and its inverse
trim-to-compact — response trimming + hint inclusion
descriptions — anti-pattern regex checks for the description copy that misled real models in earlier versions ("ignored by engines", "Default 'auto'", etc.) — failing build if they reappear
searxng-client — HTTP client with
MockAgent: malformed JSON, HTML 502 pages, 429 rate-limit handling, multi-page fanout dedup, all-pages-fail throwsurl-reader — extraction modes + HTTP integration
Design notes
Why four tools instead of one with optional engine/category params? Cleaner agent decision-making. With distinct tools the LLM sees explicit purposes; with one fat tool it has to remember when to set which optional flags. Trade-off: more entries in the MCP tool list, mostly identical handler code. Net: better agent ergonomics, especially for smaller models.
Why
format: "compact"as default? SearXNG's full result objects are several times heavier than just url+title+content+engine. For the typical agent workflow (rank candidates, pick a few to fetch in detail), the compact form is what the LLM actually uses.format: "full"is one parameter away when you need scores, dates, authors, or DOI.Why dynamic descriptions? Static descriptions either list every upstream engine (most aren't enabled on a given instance — wastes context) or list none (LLM has no idea what to put in
engines). Live introspection of/configat server startup gives the LLM exactly the right hint for this instance.Why convert silent-wrong into informatively-wrong? Real LM Studio testing showed agents repeatedly stuck in retry loops because failed searches looked successful (zero results, looked like "no matches"; or 60 garbage results, looked like the search ran). The validation + zero-result-hint pattern surfaces the actual cause every time. The description-anti-pattern test suite locks in copy that was empirically shown to mislead models.
Security notes
This package is designed to run locally, inside the user's trust boundary, alongside an MCP-speaking LLM client (Claude Desktop, LM Studio, Cursor, etc.). The trust model assumes:
the LLM is acting on the user's behalf
the user controls what model is connected to the server
the MCP transport is stdio, not exposed to remote callers
Within that boundary, two surfaces are worth knowing about:
web_url_readwill fetch any HTTP(S) URL the model hands it, with up to five redirects. On a host that can route to private networks, the model can therefore reach intranet services, link-local addresses, or cloud-instance metadata endpoints (169.254.169.254, etc.). This is by design for a local research tool but means you should not run this MCP server in topologies where an untrusted party can pick the URLs (e.g. a hosted MCP gateway facing the public internet). If you do want to lock it down, setSEARXNG_DEEPDIVE_BLOCK_PRIVATE=1to refuse private / loopback / link-local / metadata destinations — enforced on every redirect hop and against the actual dialed IP, so DNS rebinding can't slip past — and allow specific internal hosts back in withSEARXNG_DEEPDIVE_ALLOWED_HOSTS. Non-HTTP(S) schemes — includingfile://— are always rejected, so the tool never reads local files. Body size is capped at 10 MB, a single request has a 45 s total deadline, and the returned Markdown is length-capped (paginate withstartChar/maxLength), so a malformed or oversized upstream can't trivially exhaust memory or the caller's context window.The
searchtools forward the model's query verbatim to SearXNG. SearXNG is the trust boundary for upstream engine traffic; this package does not add additional rate-limiting or query rewriting.Tool output is adversarial input — prompt injection is possible. Search-result snippets and the Markdown returned by
web_url_readboth contain text the model will read as part of its working context. A page or snippet you don't control can carry instructions ("ignore previous instructions and …"). This isn't a defect in this MCP server — it's inherent to any tool that returns external text — but agent loops that auto-act on tool output without human review are the threat model. Treat tool output as untrusted input, especially forweb_url_readagainst URLs the model picked rather than the user.
This package is provided as-is under MIT with no warranty or liability for damages — see LICENSE. Report suspected vulnerabilities privately via GitHub Security Advisories rather than opening a public issue. See SECURITY.md.
License
MIT — see LICENSE.
Available Tools
4 toolssearchA
Broad web search across the SearXNG instance's full enabled engine pool.
Use this for general-purpose queries when you don't have a specific source preference. Returns a merged, deduplicated result list from across this instance's enabled engines — actual yield depends on which engines respond at query time.
Knobs: • pages=N (max 5) — fetch multiple pages and merge for more results in one call. • format='full' — full result objects with metadata; default 'compact' is token-efficient. • time_range — narrow to recent results (note: not all engines support this — see field doc).
When to reach for a different tool: • search_on_engines — you want specific sources (e.g. just ArXiv + PubMed) • search_by_category — you want all engines in a category (e.g. all science)
Inputs: query (required), pages, format, pageno, +3 more.
search_by_categoryA
Search within one or more categories — runs every engine tagged with each (e.g. categories: ["science"]).
Use when you want broad coverage of a content type without enumerating engines.
NOTE: this instance's live engine/category list is currently unavailable — SearXNG was unreachable when the server started. Names you pass are forwarded and validated once SearXNG is reachable; restart the server after SearXNG is up to see the enabled list here.
Multi-page fanout (pages=N) and other knobs work the same as the broad search tool.
Inputs: query (required), categories (required), pages, format, +4 more.
search_on_enginesA
Search using ONLY the specified engines (e.g. engines: ["arxiv", "duckduckgo"]).
Use when you have a specific source preference instead of the broad search tool.
NOTE: this instance's live engine/category list is currently unavailable — SearXNG was unreachable when the server started. Names you pass are forwarded and validated once SearXNG is reachable; restart the server after SearXNG is up to see the enabled list here.
Multi-page fanout (pages=N) and other knobs work the same as the broad search tool.
Inputs: query (required), engines (required), pages, format, +4 more.
web_url_readA
Fetch a URL and convert its HTML content to clean Markdown.
Use after search (or its variants) when you have a URL and want the
actual page text, not just the search snippet. Lightweight HTTP +
HTML→Markdown — handles ~80% of the static-HTML web (Wikipedia, docs
sites, blogs, news, GitHub READMEs).
What this DOES NOT handle: • JavaScript-rendered pages (React/Vue/Angular SPAs) — content loads after the initial HTML, which we don't execute. Returns minimal or empty markdown for these. • Bot-protected pages (Cloudflare challenge, captcha) — typically fail with HTTP 403/503. • Binary resources (PDF, images, archives) — returns an explanatory hint instead of garbled bytes.
For those cases, fall back to a Chromium-backed reader (e.g. Crawl4AI
exposed via the SearXNG-Compose reader profile).
Token-efficient extraction modes (priority order — first one set wins): • readHeadings:true — returns ONLY the heading list (hierarchical TOC). Cheapest survey of a long page. • section:'' — returns content under first matching heading, up to the next same-or-higher heading. Use after readHeadings to jump. • paragraphRange:'3-7' — 1-indexed paragraph slice; supports 'N' (single) or 'N-M' (range). • startChar + maxLength — character window pagination. Response includes total_length and truncated so you can plan follow-up calls.
If no extraction mode is set, returns the full Markdown.
Recommended workflow for long pages:
web_url_read(url, readHeadings: true) ← TOC scan
web_url_read(url, section: '') ← targeted read Far more token-efficient than fetching the full page up front.
Inputs: url (required), section, maxLength, startChar, +2 more.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
TDQS
Scored across 4 tools
Each tool has a clearly distinct purpose: 'search' uses all engines, 'search_on_engines' specifies engines, 'search_by_category' uses categories, and 'web_url_read' fetches URL content. No overlap or ambiguity.
All tool names follow a consistent 'action_noun' pattern in snake_case. The three search tools share the 'search' base with clear qualifiers, and 'web_url_read' follows the same convention. Perfectly predictable.
4 tools is ideal for a search-focused MCP server: three search variants covering broad, specific, and category-based searches, plus a URL reader. No unnecessary tools, and no obvious missing core functionality.
The tool set covers the main search and fetch operations well. Minor gap: there's no tool to list available engines or categories, but this is noted as a server startup issue rather than a missing tool. The core lifecycle is complete.
Maintenance
Related MCP Connectors
Web search API and MCP server for developers building AI agents.
Free web search for AI agents. No API key required. Hosted MCP in active development.
Docs: https://docs.keenable.ai/mcp-server Keenable is a free, remote MCP server that gives agents access to the web index. Search the web with ranked results and date/site filters, then fetch any indexed page as clean markdown. Works out of the box with no account or API key.
Related MCP Servers
- AlicenseBqualityDmaintenanceAn MCP server for connecting agentic systems to search systems via searXNG.1132MIT
- FlicenseAqualityCmaintenanceAn MCP server for SearXNG that provides web search capabilities with concise model-visible output while preserving full result payloads in metadata. It supports search, parallel fetching, URL extraction, and research workflows through both local stdio and streamable HTTP transports.274-
- AlicenseAqualityDmaintenanceAn MCP server that integrates the SearXNG API for web search and URL content extraction with advanced features like pagination, caching, and proxy support.414,935 npm2MIT
- AlicenseNot gradedqualityFmaintenanceAn MCP server that integrates the SearXNG API to provide web search with pagination, filtering, and URL content extraction.11 npmMIT