aggregate-search
Provides an optional self-hosted SearXNG search backend for the web_search tool, allowing the server to query a SearXNG instance configured via SEARXNG_BASE_URL and include its results in aggregated, deduplicated, weighted RRF search results.
This server does not currently have a deployable release on Glama. Follow the repository's setup instructions to install and configure it for your MCP client, including any required credentials and local dependencies.
aggregate-search โ Multi-Engine Search MCP Server
Overview
aggregate-search turns multiple search backends โ Bocha / Zhipu (China-friendly), Tavily / Serper (overseas), keyless DuckDuckGo and an optional self-hosted SearXNG โ into a single MCP server (web_search / read_url / engine_status) served over stdio with FastMCP. Chinese queries automatically boost domestic engines (weight 1.3) and English queries boost overseas ones; when an overseas engine is unreachable it lands in a 300 s health blacklist and the request falls back to domestic sources instead of timing out.
On top of the fusion layer sits a lexical relevance filter (drops off-topic results, with a full audit trail), a near-duplicate merger for same-site URL-parameter variants, and an optional embedding/LLM re-ranker that promotes authoritative pages. Every iteration is tracked by a frozen 14-query gold set plus an 8-query hold-out set in eval/ โ gold Hit@5 11/11, junk results 0/70, hold-out generalization confirmed with zero false kills.
๐จ๐ณ ไธญๆ็่ฏดๆ่ง README_CN.md
Related MCP server: evo-scry
Highlights
๐ Weighted RRF fusion โ
score = ฮฃ w/(60+rank)with a cross-engine hit bonus (+0.25), so results confirmed by multiple engines rank higher; each result carries anenginesfield listing its sources๐งน Relevance filter + near-duplicate merge โ lexical coverage scoring (
AGG_RELEVANCE=1default) drops off-topic results with adropped_items/downweighted_itemsaudit trail, and merges same-site URL-parameter variants (title similarity โฅ 0.9)๐ฏ Optional re-ranking โ
AGG_RERANK=embedding(Zhipu embedding-3 cosine) orllm(glm-4-flash intent ranking, RankGPT-style with graceful degrade), lifting authoritative pages that lexical layers cannot see๐
read_urlfull-text tool โ fetch a page and get clean markdown (trafilatura), direct-first with proxy retry; closes the search โ read loop for agents๐ฉบ Health blacklist & failure taxonomy โ failing engines are benched for 300 s; every failure is classified (
quota_exhausted/auth_failed/network_unreachable/timeout/rate_limited/server_error) with an actionable hint inengine_errorsโก Query cache + soft deadline โ TTL cache (300 s, LRU 128, never caches failures) and
AGG_SOFT_DEADLINE_Searly return from completed engines on tail-latency spikes๐ Proxy auto-negotiation โ overseas engines go through
AGG_PROXY(env โ common Clash ports โ Windows registry proxy), validated by a real CONNECT + TLS handshake before use, with automatic direct-retry fallback; domestic engines always connect directly๐งช 90 offline tests + stdio E2E + real-quota check โ pytest suite (fusion / engines / orchestrator / relevance / rerank / cache) runs without network or quota;
verify_e2e.pydrives the server through a real MCP stdio client;real_check.pyexercises live keys๐งฉ Pluggable engines โ add one adapter file + one
config.SPECSregistration; per-engine kill switch viaAGG_ENABLE_<NAME>=0
Tech Stack
Architecture
query (zh/en auto-detected)
โ
โโโโโโโโโ orchestrator โโโโโโโโโ
โ parallel dispatch ยท timeoutsโ
โ soft deadline ยท query cache โ
โผ โผ
domestic engines overseas engines
Bocha ยท Zhipu Tavily ยท Serper (+ keyless DuckDuckGo,
always direct via validated proxy optional SearXNG)
โ โ
โโโโโโโโโโโ fusion โโโโโโโโโโโโโ
URL normalization โ dedup โ weighted RRF (k=60)
+ cross-engine bonus โ per-host cap (2)
โ near-duplicate merge (same-site variants)
โ
โผ
relevance filter AGG_RELEVANCE=1
coverage scoring ยท auth-domain boost ยท audit trail
โ
โผ
optional re-rank AGG_RERANK
embedding cosine / LLM intent order, graceful degrade
โ
โผ
MCP tools: web_search / read_url / engine_status (stdio)Getting Started
bash setup.sh # uv venv (Python 3.12) + deps
cp .env.example .env # fill at least one engine key
.venv/Scripts/python.exe -m pytest -q # 90 offline tests (no network)
.venv/Scripts/python.exe verify_e2e.py # stdio end-to-end (mocked)
.venv/Scripts/python.exe real_check.py # live quota checkRegister with any MCP client (ZCode / Claude Code etc.):
"aggregate-search": {
"type": "stdio",
"command": "<abs-path>/.venv/Scripts/python.exe",
"args": ["<abs-path>/server.py"],
"cwd": "<abs-path>"
}No key ships with this repository โ obtain your own: Bocha https://open.bochaai.com (CN, 1000 free), Zhipu https://open.bigmodel.cn, Tavily https://app.tavily.com (1000/month free), Serper https://serper.dev, or point SEARXNG_BASE_URL at a self-hosted instance (no key). See .env.example.
Configuration
BOCHA_API_KEY/ZHIPU_API_KEY/TAVILY_API_KEY/SERPER_API_KEY/SEARXNG_BASE_URLโ all optional; whichever is set gets used.AGG_ENABLE_DUCKDUCKGO=1enables the keyless DuckDuckGo fallback engineAGG_RELEVANCE(1) โ lexical relevance filter;AGG_REL_MIN_COV(0.4) /AGG_REL_PENALTY(0.3) /AGG_AUTH_BOOST(1.3) tune it; 0 = pass-throughAGG_RERANK(off) โembedding(Zhipu embedding-3,AGG_RERANK_MODEL) /llm(glm-4-flash,AGG_RERANK_LLM_MODEL) or stacked"embedding,llm";AGG_RERANK_ALPHA(0.4) blends similarity into the score,AGG_RERANK_POOL(20) caps candidatesAGG_CACHE_TTL_S(300, 0 = off) โ LRU query cache (128 entries, failures never cached);AGG_SOFT_DEADLINE_S(10, 0 = off) โ return completed engines' results early on tail-latency spikesAGG_MAX_PER_HOST(2) โ per-site result cap;AGG_NEAR_DUP(1) /AGG_NEAR_DUP_THRESHOLD(0.9) โ same-site parameter-variant merge;AGG_READ_MAX_CHARS(8000) โread_urloutput capAGG_DOMAIN_BOOST/AGG_DOMAIN_BLOCKโ per-domain boost/block rules (AGG_DOMAIN_BOOST_FACTOR1.5)AGG_PROXYโ egress proxy for overseas engines only; empty = env โ probe ports 7897/7890/10809 โ Windows registry proxyAGG_ENABLE_<NAME>=0โ force-disable an engine;AGG_TIMEOUT_S(20) /AGG_CONNECT_TIMEOUT_S(10) /AGG_UNHEALTHY_TTL_S(300)ZHIPU_SEARCH_ENGINEโsearch_std(default) /search_pro/search_pro_sogou/search_pro_quark
Evaluation
A frozen 14-query gold set across 5 dimensions (CN/EN freshness, authoritative-source locating, off-topic resistance, precise facts), each result scored 0/1/2 against the built-in WebSearch baseline; plus an 8-query hold-out set (never used for tuning) to check filter over-fitting. Methodology, per-item scoring evidence and raw engine outputs are all kept in eval/.
Metric (v6) | aggregate-search | built-in WebSearch |
Gold Hit@5 (11 applicable) | 11/11 | 11/11 |
MRR | 0.848 | 1.000 |
Junk results (score=0) | 0/70 | 0/62 |
Result coverage | 70/70 | 62/70 |
Latency | mean 2.7 s / median 2.6 s | not measured |
v1โv6 evolution (coverage filter โ per-engine cap & dedup fixes โ semantic re-rank โ near-duplicate merge): junk 5/70 โ 0/70, Hit@5 10/11 โ 11/11 โ eval/report.md
Hold-out A/B: with the filter on, junk 5/40 โ 0/40 and zero false kills on normal queries โ eval/holdout_report.md
Repository Layout
server.py MCP entry (stdio): web_search / read_url / engine_status
aggregate_search/
config.py engine registry + zh/en weights + key detection
models.py unified SearchResult model
fusion.py URL normalization, dedup, weighted RRF, cross-source bonus, per-host cap, near-duplicate merge
relevance.py lexical relevance filter + authoritative-domain boost + audit trail
rerank.py embedding / LLM re-ranking with graceful degradation
reader.py read_url full-text extraction (trafilatura) + proxy retry
orchestrator.py parallel dispatch, health blacklist, failure taxonomy, proxy probing, cache, soft deadline
engines/ per-engine adapters (add file + register in config.SPECS)
tests/ 90 mocked tests (fusion / engines / orchestrator / relevance / rerank / cache)
eval/ frozen gold set, per-version scores, hold-out report, raw outputs
verify_e2e.py stdio E2E via a real MCP client
real_check.py live-key verification (spends quota, prints failure kinds)
CHANGELOG.md v1โv6 iteration log
docker-compose.yml optional self-hosted SearXNG (+ searxng-settings/)
HANDOFF_ๆฅ็ฎก่ฏๆญ.md engineering diagnosis report (CN)
ๅ้ด่ฐ็ .md survey of comparable open-source search MCPs (CN)
.env.example key templateKnown Limitations
Engine adapters follow official docs; after API changes only the adapter file needs updating
Re-ranking can only reorder what engines return โ an authoritative site absent from the result pool stays absent (recall gap, see eval/holdout_report.md H8); per-engine depth or engine-side fixes are the way out
Overseas engines need a working egress proxy; on machines without one, disable them via
AGG_ENABLE_<NAME>=0to skip cold-start timeouts
License
MIT ยฉ 2026 TZH-user โ see LICENSE.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelogโฆ
This server cannot be deployed
Maintenance
Related MCP Connectors
One MCP server for all web data. Scrape, search, crawl, and AI research in one install.
Multi-engine search for AI agents. Trust scoring, local corpus, MCP-native. Self-hostable, BYOK.
MCP server for Google search results via SERP API
Related MCP Servers
- AlicenseNot gradedqualityDmaintenanceAn MCP server that enables web search capabilities by integrating with a SearXNG instance to aggregate results from over 130 engines. It allows users to perform filtered searches across categories like news, science, and social media while supporting advanced parameters for language and time range.14MIT
- AlicenseNot gradedqualityDmaintenanceMCP server for internet search via direct Google and DuckDuckGo HTML scraping with AI-powered result normalization and optional summarization, requiring no API keys for search.MIT
- AlicenseNot gradedqualityDmaintenanceAn MCP server that aggregates web search results from multiple engines and optionally renders pages to Markdown, providing a unified search interface.6 npm3ISC
- AlicenseAqualityDmaintenanceMulti-source web search MCP server with RRF fusion, 4-layer URL extraction, and provider health tracking.620 npmMIT