Search Memory MCP

Offline-first MCP server for web search, content fetching, and a local knowledge base. It requires no external API keys and uses local models for intent classification, optional cross-lingual search, semantic re-ranking, hybrid retrieval, and extractive deep-search answers.

Features

Search & fetch

• Browser context pooling with a persistent Playwright browser instance.

• Web search through configurable providers with health tracking and ordered fallback. Supported scrapers: DuckDuckGo, Bing, Brave, Google. Optional SearXNG meta-search provider (self-hosted or trusted public instance) via SEARXNG_BASE_URL.

• Federated search across providers with URL normalization, cross-provider deduplication, and Reciprocal Rank Fusion (RRF).

• Opt-in intent-aware search routing (strategy=auto) with heuristics, local classifier fallback, and versioned provider profiles.

• Domain filter (domain) and date-range filter (from_date / to_date).

• Query rewrite for local-index searches and opt-in web multi-query expansion (expand_query=true): abbreviation expansion, question normalization, and news year bias.

• Optional cross-encoder reranking (ENABLE_RERANKER).

• Deep-search answers with paragraph/sentence term scoring, stopword filtering, and per-source citations.

• Structured JSON output (format: "json") for machine-readable search results and answers.

Local knowledge base (RAG)

• Document chunking (paragraph-first with overlap).

• Hybrid retrieval: FTS5 keyword search + sqlite-vec semantic search fused with RRF. Degrades to FTS-only when embeddings are unavailable.

• ingest_document, index_url, search_index, and list_index tools for building a citable local corpus.

• Entity graph over indexed documents with find_related (related docs + co-occurring entities).

Memory & observability

• Session memory (remember / recall / forget) with topic, tags, and session scoping.

• Per-search execution traces (stages, timings, cache). Recent searches surface in server_status; TRACE_SEARCHES=true logs full traces.

• Retrieval eval harness (npm run eval:retrieval) with recall@k, precision@k, and MRR.

Safety & infrastructure

• HTTP-first page fetching with GitHub Raw and RSS fast paths plus Playwright fallback.

• SSRF protection for fetch_content and index_url by blocking localhost and private network targets.

• Token-bucket rate limiting for search and fetch tools.

• Semantic cache backed by SQLite and sqlite-vec, namespaced per execution strategy/plan.

Related MCP server: Local Web Search MCP Server

Requirements

• Node.js 20.9.0 or newer.

• Node.js 24 is the development baseline (.nvmrc); CI uses the same version. After switching Node versions, reinstall dependencies so native modules such as better-sqlite3 match the active runtime.

• npm.

• Network access during installation for npm packages, Playwright Chromium, and first-run model downloads.

Installation

npm install
npm run build

The postinstall script downloads Playwright Chromium. On first use of model-backed features, Transformers.js downloads the required model files to the local Hugging Face cache. The first request that loads a model can be slow; later requests reuse the local cache. Keep ENABLE_CROSSLINGUAL=false and ENABLE_RERANKER=false for the lightest first run. Obvious strategy=auto intents are resolved by heuristics without loading the intent classifier; ambiguous auto queries may trigger a first-run classifier download.

MCP Client Configuration

Add the built server to your MCP client config:

The package, command and MCP server identity are search-memory-mcp. Use your actual checkout path in the configuration below. Existing clients that launch node with an absolute build/index.js path can keep that path even if the checkout directory still has its previous name. Restart the MCP connection after rebuilding to load the updated server identity. Keep existing CACHE_DB_PATH values to retain stored data.

{
  "mcpServers": {
    "websearch": {
      "command": "node",
      "args": ["path/to/search-memory-mcp/build/index.js"],
      "env": {
        "RATE_LIMIT_SEARCH_PER_MIN": "10",
        "RATE_LIMIT_FETCH_PER_MIN": "20",
        "SEARCH_PROVIDERS": "duckduckgo,bing",
        "ENABLE_CROSSLINGUAL": "false",
        "CACHE_DB_PATH": "websearch_cache.db"
      }
    }
  }
}

If the package is installed globally or through a package runner, use the binary entrypoint:

{
  "mcpServers": {
    "websearch": {
      "command": "search-memory-mcp",
      "args": [],
      "env": {
        "SEARCH_PROVIDERS": "duckduckgo,bing",
        "ENABLE_CROSSLINGUAL": "false"
      }
    }
  }
}

For package-runner based clients, the command can be npx with args set to ["-y", "search-memory-mcp"] once the package is available from the configured npm registry.

Tools

Search strategies

Semantic query cache keys are namespaced by execution strategy (and by plan fingerprint for auto), so a cached fallback result is never reused for aggregate or a different auto plan. Deep-search page content continues to use the normal content cache.

Domain and date bounds are exact cache constraints, isolated from semantic query similarity. Provider candidates are cached and the requested domain/date filters are reapplied on every hit, including before deep page fetching. A cache entry with no eligible candidates triggers provider execution. Namespace filtering happens before the vector-store result limit so unrelated strategies cannot crowd out eligible entries.

SEARCH_PROVIDERS is an allowlist as well as the configured provider set. Auto routing never activates a provider omitted from SEARCH_PROVIDERS; the routing profile only changes ordering and how many configured providers are selected as primary candidates.

For aggregate auto profiles, secondary configured providers are contacted only if all selected primary providers return no usable result. A partial primary success is accepted instead of widening the request just to increase result count. This limits scraping load and reduces unnecessary blocking/CAPTCHA exposure.

Current routing profile: v1.

These provider preferences are initial hypotheses, not permanent quality claims. They are versioned so later releases can tune them from deterministic and live evaluation evidence without scattering routing conditionals through the server.

Example: intent-aware search

{
  "query": "PostgreSQL connection pooling best practices",
  "strategy": "auto",
  "max_results": 5
}

Use domain for targeted searches such as react.dev or github.com. Intent detection always receives the original query; site:<domain> is appended only afterward for provider execution.

{
  "query": "server components reference",
  "domain": "react.dev",
  "strategy": "auto",
  "max_results": 5
}

Use from_date / to_date (inclusive YYYY-MM-DD) to filter by detected publish dates in titles and snippets. Results without a detectable date are kept by default.

Use deep=true only when the client needs the server to fetch top pages and extract a likely answer from page text. Answers include per-source citations ([Source N]) mapped to the fetched URLs. The MCP client LLM remains responsible for final reasoning and summarization.

Search snippets with old detected dates include a short freshness warning so clients can treat stale sources carefully.

Example: federated search

{
  "query": "postgres connection pooling strategies",
  "strategy": "aggregate",
  "max_results": 5
}

Example: multi-query search

{
  "query": "how to configure db backup",
  "strategy": "auto",
  "expand_query": true,
  "max_results": 5
}

expand_query defaults to false. When enabled, the server searches the original query plus up to two variants, then deduplicates URLs and combines query rankings with RRF before reranking. This can make up to three times as many provider requests. The original query determines intent and locale; every variant keeps the domain restriction and uses the same provider plan. Date filters apply to the combined results. Expanded searches use a separate cache namespace, and cache hits skip all provider requests.

Example: structured JSON output

{
  "query": "postgres pooling",
  "format": "json",
  "max_results": 5
}

Returns a stable payload with query, resultCount, results[] (title, url, snippet, source, optional scores), and meta.

Example: local knowledge base

{ "content": "PgBouncer multiplexes PostgreSQL connections...", "title": "PgBouncer Guide", "source": "https://example.com/pgbouncer" }

Then search it:

{ "query": "connection pooler", "max_results": 5 }

Or explore the entity graph:

{ "entity": "PgBouncer" }

fetch_content uses fast source-specific paths before opening a browser:

• GitHub repository, blob, tree, and raw URLs are read from raw.githubusercontent.com when possible.

• RSS or Atom feed URLs, plus common blog/news feed paths, are converted into a Markdown list of recent items.

• Regular HTML pages still use HTTP-first Readability parsing with Playwright fallback.

Configuration

Docker

npm run docker:build
npm run docker:up

Docker Compose stores the SQLite cache in a named volume mounted at /app/data and stores Hugging Face models in a separate named volume. The container sets CACHE_DB_PATH=/app/data/websearch_cache.db.

Development

See the production roadmap and sector comparison for release scope, production gaps, and measurable launch gates.

npm run build
npm run typecheck
npm test
npm run smoke:mcp
npm run eval:retrieval
npm audit --audit-level=moderate
npm pack --dry-run --json

npm run smoke:mcp starts the compiled server over stdio, verifies the web_search strategy values (fallback, aggregate, auto), checks the knowledge/memory tools, confirms routing diagnostics from server_status, and confirms that fetch_content blocks localhost. It does not perform a live provider search, keeping CI independent of search-engine HTML/network availability.

npm run eval:retrieval runs an offline FTS-only retrieval baseline (recall@k, precision@k, MRR) over the knowledge index using fixtures in evals/retrieval/cases.jsonl. Embeddings are disabled for both ingestion and search, so the evaluation does not load or download models. It does not measure semantic/hybrid retrieval quality.

Deterministic TR/EN routing fixtures live in evals/search-routing/queries.jsonl and are exercised by the normal Vitest suite. They validate intent coverage, conservative heuristic behavior, ambiguity defer cases, and provider-allowlist enforcement without loading the real classifier or contacting providers.

Troubleshooting

• If startup fails after install, run npx playwright install chromium.

• If better-sqlite3 reports NODE_MODULE_VERSION mismatch, switch to the Node version in .nvmrc and run npm ci using that runtime before building again.

• If the first model-backed request is slow, allow the Transformers.js model download to complete and retry.

• If search returns no results, change SEARCH_PROVIDERS order/set or try a direct fetch_content URL.

• If aggregate mode is too slow or triggers provider blocking, use the default fallback strategy.

• If auto chooses too broad a search plan for your use case, use explicit fallback or aggregate; explicit strategies bypass the auto planner.

• If Docker cannot find Chromium, rebuild the image with npm run docker:build.

• If cache files appear in the project root, set CACHE_DB_PATH to a dedicated data directory.

• If knowledge search only hits keywords, embeddings may still be indexing; search_index falls back to FTS-only until vectors are ready.

npm Packaging

The npm package includes only build/, README.md, LICENSE, and SECURITY.md. npm pack runs npm run build through prepack so the package contains compiled JavaScript instead of local planning files, tests, caches, or source-only artifacts.

Security

See SECURITY.md for reporting instructions and current dependency audit notes.

License

ISC