mcp-docs
by yrafique
README.md
<div align="center">
# mcp-docs
### Intelligent, fully-offline documentation search for Nokia **NSP** & **SR OS**, in a single container.
Ask a real engineering question. Get the exact passage back, ranked by a purpose-built
retrieval stack, with a deep-link citation to the precise page and heading. **With zero
internet, zero credentials, and one `docker run`.**
</div>
---
> [!WARNING]
> **Experimental proof-of-concept: not an official Nokia product.** A personal/community
> experiment for evaluating MCP-based documentation search. Not built, supported, or endorsed
> by Nokia; no warranty; not for production. Nokia product names/docs are the property of Nokia,
> referenced here for interoperability and evaluation only.
**mcp-docs** turns the sprawling Nokia NSP and SR OS documentation set into a single, intelligent,
air-gapped search endpoint that AI agents and MCP-aware IDEs can ground their answers on. The
corpus, the search engine, and the AI models all live **inside one container**. Hand it to anyone,
run it on a laptop or a locked-down NOC, and it just works.
---
## Architecture
Docs are fetched with headless Chromium (driven by Playwright), then parsed and indexed once per
release, offline, into an optimized SQL dump. At boot, the single container imports the releases
it's asked for into ParadeDB, and the co-located MCP server runs the full hybrid-plus-reranker
pipeline: the database and the server live in the same container, with no external services, ever.
---
## Why it's different
| | |
|---|---|
| **Completely offline** | Corpus, embedding model, reranker, and database are all baked in. Runs `--network none`, air-gapped. No internet, no Hugging Face, no NSP connection, no credentials. |
| **One self-contained container** | ParadeDB + the MCP server + models + docs in a single image. `docker run` → ready. Nothing else to stand up. |
| **Multi-release, on demand** | Each product release lives in its own schema (`nsp_2604`, `sros_263`, …), built at boot from its own optimized SQL dump. Pick what you want with `RELEASES=`; defaults to **NSP 26.4 + SR OS 26.3**. Add a release = drop a dump + one registry line. |
| **Genuinely intelligent search** | Not keyword grep. A hybrid **BM25 + semantic-vector** retriever fused by Reciprocal Rank Fusion, re-ordered by a **cross-encoder reranker**, with Nokia-aware query expansion and conceptual-vs-CLI routing. |
| **Deep-link citations** | Every hit comes back with a **section-precise `deep_url`** that goes straight to the exact page, heading, or CLI command. |
| **Measured, not vibes** | The retrieval stack was A/B-tested on deep, multi-domain engineer questions and LLM-judged: **+12 quality points** over plain hybrid search. |
| **MCP-native** | Four clean tools any MCP client (Claude Code, IDEs, agents) can call directly. |
---
## Quick start
```bash
docker run -p 9705:9705 mcp-docs # defaults: NSP 26.4 + SR OS 26.3
```
On boot it starts its internal database, imports the requested releases into per-version schemas,
builds the search indexes, and serves the MCP endpoint at **`http://<host>:9705/mcp`**. Air-gapped?
```bash
docker run --network none -p 9705:9705 mcp-docs # fully offline
```
Only want one release, or want to add more:
```bash
docker run -p 9705:9705 -e RELEASES="nsp-26.4" mcp-docs # just NSP 26.4
docker run -p 9705:9705 -e RELEASES="nsp-26.4 sros-26.3" mcp-docs # both (default)
```
Build it yourself: `make up` (or `docker build -f docker/Dockerfile -t mcp-docs .`).
---
## How it works: from raw docs to intelligent answers
**Ingest pipeline (offline, done once per release → ships as an optimized SQL dump):**
```
Playwright + Chromium ──▶ parse & chunk ──▶ classify & deep-link ──▶ embed & index ──▶ optimized SQL
headless capture of section-aware tag every chunk with BM25 (ParadeDB) input .sql.gz
the Nokia doc portal, passages that page, heading, CLI mode, + vector (HNSW) per release,
offline keep their context mgmt domain, deep URL baked in load at runtime
```
Each chunk keeps *where it came from*: page, heading, whether it's MD-CLI vs classic-CLI, which
management plane it belongs to, and a link straight back to the source. That metadata is what makes
the answers precise and citable.
**Search pipeline (at query time, in the container):**
```
your question ──▶ Nokia synonym expansion ──▶ ┌ BM25 (exact terms) ┐ ──▶ RRF fuse ──▶ cross-encoder ──▶ CLI-vs-concept ──▶ cited
(pseudowire → Epipe…) └ vector (meaning) ┘ rerank routing answer
```
- **BM25** nails the exact CLI/REST/alarm/YANG tokens these docs are full of.
- **Semantic vectors** catch paraphrases and intent ("bring up a router" → commissioning).
- **Reciprocal Rank Fusion** blends both signals.
- **A cross-encoder reranker** then reads the candidates and re-orders by true relevance, the single
biggest quality lever.
- **Conceptual-vs-CLI routing** keeps a "why is X slow?" question from being answered by a raw command
reference.
The result: the *right passage*, ranked first, cited to the exact spot.
---
## Multi-release
One container can hold many NSP and SR OS releases side by side, each isolated in its own schema and
searchable independently or together.
```bash
docker run -p 9705:9705 -e RELEASES="nsp-26.4 nsp-25.11 sros-26.3" mcp-docs
```
Adding a release is a two-line change: drop `data/nsp-25.11.sql.gz` and register it in `versions.yml`:
```yaml
- name: nsp-25.11
product: nsp
version: "25.11"
schema: nsp_2511
dump: data/nsp-25.11.sql.gz
```
Search then routes automatically: `product="nsp"`, `version="26.4"`, or `"all"` (both families).
---
## Tools
| Tool | What it does |
|------|--------------|
| **`docs_search`** | Hybrid search across the selected release(s). Filter by `product`, `version`, `guide`, `cli_mode`, `mgmt_domain`. Returns ranked passages, each with a citable `deep_url` and its source release. |
| **`docs_get_chunk`** | Fetch a passage plus its neighbours for full surrounding context. |
| **`docs_list_guides`** | Catalogue the ingested guides/books per release. |
| **`docs_list_versions`** | Show which NSP / SR OS releases are built and searchable. |
---
## Connect
Point any MCP client at the streamable-HTTP endpoint:
```json
{
"mcpServers": {
"mcp-docs": { "type": "http", "url": "http://<host>:9705/mcp" }
}
}
```
No credentials, no secrets, no external services. Hand over the container and it just works.
---
## Under the hood
| | |
|---|---|
| **Engine** | ParadeDB (Postgres + `pg_search` BM25 + `pgvector` HNSW): top-tier lexical and vector search in one embedded store. |
| **Models** | `bge-small` embeddings and an `ms-marco-MiniLM-L-12` cross-encoder reranker, both baked in, CPU-only: about 510 ms per search with reranking, 120 ms without. |
| **Portable** | Everything is bundled and runs on CPU: no GPU, no cloud, no network. |
<sub>Optional roadmap: a live-NSP mode that auto-detects your running release and routes docs to it (requires network + a read-only token), kept strictly optional so the default stays fully offline.</sub>
---
<div align="center">
<sub>Section-precise, intelligent hybrid search over the Nokia NSP & SR OS documentation set: offline, in one container.</sub>
<sub>Experimental proof-of-concept · not an official Nokia product · no warranty · provided as-is.</sub>
</div>
This server cannot be deployed
Maintenance
ActivitySlowing
ResponsivenessNo issues