MMWRAG

A bilingual (Russian/English) RAG system over scientific literature (textbooks/papers): vision PDF parsing, BGE-M3 hybrid (dense+sparse) retrieval with a cross-encoder reranker, an MCP tool server, a LangGraph agent layer, and an HTTP serving API with async client-driven ingestion. Every non-trivial decision here — retrieval pool composition, reranker choice, context budget, LLM provider, whether the agent layer even helps — was made by measurement against a gold set, not intuition. See DECISIONS.md for the full log.

Features

• Vision PDF parsing behind a swappable interface (cloud PaddleOCR-VL / local PP-StructureV3) — required because the text layer doesn't encode formula structure.

• Structure-aware chunking (~512-token packing over blocks, page spans kept for citations).

• BGE-M3 dense + sparse embeddings; Qdrant hybrid search with server-side RRF.

• Cross-encoder reranker (bge-reranker-v2-m3) — measured hit@5 0.761 → 0.848.

• Book-aware cross-lingual routing — search(book_id=...) targets a specific book/language.

• MCP server (search, list_books) over streamable HTTP — no answer generation.

• LangGraph agent — multi-round, book-routed tool-calling search over MCP; beats single-shot answers on multi-hop questions.

• HTTP serving API (src/serve/) — SSE-streamed /answer//answer/agentic, async POST /ingest + status polling, health/readiness/Prometheus.

• Self-hosted observability (Langfuse) — full span tree, a true no-op when disabled.

• Two LLM providers behind one interface (OpenRouter, GigaChat), retry/backoff always on.

• Eval harness — page-level hit@k/MRR/recall@k, plus an LLM-judge harness for end-to-end answer quality.

Related MCP server: MyDocsMCP

Architecture

INDEXING   PDF ─parse─> Page[] ─chunk─> Chunk[] ─BGE-M3 (dense+sparse)─> Qdrant
QUERY      question ─HybridRetriever (RRF)─> top-N ─rerank─> top-k ─LLM─> Answer
MCP        client ─/mcp─> search(query, top_k, book_id) / list_books()
AGENT      question ─LangGraph tool-calling loop (book-routed)─> synthesize ─> Answer
SERVING    POST /answer, /answer/agentic (SSE) · POST /ingest, GET /ingest/{job_id}

Full package layout, swappable interfaces, and the agent/serving/observability design are in ARCHITECTURE.md.

Quickstart

# 1. dependencies (paddlepaddle-gpu is a manual prereq for the PARSING path only —
#    everything else, including embedding/reranking, runs on CPU too, just slower)
uv sync

# 2. vector database (+ optional embed-service/app containers — see docker-compose.yml)
docker compose up -d            # Qdrant on :6333

# 3. bring your own PDF and index it
#    parsing needs PADDLEOCR_TOKEN in .env (see .env.example);
#    pipeline: parse(pdf) -> chunk_pages(...) -> index_chunks(...)  (see notebooks/ for examples)

# 4. run the MCP server (for an LLM/agent client)...
uv run python -m src.mcp.server        # streamable-http on 127.0.0.1:8000
# ...or the HTTP serving API (for ordinary clients)
uv run python -m src.serve.app         # SERVE_API_KEY required, see .env.example

The corpus is not included (copyright). Search/MCP/serving need Qdrant + the local models (BGE-M3, the reranker); CPU works (slower), GPU is faster. Parsing additionally needs a PaddleOCR-VL cloud token. A Docker image is also published — see Docker image below.

Results

Six of the decisions in DECISIONS.md, each backed by a gold-set measurement (own metric/gold set per panel — bars aren't comparable across panels):

The single most interesting result — and the one with the most involved diagnosis — is why cross-lingual retrieval was failing:

Technical deep dive: diagnosing RU→EN ≈ 0

A Russian→English cross-lingual query scored hit@10 ≈ 0 in a cross-book search — by far the worst slice in the eval suite. There were three plausible causes: sparse retrieval dragging in Russian noise, BGE-M3 having a hard ceiling on cross-script matching, or the correct English chunk being crowded out by a same-language, same- concept competitor. A single before/after number can't tell those apart, so the fix was a 2×2 diagnostic: pool (hybrid vs. dense-only) × search scope (whole index vs. scoped to the target book), reranker held out entirely so it couldn't mask the result.

What the four numbers show: scoping the search to the target English book alone lifts hit@10 from 0.00 to 0.71 — ruling out an embedding ceiling (BGE-M3 clearly can match Russian queries to English text). And dense-only vs. hybrid in the cross-book condition (0.06 vs 0.00) is far too small a gap to explain a 0.71-point swing — ruling out sparse drag as the dominant cause. What's left is crowding: in a cross-book search, the same concept's Russian source (present in the much larger Russian corpus) systematically outranks the correct English one.

That result killed a planned "script-aware fusion" fix (it targeted sparse reweighting — the wrong cause) and replaced it with book-aware routing (search(book_id=...)), which is why the MCP tool takes a book_id parameter at all and why the agent layer routes sub-searches per book instead of always searching the whole index. Full walkthrough, including why the reranker was deliberately excluded from the diagnostic run, is in DECISIONS.md.

Demo

A real session against the MCP server (notebooks/mcp_smoke.py, output trimmed to metadata):

tools: ['search', 'list_books']

list_books:
  {'book_id': 'zorich_v1', 'title': 'Zorich — Mathematical Analysis I', 'language': 'ru', 'chunks': 1472}
  {'book_id': 'zorich_v2', 'title': 'Zorich — Mathematical Analysis II', 'language': 'ru', 'chunks': 2526}
  {'book_id': 'lebl', 'title': 'Lebl — Basic Analysis I', 'language': 'en', 'chunks': 722}

search RU (all books), top 3:
  zorich_v1 159 2.125
  zorich_v1 158–159 0.297
  zorich_v2 517 -0.357

search RU routed to lebl (cross-lingual), top 3:
  lebl 135–136 0.123
  lebl 167 -0.047
  lebl 208 -0.141

The last call shows book-aware cross-lingual routing: a Russian query with book_id="lebl" returns the English source (Lebl, p.135–136) that a plain cross-book search buries behind the Russian equivalent — the exact failure mode diagnosed above.

Docker image

A Dockerfile/docker-compose.yml for the HTTP serving app (src/serve/) are included; a pre-built image is also published to GHCR on release:

docker pull ghcr.io/mikrominiw/mmwrag:latest

GPU is optional (see docker-compose.yml); the image runs the serving API on CPU too, just slower for embedding/reranking.

Project structure

src/
  parse/           vision PDF -> Page[]   (cloud / local engines, idempotent cache)
  chunk/           Page[] -> Chunk[]      (structure-aware packing, page spans)
  index/           Chunk[] -> BGE-M3 -> Qdrant   (Embedder / VectorStore interfaces)
  query/           HybridRetriever + RerankingRetriever + ContextBuilder; answer()
  mcp/             MCP server: search / list_books (pure core + thin FastMCP server)
  agent/           LangGraph agent: multi-round, book-routed tool-calling search
  serve/           FastAPI HTTP API: SSE /answer(/agentic), async /ingest, health/metrics
  ingest/          async ingestion job: Job/JobStore, sequential worker, pipeline
  embed_service/   BGE-M3 embedder + cross-encoder reranker as a standalone HTTP service
  observability/   Langfuse tracing, a true no-op when disabled
  eval/            page-level hit@k/MRR/recall@k; end-to-end LLM-judge eval
tests/             unit tests (pure logic on fakes; integration tests skip offline)
notebooks/         runnable examples & measurement runners (*_smoke, eval_*, diag_*)

Status & roadmap

The full pipeline (parse → chunk → index → hybrid+rerank retrieval → MCP → agent → HTTP serving → observability → network embed-service → async ingestion) is done and measured. GraphRAG was considered and explicitly not pursued — see DECISIONS.md for why.

License

MIT © 2026 mikrominiw