Context Retrieval System Powered by RAG and MCP Server

A hybrid context-retrieval system built on the Model Context Protocol (MCP) and Retrieval-Augmented Generation (RAG). It exposes multimodal PDF parsing and a fully local vector-search pipeline as MCP tools, so AI agents (Claude Desktop, Cursor, or any MCP-compatible client) can fetch real-time document context on demand instead of relying on pre-loaded, static knowledge.

Highlights

• MCP-native — 7 JSON-in/JSON-out tools served over stdio (local) or streamable-HTTP (network), built on the official mcp Python SDK.

• Multimodal parsing — PyMuPDF extracts structured text and base64-encoded images; Camelot (lattice/stream, with a PyMuPDF fallback) extracts complex tables straight into LLM context windows.

• Local RAG pipeline — LangChain RecursiveCharacterTextSplitter chunking, on-device fastembed embeddings (BGE-small, ONNX), and a FAISS vector store. No external embeddings API, no per-query cost, no data leaving the machine.

• Sub-linear search — cosine similarity over L2-normalised vectors; above a configurable threshold the store switches to an IVF (inverted-file) index, keeping query latency approximately O(log N) as the corpus grows.

• Persistent & re-entrant — the FAISS index, raw vectors and chunk metadata persist to disk; re-indexing a changed document replaces its chunks (content-hash doc IDs).

• Safe by default — every tool path is validated (null bytes, existence, extension, optional allow-list roots) and returns structured JSON errors instead of crashing the server.

• Containerized + CI — Dockerfile (with the embedding model pre-baked) and GitHub Actions running lint, the full test suite including end-to-end MCP protocol tests, and a containerized smoke test.

Related MCP server: MinerU Document Explorer

Setup

git clone https://github.com/ESPChong/context-retrieval-system-RAG-MCP.git
cd context-retrieval-system-RAG-MCP

python3.12 -m venv .venv
.venv/bin/pip install -r requirements.txt
.venv/bin/pip install -e . --no-deps      # optional: `python -m context_retrieval` from anywhere

# stdio mode is what Claude Desktop / Cursor launch:
.venv/bin/python -m context_retrieval

The embedding model (~35 MB) downloads from Hugging Face once and is cached locally; after that the system runs fully offline.

Try the pipeline without any MCP client:

.venv/bin/python scripts/demo.py

Tool reference

Connect an MCP client

{
  "mcpServers": {
    "context-retrieval": {
      "command": "/absolute/path/to/context-retrieval-system-RAG-MCP/.venv/bin/python",
      "args": ["-m", "context_retrieval"],
      "env": {
        "CONTEXT_RETRIEVAL_DATA_DIR": "/absolute/path/to/context-retrieval-system-RAG-MCP/data",
        "CONTEXT_RETRIEVAL_ALLOWED_ROOTS": "/absolute/path/to/your/pdfs"
      }
    }
  }
}

(ready-to-edit copy in docs/claude-desktop-config.json)

Same mcpServers object; copy in docs/cursor-mcp.json.

# stdio mode (pipe JSON-RPC, e.g. from an MCP host):
docker build -t context-retrieval .
docker run -i --rm -v "$PWD/docs:/docs" context-retrieval

# network mode (streamable-http on http://localhost:8000/mcp):
docker compose up --build

Once connected, just ask: "Index /docs/report.pdf and tell me what it says about retention policy." — the client will chain index_document → search_context (or the extraction tools) automatically.

Configuration (env vars)

Testing

.venv/bin/python -m pytest -v

19 tests, including end-to-end MCP protocol tests that spawn the real server over stdio and exercise the full handshake → tool-call → RAG-roundtrip path. A deterministic 3-page sample PDF (prose / ruled table / embedded image) is generated by the suite itself.

.venv/bin/python scripts/make_sample_pdf.py my-sample.pdf   # inspect it yourself

Design notes

• Why local embeddings? Removing the external embeddings API eliminates per-query cost and network latency, and keeps document content on the machine. fastembed runs quantized-ready ONNX models — a 384-dim BGE-small that embeds ~2,600 chunks/sec on a laptop CPU.

• What "≈ O(log N)" means here. Flat exact search scans all N vectors. The IVF index partitions the space into nlist ≈ 4·√N clusters and compares the query against only nprobe of them plus their members — sub-linear, logarithmic-style growth in practice, at the cost of a small recall trade-off tunable via nprobe.

• Chunk provenance. Chunking runs per page, so every retrieved fragment carries page + source, letting the host model cite where an answer came from — the antidote to hallucinated citations.

• Graceful degradation. Camelot's lattice mode needs Ghostscript; when it is missing (thin environments), auto silently falls back to stream, then to PyMuPDF's detector, and table extraction still succeeds.