Multimodal RAG Service

Production-minded retrieval for text, tables, images, and PDFs. Sentence-transformer text vectors and CLIP image vectors are stored in separate FAISS indexes; BM25 sparse results are combined using weighted reciprocal-rank fusion.

Run locally

python -m venv .venv
.venv\\Scripts\\Activate.ps1
pip install -e ".[dev]"
Copy-Item .env.example .env
uvicorn app.main:app --reload

The API documentation is available at http://localhost:8000/docs. Models download automatically at their first embedding request.

API

curl -F "files=@report.pdf" http://localhost:8000/v1/documents
curl -X POST http://localhost:8000/v1/search -H "Content-Type: application/json" -d "{\"query\":\"revenue trend\",\"top_k\":5}"
curl -N -X POST http://localhost:8000/v1/chat/stream -H "Content-Type: application/json" -d "{\"query\":\"Summarize the report\"}"

MCP integration

Run the REST API, then start the stdio MCP bridge:

rag-mcp

Set RAG_API_URL to point at a non-default API address. The bridge provides search_knowledge_base, ask_knowledge_base, list_documents, and service_health tools to MCP clients.

Evaluation

Supply JSONL rows containing query, relevant_document_ids, and optionally reference_answer:

python -m app.evaluation --dataset eval/golden.jsonl

Reported metrics include Recall@K, Precision@K, MRR, p50/p95 retrieval latency, ROUGE-L, and BLEU-1.

Container deployment

docker compose up --build

The compose configuration mounts a persistent named volume at /service/data.