mcp-rag-tools

An MCP (Model Context Protocol) server that exposes retrieval and fact-checking tools from rag-document-qa as standalone, callable tools — usable by any MCP-compatible client (Claude Desktop, autonomous agents, etc.) without duplicating the underlying RAG logic per client.

Live endpoint: https://c6q1zpx6ki.execute-api.ap-south-1.amazonaws.com

What it does

Two tools, callable over HTTP or via the MCP stdio protocol:

• search_documents — retrieves relevant document chunks for a question. Pipeline: LLM query rewrite → FastEmbed (ONNX) similarity search over ChromaDB (top 15) → cross-encoder re-ranking against the original question → returns top-k chunks with page number and source filename.

• evaluate_answer — fact-checks a generated answer against given context, classifying it as FULLY_SUPPORTED, PARTIALLY_SUPPORTED_AND_HONEST, or UNSUPPORTED (hallucination).

Related MCP server: rbac-rag-assistant

Architecture

MCP client (stdio)  ──▶  mcp_server.py  ──▶  rag_core.py  ──▶  ChromaDB / FastEmbed / Groq
                                                  ▲
HTTP client         ──▶  app.py (FastAPI+Mangum) ┘
        │
        ▼
   API Gateway ──▶ Lambda (Docker container image)

rag_core.py holds the actual retrieval/verification logic, shared by both entry points:

• mcp_server.py — exposes it over stdio via the MCP SDK, for direct MCP-client use

• app.py — wraps it in FastAPI, adapted for Lambda via Mangum, for HTTP/public access

Deployment

Packaged as a Lambda container image (not a zip) because chromadb and onnxruntime ship platform-specific compiled binaries — installing them on Windows produces Windows binaries that Lambda's Linux runtime can't load. The Docker build installs dependencies inside a Linux-based image (AWS's official public.ecr.aws/lambda/python:3.12 base), producing Linux-native binaries instead.

Pipeline: Docker build → push to ECR → Lambda (image-based) → API Gateway (HTTP API) → public HTTPS endpoint.

Problems hit and fixed

Three separate Lambda filesystem/dependency issues surfaced only at deploy time, none reproducible locally:

1. FastEmbed model download failed — Lambda's filesystem is read-only outside /tmp; FastEmbed's default HuggingFace download-and-cache path isn't writable at runtime. Fixed by pre-downloading both the embedding model and the cross-encoder reranker into the Docker image at build time, pointed at a directory baked into the image (/var/task/fastembed_cache).

2. ChromaDB failed to initialize — same read-only filesystem issue; PersistentClient needs to write lock/index files, not just read them. Fixed by copying the baked-in chroma_db/ directory into Lambda's writable /tmp on cold start.

3. ChromaDB's native bindings crashed silently — Amazon Linux's system sqlite3 is older than Chroma's Rust bindings require. Fixed with pysqlite3-binary, monkey-patching Python's sqlite3 module to use it before Chroma imports.

Diagnosed entirely from CloudWatch Logs (aws logs tail), each fix verified locally against the Lambda Runtime Interface Emulator before redeploying.

Also hit: Docker's default multi-platform/attestation manifest format isn't supported by Lambda's image import — fixed with --provenance=false --sbom=false on build.

Stack

MCP Python SDK · FastAPI · Mangum · LangChain · ChromaDB · FastEmbed (ONNX) · Groq · Docker · AWS Lambda · API Gateway · ECR

Known limitations

• Cold starts are slow (~10-20s) due to loading the ONNX embedding model and Chroma initialization on a fresh container

• Hybrid BM25+vector retrieval exists in rag-document-qa's history but is disabled here, matching the parent project's current live behavior

• No auth on the public endpoint — fine for a portfolio demo, not production-ready as-is