RAG MCP Server (Pinecone)

FastMCP server exposing search_docs and ask_docs tools backed by a Pinecone vector index. Embeddings (Qwen3-Embedding-0.6B) and answer generation (Qwen2.5-1.5B-Instruct GGUF) run locally via transformers / llama.cpp — no LLM API key required, only a Pinecone account.

Setup

Copy .env.example to .env and fill in:

PINECONE_KEY=your_pinecone_api_key
PINECONE_INDEX=your_index_name

Drop PDFs into data/.
Install deps and ingest:
```
uv sync
uv run python ingest.py
```
Run the server:
```
uv run python main.py
```
Query it:
```
uv run python client.py "your question"
```

Docker

Build and run with Docker Compose (recommended — persists the HuggingFace model cache in a named volume so models aren't re-downloaded on every restart):

docker compose up --build

The server listens on http://localhost:8000/mcp. data/ is mounted as a volume, so PDFs added on the host are visible inside the container.

To ingest PDFs into Pinecone from inside the running container:

docker compose exec rag-mcp-server uv run python ingest.py

Without Compose

docker build -t rag-mcp-server .
docker run --rm -p 8000:8000 --env-file .env -v "$(pwd)/data:/app/data" rag-mcp-server

Notes

The image installs build tools to compile llama-cpp-python; first build is slow, subsequent ones are cached.
Models are downloaded from HuggingFace on first run, not baked into the image — mount a volume over /root/.cache/huggingface (already done in docker-compose.yml) to avoid re-downloading.
MCP_HOST/MCP_PORT env vars override the listen address (default 0.0.0.0:8000 in the container, 127.0.0.1:8000 for local uv run).

Files

ingest.py — chunk PDFs from data/, embed, upsert into Pinecone.
server.py — MCP tools search_docs (retrieval + rerank) and ask_docs (retrieval + local generation).
models.py — local embedding/generation models.
main.py — server entrypoint.
client.py — example MCP client for manual testing.
eval.py — Ragas evaluation (Faithfulness, AnswerRelevancy) of the RAG pipeline using the local Qwen models; writes eval_results.csv.