Document Q&A MCP Server

A medium-complexity MCP server that lets an MCP client (like Claude Desktop) ingest documents (PDF / DOCX / TXT / MD) and answer questions about them using retrieval-augmented generation (RAG).

How it works:

1. add_document extracts text, splits it into overlapping chunks, embeds each chunk locally with sentence-transformers, and stores it in a ChromaDB collection persisted to disk.

2. ask_question embeds your question, retrieves the most similar chunks from Chroma, and sends them + your question to a Groq-hosted LLM, which answers grounded only in that context.

1. Install

cd mcp-doc-qa
python -m venv venv
source venv/bin/activate      # Windows: venv\Scripts\activate
pip install -r requirements.txt

The first run will download the small local embedding model (all-MiniLM-L6-v2, ~80MB) from HuggingFace — needs internet once, then it's cached locally.

Related MCP server: Local RAG

2. Configure

cp .env.example .env

Edit .env and set GROQ_API_KEY (free key at https://console.groq.com/keys). Defaults for everything else are sensible.

3. Test it standalone (optional but recommended)

python server.py

This starts the server on stdio and will just sit there waiting for an MCP client — that's expected, it's not a web server. Press Ctrl+C to stop. If you'd rather sanity-check the pieces without an MCP client, open a Python shell and call store.add_document(...) / generate_answer(...) directly.

4a. Run it as a REST API (FastAPI)

Instead of (or alongside) the MCP server, you can run the same logic as a regular web backend:

uvicorn api:app --reload --port 8000

Then open http://127.0.0.1:8000/docs for interactive Swagger UI, or hit it directly:

# Upload a document
curl -X POST http://127.0.0.1:8000/documents/upload \
  -F "file=@/path/to/report.pdf"

# Ask a question
curl -X POST http://127.0.0.1:8000/ask \
  -H "Content-Type: application/json" \
  -d '{"question": "What was the Q3 revenue?"}'

# List documents
curl http://127.0.0.1:8000/documents

# Delete one document
curl -X DELETE http://127.0.0.1:8000/documents/<doc_id>

Both server.py (MCP) and api.py (FastAPI) call into the same qa_service.py module, so ingestion/retrieval/answer logic lives in one place — pick whichever interface fits your use case, or run both.

4b. Connect it to Claude Desktop

Add this to your Claude Desktop config (~/Library/Application Support/Claude/claude_desktop_config.json on macOS, %APPDATA%\Claude\claude_desktop_config.json on Windows):

{
  "mcpServers": {
    "document-qa": {
      "command": "/absolute/path/to/mcp-doc-qa/venv/bin/python",
      "args": ["/absolute/path/to/mcp-doc-qa/server.py"]
    }
  }
}

Restart Claude Desktop. You should see the document-qa server's five tools available in a new chat.

Tools exposed

(The FastAPI app exposes the equivalent operations as REST endpoints — see section 4a above.)

Notes & things to tune later

• Chunking: character-based with paragraph/sentence-aware breaks (document_loader.py). Swap in a smarter splitter (e.g. token-based) if you hit weird cuts.

• Embedding model: all-MiniLM-L6-v2 is small and fast. For better recall, try all-mpnet-base-v2 (slower, bigger) via .env.

• Groq model: defaults to llama-3.3-70b-versatile. Check https://console.groq.com/docs/models for current options.

• Persistence: the Chroma DB lives in ./chroma_db — delete that folder to fully reset, or just call clear_all_documents.

• Scanned PDFs: this uses pypdf text extraction, which won't work on image-only/scanned PDFs. Add OCR (e.g. pytesseract) if you need that.