doc-rag-mcp

Turn any folder of docs (PDFs, markdown, text) into a local, queryable knowledge base exposed as an MCP server — so any MCP-compatible agent (Claude Code, or otherwise) can search your docs for relevant passages instead of reading whole files into its context window. There's a dedicated section below for Claude Code specifics, since that's what this was built and tested against, but nothing about the server itself is Claude-specific.

Embeddings run via the Voyage AI API (requires a VOYAGE_API_KEY, no multi-GB local model download); storage is Postgres with the pgvector extension. Runs entirely on your own machine (or wherever you deploy it) — your documents and queries never go anywhere except to the embedding API.

How it works

Ingestion — point it at a folder (or a single file), it chunks and embeds anything new or changed, and skips the rest:

flowchart TD
    A[docs folder] --> B[chunk text]
    B --> C{content hash<br/>changed?}
    C -- no --> D[skip, no API call]
    C -- yes --> E[embed via Voyage AI]
    E --> F[(Postgres + pgvector)]

Querying — the agent calls the MCP tool, which embeds the question and finds the closest chunks by cosine similarity:

flowchart TD
    A[MCP client] -- query_docs --> B[MCP server]
    B --> C{cached?}
    C -- yes --> G[return cached answer]
    C -- no --> D[embed query via Voyage AI]
    D --> E[(Postgres + pgvector<br/>cosine search)]
    E --> F[top-k matching chunks]
    F --> A

Both flows share one Postgres table (doc_chunks), namespaced by a collection column so multiple doc sets can coexist in the same database.

Related MCP server: ragi

1. Install

cd doc-rag-mcp
cp .env.example .env   # fill in VOYAGE_API_KEY and POSTGRES_PASSWORD at least
python -m venv .venv
source .venv/bin/activate        # Windows: .venv\Scripts\activate
pip install -r requirements.txt

Or just run make install (also used automatically by the other make targets — see Makefile for the full list: ingest, ingest-rebuild, serve, serve-http run things locally in a venv against Postgres; db starts just Postgres in Docker for that local mode; build, up, down, restart (same image, just restarts the process), redeploy (rebuild + recreate — use this after changing src/*.py, Dockerfile, or docker-compose.yaml), reindex, reindex-rebuild, logs, clear-cache drive the fully containerized setup in docker-compose.yaml instead; db-reset wipes all indexed data).

2. Add your docs

Two ways to point at your docs, via RAG_DOCS_DIR in .env:

• Drop them in docs/ (the default, subfolders are fine) — architecture docs, runbooks, research papers, whatever you've got:

  docs/
    architecture-overview.md
    api-design.pdf
    adr/
      0001-use-postgres.md

• Or point at any path already on your machine — a folder or a single file, anywhere: RAG_DOCS_DIR=/Users/you/Documents/some-project-docs. No need to copy anything into this repo. (Docker only sees what's mounted into docs/, so this option is for running ingest.py/make ingest locally rather than via make reindex.)

Supported: .pdf, .md, .markdown, .txt.

3. Build the index

Needs Postgres reachable first — make db starts one in Docker (exposed on localhost:5433) if you're running ingest/serve locally rather than fully in Docker.

make db
python src/ingest.py --docs ./docs --collection project_docs
# or: make ingest

Re-run this any time the docs change — it's incremental: unchanged files are skipped by content hash (no re-embedding, no API cost), changed/new files are re-embedded, and files removed from the folder have their chunks dropped. Pass --rebuild (or make ingest-rebuild) to force a full re-embed of everything (this also happens automatically if RAG_EMBED_MODEL, RAG_CHUNK_SIZE, or RAG_CHUNK_OVERLAP changed since the last run).

4. Connect it to your MCP client

Any MCP client connects one of two ways — both are covered in more detail in section 5 for the HTTP case:

• stdio (the common case for a local client): the client spawns src/server.py itself as a subprocess, using your venv's python binary so it picks up the installed packages:

  /full/path/to/doc-rag-mcp/.venv/bin/python /full/path/to/doc-rag-mcp/src/server.py

  Most clients want that as a command + args pair in their own config format — see your client's docs for exactly where that goes (for Claude Code specifically, see the dedicated section below).

• HTTP: the client connects to a URL instead of spawning anything — see section 5 ("Run it on a network / behind a tunnel") for starting the server this way and the auth token it requires.

Once connected, it exposes six tools:

• query_docs(query, top_k=5) — semantic search over the indexed chunks, returns the matching passages with their source file and a relevance score. If RAG_RERANK_MODEL is set, over-fetches candidates by vector search and re-scores them with Voyage's rerank API for more accurate results (see Reranking below); otherwise ranks by cosine similarity alone.

• list_documents() — lists every file currently indexed

• reindex_docs(force_rebuild=False, confirm_large_removal=False) — re-scan RAG_DOCS_DIR and embed anything new or changed, without leaving the chat to run ingest.py by hand. Incremental by default; blocks until done (can take a while for a large/changed doc set); clears the query cache automatically afterward. Refuses to remove a large fraction of previously-indexed files in one go unless confirm_large_removal=True — that pattern is much more often a misconfigured docs path than genuine bulk deletion, and the response explains what was left untouched if it trips.

• cache_stats() — hit/miss counts, hit rate, and current cache size

• usage_stats() — cumulative Voyage AI token usage (by model/operation) plus cache stats, combined. This is a local tally of calls made through this server/ingest.py, not your Voyage account's authoritative usage — check the dashboard for that.

• clear_cache() — drop the query cache (run after re-running src/ingest.py directly; reindex_docs does this for you)

Then just ask your agent things like "check the docs for how auth is handled" and it'll call query_docs on its own.

Caching

Query results are cached in-process (LRU, TTL-based) keyed by the normalized (query, top_k) pair. A repeated question skips both the embedding call and the Postgres round-trip. Tune it with env vars:

"env": {
  "RAG_CACHE_MAX_SIZE": "256",
  "RAG_CACHE_TTL_SECONDS": "600"
}

The cache lives in the running server process's memory, so it resets whenever the client restarts the server, and it will not know if you re-run src/ingest.py while it's running — call clear_cache() afterward so it doesn't keep serving answers from the old index (reindex_docs does this automatically).

Reranking (optional)

By default, query_docs ranks purely by embedding cosine similarity — fast and cheap, but it scores the query and each chunk independently, so it can miss subtleties a direct query/document comparison would catch. Set RAG_RERANK_MODEL (e.g. rerank-2.5) to add a rerank pass: the server fetches a larger candidate pool by vector search (top_k × 4, capped at 100) and re-scores those against the actual query text with Voyage's rerank API, returning the best top_k after that second pass.

This costs its own tokens on every query, in addition to the query embedding — a rerank call processes full chunk text rather than just the short query, so it's typically the larger of the two costs. Check usage_stats() (it tracks rerank calls as their own operation) to see the actual cost for your usage pattern before deciding whether to leave it on.

Streaming

MCP tool calls return one final result — there's no token-by-token streaming of the response body the way a chat completion streams. What the protocol does support is progress notifications sent while a tool is still running; a client that surfaces those (Claude Code does) shows activity immediately rather than waiting on the final payload. query_docs reports progress as each matching chunk is resolved (and logs cache hit/miss), so you see activity right away instead of blocking until the whole joined string is ready — useful mainly on a cache miss with a larger top_k, or on the first call while the embedding model is loading.

5. Run it on a network / behind a tunnel

By default the server talks stdio and only exists as a subprocess of a local MCP client. To let a remote agent reach it, run it as a standalone HTTP server instead:

make auth-token   # generates a token and writes RAG_AUTH_TOKEN into .env
make serve-http    # or, for the Docker Compose setup: make up

make auth-token creates .env (from .env.example) if it doesn't exist yet, and overwrites RAG_AUTH_TOKEN in place if it does — safe to re-run any time you want to rotate it. Clients need that token. The server refuses to start over --transport http without RAG_AUTH_TOKEN set, and every request must send Authorization: Bearer <token>; requests without it get a 401. Don't run this without a token — whatever's in docs/ becomes readable to anyone with the URL and token.

Keep --host 127.0.0.1 (the default) unless you specifically want the port open on your LAN — tunnel tools below connect out from your machine, so the server itself never needs to bind 0.0.0.0.

Exposing it — pick one

The port below is whatever RAG_PORT is set to (default 8743 — deliberately not 8000/5000/3000, since those are usually already taken by something else).

Cloudflare Tunnel (no account needed for a quick tunnel):

cloudflared tunnel --url http://localhost:8743

Gives you a https://random-words.trycloudflare.com URL. Full MCP endpoint is https://random-words.trycloudflare.com/mcp.

ngrok:

ngrok http 8743

Endpoint: https://<subdomain>.ngrok-free.app/mcp.

Tailscale Funnel (if the querying agent is on your tailnet, use serve instead of funnel to skip the public-internet exposure entirely):

tailscale funnel 8743     # public
tailscale serve 8743      # tailnet-only, no auth token strictly needed

Plain SSH reverse tunnel (if you have a VPS):

ssh -R 8743:localhost:8743 user@your-vps

Connecting a remote client

Register it as an HTTP-type MCP server rather than a command/stdio one — every client's config format is slightly different, so check your client's docs; it needs the tunnel URL (https://your-tunnel-url/mcp) and an Authorization: Bearer YOUR_TOKEN header. For Claude Code specifically, see the dedicated section below.

Quick sanity check without any MCP client at all:

curl -i https://your-tunnel-url/mcp                       # expect 401
curl -i -H "Authorization: Bearer YOUR_TOKEN" \
     -H "Accept: application/json, text/event-stream" \
     -H "Content-Type: application/json" \
     -X POST https://your-tunnel-url/mcp \
     -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-06-18","capabilities":{},"clientInfo":{"name":"test","version":"0"}}}'

Notes on running this exposed

• Quick tunnels (cloudflared/ngrok free tier) get a new random URL every restart — fine for testing, get a named/reserved tunnel if you want a stable address.

• The bearer token is a shared secret, not per-user auth — anyone with the token and URL has full read access to query_docs/list_documents. Rotate it (RAG_AUTH_TOKEN) if it leaks.

• The process needs to stay running for the tunnel to serve anything — run it under systemd, tmux, or pm2 rather than a bare foreground shell if you want it up long-term.

6. Quick smoke test (optional, without any MCP client)

PYTHONPATH=src python -c "
import db
db.EMBED_MODEL, db.EMBED_DIM  # sanity check config
voyage = db.get_voyage_client()
conn = db.get_connection()
embedding = db.embed_texts(voyage, ['how does authentication work'], input_type='query')[0]
rows = conn.execute(
    'SELECT source, content FROM doc_chunks ORDER BY embedding <=> %s LIMIT 3',
    (embedding,),
).fetchall()
for source, content in rows:
    print(source, '-', content[:80])
"

Notes / next steps

• Chunking: currently a simple ~800-char sliding window snapped to paragraph/sentence boundaries. If retrieval quality is off for long documents, consider chunking by markdown headers instead (keeps each section intact).

• Embedding model: voyage-4-large (1024 dims by default) out of the box — Voyage's best general-purpose/multilingual retrieval model as of this writing. For a cheaper/faster option try voyage-4-lite (same 1024 default, same flexible 256/512/2048 options) — set RAG_EMBED_MODEL in .env. Since the dimension is baked into the Postgres vector column, changing RAG_EMBED_DIM (not needed for either of these two, since both default to 1024) requires make db-reset (wipes all data) and a full re-ingest; switching just the model name still forces a full re-embed automatically (see chunking/rebuild notes above) since embeddings from different models aren't comparable even at the same dimension.

• Scaling: ingestion is already incremental (keyed by file hash), so re-running it stays cheap as the doc set grows. The doc_chunks table has an HNSW index for approximate nearest-neighbor search, which scales well past what a project-sized doc set needs.

• Multiple collections: rows are namespaced by a collection column in the same table, so separate KBs (e.g. per project) just need a different --collection name at ingest time and matching RAG_COLLECTION per MCP server entry — no extra database needed. Note the embedding dimension (RAG_EMBED_DIM) is shared across all collections in one database.

Claude Code

Everything above works with any MCP client. This section is just the concrete Claude Code commands/config for the two connection modes from section 4.

stdio (local, the default) — Claude Code launches the server itself:

claude mcp add doc-rag-mcp -- /full/path/to/doc-rag-mcp/.venv/bin/python /full/path/to/doc-rag-mcp/src/server.py

Or generate .mcp.json automatically — make gen-mcp-json writes:

• doc-rag-mcp: stdio, with this machine's real venv path, src/server.py path, and current .env values (RAG_DATABASE_URL, RAG_COLLECTION, VOYAGE_API_KEY) filled in.

• doc-rag-mcp-http: only if RAG_AUTH_TOKEN is set (make auth-token) — points at http://localhost:<RAG_PORT>/mcp with the real token as a bearer header. This is for the server exposed locally by make up/ make serve-http, not a tunnel — for genuine remote access over a tunnel (see below), add that entry by hand with the actual tunnel URL, since it can't be known in advance.

No manual editing either way, and it merges into an existing .mcp.json rather than overwriting it, so other hand-added entries survive. Re-run it any time .env changes (rotated password, different port, etc.).

Prefer to write it by hand instead? cp .mcp.json.example .mcp.json and fill in the real venv path, password, and API key yourself (it ships with both a stdio entry and the HTTP entry below; delete whichever you don't need):

{
  "mcpServers": {
    "doc-rag-mcp": {
      "command": "/full/path/to/doc-rag-mcp/.venv/bin/python",
      "args": ["/full/path/to/doc-rag-mcp/src/server.py"],
      "env": {
        "RAG_DATABASE_URL": "postgresql://raguser:yourpassword@localhost:5433/ragdb",
        "RAG_COLLECTION": "project_docs",
        "VOYAGE_API_KEY": "your-voyage-api-key"
      }
    }
  }
}

HTTP (remote/tunnel, from section 5) — register it as an HTTP-type server rather than a command entry:

claude mcp add --transport http doc-rag-mcp-remote \
  https://your-tunnel-url/mcp \
  --header "Authorization: Bearer YOUR_TOKEN"

or in .mcp.json:

{
  "mcpServers": {
    "doc-rag-mcp-remote": {
      "type": "http",
      "url": "https://your-tunnel-url/mcp",
      "headers": {
        "Authorization": "Bearer YOUR_TOKEN"
      }
    }
  }
}

Either way, restart Claude Code (or run /mcp to reconnect) to pick up new or changed servers.

Contributing

Issues and PRs welcome. This is a small, focused tool — keep additions in that spirit rather than growing it into a framework.

License

MIT