care-rag-assistant

A question-answering assistant over care-center procedure documents that cites its sources. It runs fully locally: data never leaves the server and no external LLM API is needed.

• Local LLM: Ollama (gemma3:4b for chat, multilingual bge-m3 for embeddings)

• RAG: PostgreSQL + pgvector (HNSW index, cosine similarity)

• API: TypeScript, Node.js, Hono, Zod

• MCP server: The same RAG pipeline exposed as MCP tools, so any MCP-capable AI client can query the documents

• Infrastructure: One command with Docker Compose

The sample documents, the system prompt and the answers are in Turkish. The project doubles as an example of a RAG pipeline working on a non-English corpus with a small local model.

The documents under docs/ are fictional demo data. They are not the procedures of any real organization and are not medical advice.

Architecture

                 ┌──────────── ingest (CLI) ─────────────┐
 docs/*.md ──▶ chunk (heading-aware, overlap) ──▶ Ollama embed ──▶ PostgreSQL + pgvector
                                                                        │
 question ──▶ Ollama embed ──▶ vector search (top-k, min score) ◀───────┘
                                   │
                    below score threshold ──▶ "not found" (model is not called)
                                   │
                    sources + question ──▶ Ollama chat ──▶ answer with [1], [2] citations
                                   │
                    HTTP API (/api/ask, /api/search)  ·  MCP (stdio)

Related MCP server: ragi

Design decisions

Running

Docker Compose

On macOS, run Ollama natively so it can use the GPU (inside Docker it is CPU-only and several times slower). Postgres and the app run in Docker:

brew install ollama && ollama serve &          # or the Ollama desktop app
ollama pull gemma3:4b && ollama pull bge-m3    # ~4.5 GB
docker compose up -d --build
docker compose exec app node dist/ingest.js

On a Linux server, everything can run in containers; the models service pulls the models on first start:

OLLAMA_URL=http://ollama:11434 docker compose --profile ollama up -d --build
docker compose exec app node dist/ingest.js

curl -s localhost:3000/api/ask \
  -H 'content-type: application/json' \
  -d '{"question":"Yüksek riskli sakinler gece ne sıklıkla kontrol edilir?"}' | jq

Response (from a real run; wording can vary slightly between runs):

{
  "answer": "Yüksek riskli sakinler gece boyunca iki saatte bir kontrol edilir [1].",
  "grounded": true,
  "sources": [
    { "id": 1, "source": "dusme-onleme-protokolu.md", "heading": "Düşme Önleme Protokolü > Gece Kontrolleri", "score": 0.719 },
    { "id": 2, "source": "dusme-onleme-protokolu.md", "heading": "Düşme Önleme Protokolü > Risk Değerlendirmesi", "score": 0.612 }
  ]
}

A question the documents do not cover (e.g. "Otoparkın şifresi nedir?", "What is the parking lot password?") returns grounded: false with a fixed "not found" answer.

Local development

ollama pull gemma3:4b && ollama pull bge-m3
docker compose up -d db
cp .env.example .env
npm install
npm run ingest
npm run dev

API

When API_KEY is set, /api/* requires Authorization: Bearer <key> (constant-time comparison).

MCP server

Exposes two tools: search_care_documents and ask_care_assistant. Add it to your MCP client configuration:

{
  "mcpServers": {
    "care-rag": {
      "command": "npx",
      "args": ["tsx", "/path/to/care-rag-assistant/src/mcp.ts"],
      "env": { "DATABASE_URL": "postgres://rag:rag@localhost:5432/rag" }
    }
  }
}

Configuration

Evaluation

eval/questions.json holds 15 questions the documents answer (each with the section that should be retrieved) and 5 off-topic questions that must be rejected. npm run eval runs them against the real Ollama + Postgres stack.

With nomic-embed-text the on-topic and off-topic score ranges overlap, so no threshold can separate them. bge-m3 leaves a clear gap, and MIN_SCORE=0.5 sits in the middle of it.

For answer generation, qwen2.5:3b produced a contradictory answer to a yes/no question about medication ("can I give a make-up dose?"); gemma3:4b answered all spot-check questions correctly and consistently across runs. Answers take about 1.5–3 s on an Apple Silicon Mac (first request after start loads the model and is slower).

Tests

npm test          # chunking, RAG flow, HTTP validation and auth
npm run typecheck

Tests use fake LLM and vector store implementations; they do not need Ollama or Postgres.

Roadmap

• Hybrid search (pgvector + PostgreSQL full-text) and reranking

• Answer-level evaluation (LLM-as-judge) in addition to retrieval metrics

• PDF / DOCX ingest

• Streaming answers (SSE)

License

MIT