rag-mcp
Provides local, offline embeddings for document indexing and query embedding using Ollama models, and is the default embedding provider for the RAG memory server.
Supports OpenAI-compatible /v1/embeddings endpoints for embedding document chunks and search queries, enabling use of OpenAI or compatible embedding services.
This server does not currently have a deployable release on Glama. Follow the repository's setup instructions to install and configure it for your MCP client, including any required credentials and local dependencies.
rag-mcp
A self-hosted RAG memory server for AI assistants, exposed over the Model Context Protocol (MCP). Give any MCP-capable LLM client (Claude Desktop / Claude Code, LibreChat Agents, Open WebUI via a proxy, your own tooling) searchable long-term memory over your own documents — runbooks, incident reports, RCAs, wiki exports, anything in markdown or PDF.
Read-only LLM surface — the model can only search; writes happen out-of-band through a batch ingestion job or a token-gated internal API.
Hybrid retrieval — dense semantic vectors + BM25 keyword sparse vectors, fused with Reciprocal Rank Fusion. Exact tokens (
CrashLoopBackOff, error codes, resource names) are found even when embeddings miss them.Optional cross-encoder reranking (Cohere/Jina-compatible) for precision.
Pluggable embeddings — local Ollama (offline default) or any OpenAI-compatible
/v1/embeddingsendpoint. One env var to switch.Markdown + PDF ingestion with YAML front matter, heading-aware chunking, idempotent re-runs.
Vendor-neutral MCP — works with any client that speaks streamable-http MCP.
Architecture
WRITE PATH — populate the KB
┌───────────────────────────┐
│ knowledge/** │ your markdown & PDF docs,
│ runbooks · incidents · │ optional YAML front matter
│ wikis · post-mortems │
└─────────────┬─────────────┘
│
▼
┌───────────────────────────┐ embed ┌─────────────────────────────┐
│ rag-ingest │ chunks │ embedding provider │
│ · parses front matter │───────────►│ ollama (offline default) │
│ · heading-aware chunks │ │ or any OpenAI-compatible │
│ (PDF pages = sections) │◄───────────│ /v1/embeddings endpoint │
│ · idempotent upserts, │ vectors └──────────────▲──────────────┘
│ stale-tail cleanup │ │
└─────────────┬─────────────┘ │ the SAME
│ upsert │ provider also
▼ │ embeds queries
┌───────────────────────────┐ │ (see rag-mcp ↘)
│ Qdrant (v1.12) │ │
│ collection: rag_kb │ │
│ dense (cosine · HNSW) │ │
│ bm25 (sparse · IDF) │ │
└─────────────▲─────────────┘ │
│ │
│ 1. hybrid search: │
│ Prefetch(dense)+Prefetch(bm25) │
│ fused with Reciprocal Rank Fusion │
│ │
┌─────────────┴───────────────────────────────┐ │
│ rag-mcp :8084/mcp │ │
│ FastMCP server — READ-ONLY tool surface │ │
│ │ 2. em- │
│ rag_search search_incidents │ beds │
│ search_runbooks rag_collections │◄────────┘
│ rag_health │ the
│ (+ optional step 3: cross-encoder rerank │ query
│ via a Cohere/Jina-compatible /rerank) │
└──▲──────────────────────▲───────────────────┘
│ │
│ MCP │ POST /internal/knowledge/*
│ streamable-http │ capture · similar · feedback · stats
│ (search queries) │ token-gated; NEVER visible to the LLM
│ │
┌──┴───────────────────┐ │
│ MCP clients │ ▼
│ Claude Code · │ trusted automation
│ LibreChat · │ (agent / CI capturing
│ your own agents │ incidents + human feedback)
└──────────────────────┘Related MCP server: Markdown RAG MCP
Quickstart
Prerequisites: Docker (+ Compose v2), and for the default offline embedding path Ollama on the host:
ollama pull nomic-embed-text # once
git clone https://github.com/mmelmesary/rag-mcp.git && cd rag-mcp
cp .env.example .env # defaults work out of the box
docker compose up -d --build # starts Qdrant + rag-mcp
docker compose run --rm rag-ingest # index the sample docs in ./knowledgeVerify:
curl -s http://localhost:8084/mcp # MCP endpoint (400 without handshake = alive)
curl -s http://localhost:6333/collections # rag_kb exists with pointsWire it into an MCP client
The server speaks streamable-http at http://localhost:8084/mcp.
Claude Code (.mcp.json in your project):
{
"mcpServers": {
"rag": {
"type": "http",
"url": "http://localhost:8084/mcp"
}
}
}LibreChat (librechat.yaml):
mcpServers:
rag:
type: streamable-http
url: http://rag-mcp-server:8084/mcp # container name when compose-networkedAny other MCP client: register an HTTP/streamable-http server at the URL above.
Tools exposed to the model
Tool | Purpose |
| Semantic search across the whole KB |
| "Has this happened before?" — |
| "What's the procedure?" — |
| List collections + point counts |
| Reachability of Qdrant and the embedding provider |
Filters: cluster is a soft narrow (empty same-cluster result retries fleet-wide);
component/doc_type are hard filters.
Add your own documents
Drop markdown or PDFs under knowledge/ (subfolders become the default doc_type,
e.g. knowledge/incidents/* → incident). Markdown supports optional YAML front
matter:
---
title: Longhorn volume stuck attaching
type: incident
tags: [longhorn, storage]
component: longhorn
cluster: prod-eu
---
# Body...Front matter is entirely yours: type, component, and cluster become
searchable filter fields, but they are free-form labels — use them for any
grouping that fits your domain (environments, customers, products, teams) or
omit them for plain semantic search.
Then re-index — no rebuild needed (the job bind-mounts ./knowledge):
docker compose run --rm rag-ingestIngestion is idempotent (chunk IDs derive from (source, chunk index)); shrunken
docs get their stale tail chunks deleted. Run it from CI/cron whenever docs change.
Known limitations: deleting or renaming a file does not remove its old chunks — use
docker compose run --rm rag-ingest --recreate after removals/renames.
Switching to a hosted embedding provider
# .env
EMBEDDINGS_PROVIDER=openai
EMBEDDINGS_BASE_URL=https://api.openai.com # or LiteLLM proxy / Azure gateway / TEI
EMBEDDINGS_API_KEY=sk-...
EMBEDDINGS_MODEL=text-embedding-3-smallThen rebuild the collection (ingest and query must always share provider+model):
docker compose run --rm rag-ingest --recreateConfiguration
All configuration is environment-driven — see .env.example for the full annotated list and docs/DESIGN.md for the design rationale, retrieval-pipeline deep-dive, internal write API, and complete env table.
Internal write API (optional)
Besides the read-only MCP tools, the server exposes plain-HTTP routes
(/internal/knowledge/capture|similar|feedback|stats) so a trusted automation
process can write incident records ("knowledge flywheel"). These are not
visible to the LLM. Gate them with RAG_INTERNAL_TOKEN. See
docs/DESIGN.md.
Development
python3 -m pip install -r requirements.txt pytest
python3 -m pytest tests/
python3 server.py # run against a local Qdrant on :6333
QDRANT_URL=http://localhost:6333 python3 ingest.py --path knowledgeLicense
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
This server cannot be deployed
Maintenance
Related MCP Connectors
Cloud or self-hosted knowledge for AI agents: hybrid search, reranking, GraphRAG, scoped MCP tools.
Team docs served to AI agents over MCP - search, Markdown reads, version pinning, read audit.
Self-hosted AI workspace: semantic RAG document search, long-term memory, and AI chat via MCP.
Make your knowledge agent-ready. One MCP endpoint, 5 connectors, 3 search modes.
Related MCP Servers
- FlicenseNot gradedqualityBmaintenanceEnables any MCP-compatible AI assistant to search, filter, and retrieve information from a local document collection using a hybrid search pipeline with vector, BM25, reranking, and LLM enrichment.4-
- AlicenseNot gradedqualityDmaintenanceProvides semantic search over markdown documentation using RAG, allowing natural language queries and integration with MCP clients.1MIT
- FlicenseNot gradedqualityDmaintenanceProvides MCP tools to search engineering runbooks and historical incidents using semantic retrieval, supporting evidence-grounded incident investigation.-
- AlicenseAqualityCmaintenanceEnables any MCP host to search and answer over your own documents with hybrid BM25+dense retrieval, cross-encoder reranking, and grounded, cited responses that refuse when no evidence is found.5MIT