Local MCP RAG Server

An offline, privacy-first Model Context Protocol (MCP) server that empowers LLM clients (like Claude Desktop) to chat with your local markdown engineering documentation.

How it Works

1. Indexer: Reads .md files, generates embeddings via Local Ollama (nomic-embed-text), and stores them in ChromaDB.

2. MCP Server: Exposes a search_internal_docs tool using the FastMCP framework over stdio.

3. Zero Cloud Leaks: All indexing and retrieval happens completely locally on your machine.

Related MCP server: rag-mcp

Setup

1. Requirements

• Python 3.12+

• Ollama running locally.

ollama pull nomic-embed-text
pip install -r requirements.txt

2. Index your docs

Place markdown files in /internal_docs and run:

python src/indexer.py

3. Connect to Claude Desktop

Add this to your claude_desktop_config.json (Mac: ~/Library/Application Support/Claude/claude_desktop_config.json):

{
  "mcpServers": {
    "local-architecture-brain": {
      "command": "python",
      "args": ["/absolute/path/to/local-mcp-rag-server/src/server.py"]
    }
  }
}

Restart Claude Desktop, and you can now ask it: "How does our internal payment service work?