Skip to main content
Glama
Youssef-AMARZOU

Local RAG MCP Server

by Youssef-AMARZOU
OverviewSchemaRelated ServersScoreDiscussions
Python

AI Orchestration MVP — Local-First RAG Pipeline

Bypass cloud API costs. Zero latency. Total privacy.

A production-ready, modular RAG (Retrieval-Augmented Generation) pipeline that runs entirely on your machine. Built with custom TCP socket framing and the Model Context Protocol (MCP) for modular, maintainable AI architectures.

Features

Local-First Architecture Every component runs locally. No API keys. No recurring cloud bills. No data leaves your machine.

Custom TCP Socket Framing Length-prefixed binary protocol for reliable message passing between components. Production-grade networking without HTTP overhead.

Model Context Protocol (MCP) Modular tool-calling interface. Register tools, discover them dynamically, and execute them over TCP — decoupling the LLM from your business logic.

RAG Pipeline

  • ChromaDB vector store for document embeddings

  • Sentence-transformers for local embedding generation

  • Ollama integration for local LLM inference (Llama 3.2, Mistral, etc.)

  • Ingest text, files, or entire directories

Streamlit Chat Interface Ready-to-use conversational UI. Start the server, ingest documents, and start asking questions.

Related MCP server: LightRAG MCP Server

Architecture

┌──────────────────────────────────────────────────┐
│                Streamlit UI (port 8501)            │
│  ┌──────────────────────────────────────────────┐ │
│  │            Orchestrator Instance               │ │
│  │                                                │ │
│  │  ┌─────────────┐   ┌──────────────────────┐   │ │
│  │  │  TCP Server  │◄──┤  MCP Server           │   │ │
│  │  │  (port 5555) │   │  - query_rag tool     │   │ │
│  │  └──────┬──────┘   │  - ingest_file tool    │   │ │
│  │         │          │  - ingest_text tool    │   │ │
│  │         │          │  - status tool         │   │ │
│  │         │          └──────────┬─────────────┘   │ │
│  │         │                     │                  │ │
│  │         │          ┌──────────┴─────────────┐   │ │
│  │         │          │  RAG Pipeline            │   │ │
│  │         │          │  ┌────────────────────┐ │   │ │
│  │         └──────────┤  │  ChromaDB (vector) │ │   │ │
│  │                    │  └────────────────────┘ │   │ │
│  │                    │  ┌────────────────────┐ │   │ │
│  │                    │  │  SentenceTransformer│ │   │ │
│  │                    │  └────────────────────┘ │   │ │
│  │                    │  ┌────────────────────┐ │   │ │
│  │                    │  │  Ollama (local LLM)│ │   │ │
│  │                    │  └────────────────────┘ │   │ │
│  │                    └─────────────────────────┘   │ │
│  └──────────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────────┘

External TCP clients can connect on port 5555
and use the MCP protocol to query tools remotely.

Quick Start

1. Prerequisites

# Install Python dependencies
pip install -r requirements.txt

# Install and start Ollama (for local LLM)
# https://ollama.com
ollama pull llama3.2
ollama serve

2. Start the Orchestrator

python orchestrator.py

3. Launch the Chat UI

streamlit run streamlit_app.py

Open http://localhost:8501 in your browser.

4. Ingest Documents

In the sidebar:

  • Paste text directly, or

  • Provide a file path

Then ask questions in the chat.

TCP Client Example

from tcp_network_module import TCPClient
from mcp_server import MCPClient

def send(msg):
    client.send(msg)

client = TCPClient("127.0.0.1", 5555)
mcp_client = MCPClient(send)

def handle_action(msg):
    mcp_client.handle_response(msg)

client.on("mcp", handle_action)
client.connect()

# Query the RAG pipeline
client.send({"action": "call_tool", "tool": "query_rag",
             "arguments": {"question": "What is in my documents?"}})

Project Structure

ai-orchestration-mvp/
├── tcp_network_module.py   # TCP framing (server + client)
├── mcp_server.py           # Model Context Protocol implementation
├── rag_pipeline.py         # ChromaDB + Ollama RAG pipeline
├── orchestrator.py         # Main coordinator (entry point)
├── streamlit_app.py        # Chat UI
├── requirements.txt        # Python dependencies
├── examples/
│   └── basic_usage.py      # TCP client example
└── README.md               # This file

Why This Architecture?

Cloud APIs

This Architecture

$50–$500+/month recurring

$0 recurring (one-time setup)

500ms–3s latency

50–200ms local latency

Data sent to third parties

Data never leaves your machine

Rate limits apply

No rate limits

Requires internet

Works fully offline

Requirements

  • Python 3.10+

  • Ollama (free, local LLM runner)

  • 8GB+ RAM recommended

License

MIT

This server cannot be installed

F
license - not found
Not graded
quality - not tested
C
maintenance

How are these scores calculated?

Maintenance

–Maintainers
–Response time
–Release cycle
–Releases (12mo)
Commit activity

Resources

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Related MCP Servers

  • DocAgent-MCP

    wuliuke
    F
    license
    Not graded
    quality
    C
    maintenance
    Enables local document question-answering and retrieval via MCP, supporting multi-turn conversation, intent recognition, and tools for document search, Q&A, and summarization.
    5

  • RAG MCP Server

    arbaz-builds
    F
    license
    Not graded
    quality
    B
    maintenance
    Exposes a Retrieval-Augmented Generation pipeline as MCP tools, allowing users to index documents and query them through any MCP-compatible client like Claude or IDEs.

View all related MCP servers

Related MCP Connectors

  • Backtest360

    MCP server exposing the Backtest360 engine API as tools for AI agents.

  • anythingmcp

    Self-hosted MCP gateway: turn any API, database or MCP server into AI connectors — no code.

  • saagarpatel.dev Portfolio

    Agent-native MCP server over the public saagarpatel.dev corpus. Read-only, stateless.

View all MCP Connectors

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/Youssef-AMARZOU/ai-orchestration-mvp'

If you have feedback or need assistance with the MCP directory API, please join our Discord server