🔌 MCP Docs Assistant

生产级检索增强生成（RAG）流水线，基于官方 Model Context Protocol 文档构建——通过 REST、MCP 工具和 Docker 提供。

用自然语言提问关于 MCP 的问题（架构、构建服务器/客户端、工具/资源/提示词、安全等），即可获得基于真实文档、可验证答案来源的回答，内置护栏、PII 脱敏、重排序、语义缓存和幻觉检测。

✨ 功能特性

Related MCP server: RLM Knowledge Base

Architecture

flowchart TD
    A[User Question] --> B[Guard Input<br/>NeMo Guardrails]
    B -->|blocked| Z[Refusal message]
    B -->|allowed| C[Mask Input PII<br/>Presidio]
    C --> D[Condense Follow-up<br/>into standalone question]
    D --> E{Semantic<br/>Cache Hit?}
    E -->|yes| F[Return cached answer]
    E -->|no| G[Retrieve Top-15<br/>Qdrant Vector Store]
    G --> H[Rerank Top-5<br/>Cross-Encoder]
    H --> I[Fit to Token Budget]
    I --> J[Generate Answer<br/>Portkey: Gemini / Groq]
    J --> K[Guard Output<br/>leak / PII pattern check]
    K --> L[Hallucination Check<br/>LLM-as-judge]
    L --> M[Mask Output PII]
    M --> N[Cache + Store History]
    N --> O[Return Answer]

Everything in the direct graph is run by modules in [rag_pipeline/] (./rag_pipeline), which are wired in a LangGraph StateGrap in [rag_pipeline/graph.py]. rag_core.py dial every dependency once-as a singleton-and builds a stable wrapper of APIs-chat()、search()、get_history (), cache_stats () — that are used identically by the REST layer ([main.py)] and the MCP layer ([mcp_server.py]). No matter which interface a request comes from, same vector store / cache / history is used.

📁 Project Structure

mcp-docs-rag-assistant/
├── main.py                    # FastAPI app — REST endpoints + mounts MCP at /mcp
├── mcp_server.py               # MCP tools (stdio standalone, or mounted in main.py)
├── rag_core.py                 # Singleton facade wiring the whole pipeline together
├── rag_pipeline/
│   ├── config.py                 # Env vars / secrets (single source of truth)
│   ├── logging_setup.py          # Logging + Logfire
│   ├── gateway.py                 # Portkey multi-key LLM gateway
│   ├── errors.py                   # Retry + safe-node error handling
│   ├── ingestion.py                 # MCP docs loader + splitter
│   ├── vectorstore.py                # Embeddings + persistent Qdrant store
│   ├── reranker.py                    # Cross-encoder reranking
│   ├── pii_masking.py                  # Presidio PII masking
│   ├── guardrails.py                    # NeMo Guardrails (Colang 2.x)
│   ├── token_management.py               # Context window budgeting
│   ├── semantic_cache.py                  # Embedding-similarity cache
│   ├── query_condensation.py               # Follow-up question rewriting
│   ├── hallucination.py                     # Runtime hallucination judge
│   └── graph.py                              # LangGraph StateGraph — full pipeline
├── scripts/
│   └── evaluate_ragas.py        # Offline RAGAS evaluation (25 reference Q&A)
├── tests/
│   └── test_pipeline.py         # Fast smoke tests (no API keys needed)
├── configs/guardrails/           # Colang rail files (generated at first run)
├── data/                          # Persisted Qdrant vector store (gitignored)
├── notebooks/                      # Original development notebook
├── Dockerfile
├── docker-compose.yml
├── requirements.txt
└── .env.example

Getting Started

Prerequisites

• Python 3.11+

• LLP API keys: Google AI Studio (Gemini, ×2)、Groq (×2)、Portkey (gateway + config id)

1. clone and set up a virtual environment

git clone https://github.com/<your-username>/mcp-docs-rag-assistant.git
cd mcp-docs-rag-assistant
python -m venv venv
venv\Scripts\activate          # Windows
# source venv/bin/activate     # macOS/Linux

2. Install dependencies

pip install -r requirements.txt
python -m spacy download en_core_web_sm   # required by Presidio for PII detection

3. Configure environment variables

cp .env.example .env

Open .env and fill in with your own keys (GEMINI_API_KEY_1/2, GROQ_API_KEY_1/2, PORTKEY_API_KEY, PORTKEY_CONFIG_ID).

4. Launch the server

uvicorn main:app --reload

First time only:: the vector store doesn't exist yet, so server ingests MCP docs and embs in limited-rate batches—this can take 5–10 minutes. On later runs it loads the stored store from data/qdrant_mcp_db/ instantly.

Once you see Startup: RAG pipeline ready., go to:

• http://localhost:8000/docs — interactive Swgger UI, try POST /chat

• http://localhost:8000/health — health check

📡 RST API

🔌 Using it as a MCP server

Standalone (stdio) — for Claude Desktop

Run directly:

python mcp_server.py

Or point a local MCP host at it, e.g. in Claude Desktop's config (claude_desktop_config.json):

{
  "mcpServers": {
    "mcp-docs-assistant": {
      "command": "python",
      "args": ["E:\\mcp-docs-rag-assistant\\mcp_server.py"]
    }
  }
}

Remote (streamable-http) — via FastAPI

When main.py is running, the same MCP tools are reachable at:

http://localhost:8000/mcp

Available tools: ask_mcp_docs, search_mcp_docs, get_conversation_history, cache_stats.

Docker

The only prerequisite is Docker Desktop (with Docker Compose), so you don't need to install Python, pip dependencies, or SpaCy model separately. All is done inside the image at build time (see the Dockerfile — it runs pip install -r requirements.txt and python -m spacy download en_core_web_sm).

# 1. Make sure .env exists (same as the local setup, step 3 above)
cp .env.example .env   # then fill in real keys

# 2. Build and run
docker compose up --build

That single command builds the image, installs everything, and starts the container. The data/ folder is mounted as a volume (see docker-compose.yml), so the vector store survives container restarts — you only pay the first-run embedding cost once.

Server is available at same as running locally: http://localhost:8000/docs.

To stop:

docker compose down

To rebuild after code changes:

docker compose up --build

🐳 Testing

Fast smoke tests — no API keys or network calls required (fake embeds):

pip install pytest
pytest tests/ -v

📄 Offline Evaluation (RAGAS)

The pipeline is evaluated against 25 hand-written MCP questions with reference answers using RAGAS:

python scripts/evaluate_ragas.py

This does not require the server to be running — it constructs the pipeline itself (same class as main.py/mcp_server.py) and displays metrics table:

• Faithfulness — is the answer based on retrieved context?

• Response Relevancy — does the answer exactly address the question?

• Context Precision — is retrieved context relevant?

• Context Recall — does the retrieved context cover the reference answer's needs?

⏱️ A few minutes: each of 25 questions performs a real retrieval + generation, then every metric is judged by LLM-as-judge.

⚙️ Configuration reference

All config in .env (see .env.example). Key vars:

🛠️ Tech Stack

FastAPI · LangChain · LangGraph · Qdrant · Portkey · Sentence-Transformers · Presidio · NeMo Guardrails · RAGAS · MCP Python SDK · Docker

📄 License

MIT — see LICENSE.