rag-mcp-agent-demo
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@rag-mcp-agent-demo Search the docs for how the SQL safety gate works."
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
RAG + MCP Agent Demo
A compact, dependency-free reference implementation of the core pieces behind a production LLM/agent stack:
a RAG pipeline (embed → store → retrieve → assemble context),
an MCP server exposing read-only tools over stdio (JSON-RPC 2.0),
a Text-to-SQL safety gate that validates generated SQL before execution,
an offline evaluation harness (retrieval recall@k + SQL-guard accuracy),
and a small agent loop that routes between tools.
This is a sanitized, self-contained demo. It contains no employer code or data.
Why this exists
Most "RAG demos" show a notebook that retrieves a chunk. Production systems need the boring parts: a swappable vector store, a hard safety boundary in front of the database, and an evaluation harness that fails CI when quality regresses. This repo isolates those parts so they can be read, tested and reused.
Related MCP server: shop-sql-mcp
Architecture
┌─────────────┐
question ───▶│ Agent │ (plan → tool → observe → answer)
└──────┬──────┘
┌─────────┴─────────┐
▼ ▼
┌────────────┐ ┌──────────────┐
│ RAG │ │ SQL tool │
│ embed→store│ │ guard() │ ← blocks DML/DDL, enforces LIMIT
│ →retrieve │ └──────┬───────┘
└─────┬──────┘ ▼
│ ┌──────────┐
▼ │ SQLite │
┌──────────┐ └──────────┘
│ corpus │
└──────────┘
MCP server (stdio, JSON-RPC 2.0)
tools: search_docs · run_sql · list_tablesQuickstart
No third-party runtime dependencies (Python 3.10+).
# run the offline evaluation harness
python eval/run_eval.py
# ask the agent a question
PYTHONPATH=src python -m rag_mcp_demo.cli "how many customers are there"
PYTHONPATH=src python -m rag_mcp_demo.cli "how does the RAG pipeline retrieve chunks"
# run the MCP server over stdio
PYTHONPATH=src python -m rag_mcp_demo.mcp_serverInstall as a package (optional):
pip install -e ".[dev]"
rag-agent "total paid revenue"
pytestExample: MCP over stdio
echo '{"jsonrpc":"2.0","id":1,"method":"tools/list"}' | PYTHONPATH=src python -m rag_mcp_demo.mcp_serverEvaluation
eval/run_eval.py reports two signals and exits non-zero if either drops below 0.8:
Metric | Meaning |
retrieval recall@3 | fraction of golden questions whose expected document is in the top-3 |
sql guard accuracy | fraction of safe/unsafe SQL cases classified correctly |
Design decisions
Vector store behind an interface — in-memory for tests, Qdrant/pgvector in production.
Safety before execution — the SQL gate rejects non-
SELECTstatements and enforces a row limit; in production this is built on a real parser (e.g.sqlglot).Tools are read-only by default — the MCP server exposes only read operations.
Evals are first-class — a golden set and a harness that fails CI on regression.
LLM-agnostic agent — the router is heuristic so the control flow is testable without a model; swap in an LLM to replace
Agent.route().
Project layout
src/rag_mcp_demo/
embeddings.py # TF-IDF embedder + cosine (swap for a real model)
vector_store.py # in-memory vector store
rag.py # corpus loading + retrieval pipeline
sql_guard.py # pre-execution SQL safety gate
db.py # sample SQLite database
agent.py # minimal agent loop
mcp_server.py # MCP server (stdio, JSON-RPC 2.0)
cli.py # `rag-agent` CLI
eval/ # golden set + evaluation harness
tests/ # pytest suite
data/corpus.jsonl # sample documentsLicense
MIT — see LICENSE.
Available Tools
3 toolslist_tablesA
List tables in the sample database (read-only).
Inputs: no named parameters.
run_sqlA
Run a read-only SQL SELECT against the sample database.
Inputs: sql (required).
search_docsC
Search the documentation corpus (read-only).
Inputs: query (required), top_k.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
TDQS
Scored across 3 tools
Each tool has a clearly distinct purpose: search_docs targets the documentation corpus, run_sql executes SQL queries, and list_tables lists database tables. There is no overlap or ambiguity between them.
All tool names follow a consistent verb_noun snake_case pattern: search_docs, run_sql, list_tables. The naming convention is uniform and predictable.
With 3 tools, the set is well-scoped for a demo RAG agent that combines documentation search and read-only database exploration. Each tool serves a distinct purpose and none feel redundant or excessive.
The core workflows of searching docs and querying the database are covered, but there is a minor gap: list_tables only lists table names without exposing schema details, and there is no tool to retrieve full document content beyond search results. Still, the surface is adequate for a demo.
Maintenance
Related MCP Connectors
Security layer for AI agents: Retrieve for controlled SQL reads, Guard for allow/deny writes.
Safe, read-only Postgres and MySQL access for AI agents. Audit log + column-level controls.
Query 40 databases from Claude, ChatGPT, or Cursor — on any device. Read-only, encrypted, audited.
Query your org's data in natural language — read-only MCP access to SQL, NoSQL, files & warehouses.
Related MCP Servers
- AlicenseAqualityDmaintenanceExposes SQLite database query tools and markdown document resources over JSON-RPC 2.0 stdio transport, enabling AI assistants to read and search documents and execute read-only SQL queries.81MIT
- FlicenseNot gradedqualityDmaintenanceProvides AI agents read-only analytical access to a SQLite database over stdio, with tools for listing tables, describing schemas, and running paginated SQL queries.-
- FlicenseAqualityCmaintenanceEnables safe read-only analysis of a local SQLite e-commerce database over stdio, providing tools for schema inspection and parameterized SELECT queries.2-
- AlicenseNot gradedqualityCmaintenanceEnables LLM agents to run read-only SQL queries and inspect database schema over stdio by default, with optional opt-in DML/DDL and PL/SQL execution that auto-commits.MIT