Policy Document Q&A Assistant — RAG + MCP Server, deployed to production (LangChain, Azure AI Foundry, PostgreSQL/pgvector, Docker, Langfuse)

A question-answering assistant over real Australian public policy documents (Queensland Government financial/ICT policy and ATO federal tax guidance), built as a Retrieval-Augmented Generation (RAG) pipeline and exposed as an MCP (Model Context Protocol) server — so it plugs into Claude Desktop or any other MCP-compatible client as a callable tool, not just a one-off script. It's live, in production, right now:

🟢 policy-rag-mcp-server.onrender.com

Open it and ask a real question — no setup, no API key, nothing to clone. Or connect your own MCP client to the same server over the network at /mcp — see Setup & Usage below.

OBJECTIVE

A retrieval-augmented Q&A assistant over real government policy and tax guidance documents, built as a full production system rather than a notebook proof-of-concept — real source PDFs, a managed vector database, a protocol-compliant MCP interface other tools and agents can actually call, containerized and deployed to a real public URL rather than left running on a laptop.

A few things mattered enough to go deep on rather than stop at "it works once, locally": the Model Context Protocol, since it's the emerging standard for how AI agents call external tools and treating it as a first-class interface (not a toy example) means actually reasoning about protocol compliance and transport design, not just stdio vs. network but why each one exists; running vector search inside a database companies already operate (PostgreSQL + pgvector) instead of bolting on a dedicated vector store; standardizing on one cloud-native AI platform end-to-end rather than mixing providers; and instrumenting observability — cost, latency, what was actually retrieved and said — from day one instead of after something breaks in front of someone.

A project that only runs on one laptop, behind one person's Docker Desktop, doesn't really prove any of that holds up — it's a demo. So the target was a system somebody else could open and use unattended, the same way a real team would actually ship one: a public URL that works cold, a database that exists independently of any single machine, and an MCP server reachable by any client on the internet — not just one sharing the same filesystem as the server. That's why the finished system serves two audiences from the same core logic: a plain chat page for a human to try instantly, and a real network-reachable MCP endpoint (streamable-http, not just local stdio) for another agent to call — the same rag_chain.py, two transports, one live deployment. Same approach as the sibling lead-scoring-databricks-pipeline project: build the real thing end to end, not a slide about it.

The application itself mirrors a realistic enterprise use case: "let staff ask natural-language questions against our own policy documents, with every answer traceable back to a real source page" — thematically relevant to government-adjacent roles specifically.

DATASET USED

8 real PDF documents (162 pages total), sourced directly from official .gov.au domains — no third-party mirrors:

FAH_Volume_1_Introduction_2025.pdf — Queensland Financial Accountability Handbook, Vol. 1
Overview-of-Queensland-Financial-Accountability-Framework-as-at-Jan-2020.pdf
ict-as-a-service_decision_framework-overview_v1_0_0.pdf
paf-policy-overview.pdf / paf-supporting-guidelines.pdf — Queensland Project Assurance Framework (incl. public-private partnerships)
n75057 [DE-81739] - 2026 Tax Time toolkit for small business_DIGITAL.pdf — ATO
n75127 [DE-73758] - Residency for tax purposes - factsheet_DIGITAL.pdf — ATO
tr2023-001.pdf — ATO Taxation Ruling 2023/1

The corpus is deliberately split across two clusters — Queensland state policy and Australian federal tax guidance — so retrieval has to genuinely discriminate between state vs. federal sources, and between a formal ruling and a plain-language summary of a related topic, rather than trivially matching on a single obvious keyword. An initial set of 11 documents (344 pages) was trimmed down to these 8 (162 pages) to keep the corpus large enough to be a real retrieval problem while staying fast to iterate on.

ARCHITECTURE & PIPELINE

flowchart LR
    subgraph Ingestion["Ingestion (run once, re-run on doc changes)"]
        A[Policy PDFs] --> B[Chunking\nLangChain RecursiveCharacterTextSplitter]
        B --> C[Embed\nAzure text-embedding-3-small]
        C --> D[(PostgreSQL + pgvector\nvia Docker)]
    end

    subgraph Query["Query time"]
        E[MCP Client\ne.g. Claude Desktop] --> F[MCP Server\nPython, containerized]
        F --> G[LangChain RAG chain]
        G --> D
        G --> H[Azure AI Foundry\ngpt-5-mini]
        H --> G --> F --> E
    end

    G -. traces .-> I[Langfuse\nobservability]

Ingestion: Each PDF is loaded page-by-page with LangChain's PyPDFLoader (preserving source filename + page number as metadata for later citations), then split with RecursiveCharacterTextSplitter at 1000 characters / 150 overlap — chosen over a smaller 500/50 split after comparing both directly on the real corpus (950 vs. 521 chunks); the larger chunk size suits these procedural, structurally dense policy documents better, giving the model more surrounding context per retrieved piece. Each of the 521 resulting chunks is embedded via Azure's text-embedding-3-small and written into a PostgreSQL database running in Docker, with the pgvector extension enabling vector similarity search directly inside the relational database rather than a separate dedicated vector store — a more production-credible pattern, since many real organizations add vector search to a Postgres instance they already run.

Query time: A question is embedded with the same model, PostgreSQL returns the top-4 most similar chunks by vector similarity, and those chunks are "stuffed" into a prompt that instructs gpt-5-mini to answer only from that retrieved context — or say "I don't know" rather than fall back on its own general knowledge. The answer is returned together with the exact source document and page number for every chunk it drew from, so every answer is independently verifiable.

Interface: The RAG chain is wrapped as an MCP server using the official Python MCP SDK, exposing two tools — query_policy_docs (the core Q&A capability) and list_indexed_documents (lets a client show what's actually covered before asking). The RAG logic and the MCP protocol layer are kept in separate files on purpose, so the underlying capability could be reused behind a different interface later without touching it. Locally, the server is containerized with Docker and runs over MCP's stdio transport — a literal subprocess pipe, spawned on demand per connection (docker compose run --rm -i), which only ever works for a client on the same machine. In production, the exact same MCPServer instance is also mounted via streamable-http transport instead, so a real MCP client anywhere on the internet can reach it too — two transports, zero duplicated RAG logic.

Observability: Every query is traced end-to-end with Langfuse — retrieval and generation nested under one connected trace per question, showing the exact prompt, retrieved chunks, latency, and token cost.

PRODUCTION DEPLOYMENT

flowchart LR
    subgraph Internet["Public internet"]
        U[Browser] -->|"/  /about  /demo  /api/ask"| S
        MC[Any MCP client] -->|"streamable-http → /mcp"| S
    end

    GH[GitHub — push to main] -->|auto-deploy| S

    subgraph Render["Render (Docker, free tier)"]
        S[FastAPI service — app.py\nsame rag_chain.py + mcp_server.py, two transports]
    end

    S --> N[(Neon\nserverless PostgreSQL + pgvector)]
    S --> AZ[Azure AI Foundry\ngpt-5-mini + text-embedding-3-small]
    S -. traces .-> LF[Langfuse]

The local Docker + stdio setup described above isn't replaced by any of this — it's still exactly how the server connects to Claude Desktop on the same machine. Production adds a second, publicly-reachable surface on top of the identical RAG logic:

Database: PostgreSQL moved from a local Docker container to Neon — serverless, managed, reachable from anywhere, with pgvector included on every plan and SSL required end-to-end.
One combined service (app.py): a FastAPI app built on the unmodified rag_chain.py/mcp_server.py, serving a public chat UI (/, /about, /demo) and the network-reachable MCP endpoint (/mcp) side by side.
Rate limiting: a public URL means anyone could otherwise run up real charges against the Azure OpenAI bill, so /api/ask is rate-limited (slowapi, configurable, default 10/minute) before this ever went live.
Hosting & CI/CD: deployed to Render as a Docker web service on the free tier, with auto-deploy on every push to main via Render's native GitHub integration — no separate CI/CD config to maintain. A small GitHub Actions workflow pings the health endpoint every 10 minutes to keep the free-tier instance warm, since the alternative — a public demo that's cold on first visit more often than not — undercuts the whole point of it being live.

MODEL USED

Both models are deployed via Azure AI Foundry (chosen deliberately over the classic "Azure OpenAI" resource type, since Foundry is named explicitly in several target job postings):

gpt-5-mini — generation, temperature=0 for faithful, literal answers grounded in retrieved text rather than creative ones.
text-embedding-3-small — embeddings, for both the document corpus and incoming questions.

This Foundry resource uses Azure's newer unified v1 API surface (.../openai/v1), so the integration uses LangChain's plain ChatOpenAI / OpenAIEmbeddings classes with base_url + api_key + model, rather than the Azure-specific AzureChatOpenAI / AzureOpenAIEmbeddings classes, which target the older dated-api-version endpoint scheme and 404 against this resource.

RESULTS & INSIGHTS

Evaluated manually with 5 real test questions run through the live system:

"What is the Financial Accountability Handbook designed to help agencies do?" → correct, near-verbatim grounded answer with accurate citations.
"What does the ICT-as-a-service decision framework help agencies decide?" → correct, captured the real nuance (deployment/service models, risk assessment).
"What is the capital of France?" (deliberately out-of-scope) → correctly answered "I don't know" instead of using the model's own general knowledge — the key anti-hallucination test.
"What is a public private partnership under Queensland's PPP policy?" → answered "I don't know." Verified independently by directly inspecting the source PDF's extracted text: the document genuinely never defines the term in plain language anywhere in its 23 pages — a real, defensible corpus gap correctly identified rather than a retrieval failure or a fabricated answer.
"What record-keeping obligations does the small business Tax Time toolkit mention?" → correct, specific detail, accurate citation.

5/5 sensible behavior — 3 correct grounded answers and 2 correct refusals (one genuinely out-of-scope, one a real corpus gap correctly recognized rather than hallucinated). Distinguishing "retrieval missed the right chunk" from "the corpus genuinely doesn't contain the answer" is a real RAG evaluation skill, and this round demonstrated both failure modes being handled correctly — not just an eyeballed pass/fail.

Verified against a real MCP client (Claude Desktop) end-to-end, both running locally via .venv and fully containerized via Docker, with every query traced in Langfuse. The streamable-http transport — the exact same mount now serving /mcp in production — was separately driven end-to-end by a real MCP client too (a full initialize → list_tools → call_tool round trip over HTTP), confirming the dual-transport architecture genuinely works rather than just starting without errors.

Verified live in production, not just locally: the same questions were re-run directly against the public deployment — no local anything involved. Asking "What does the ICT-as-a-service decision framework help agencies decide?" through https://policy-rag-mcp-server.onrender.com/demo returned the same correct, well-cited answer as the local evaluation above — proof the deployed service, Neon, and Azure AI Foundry are genuinely wired together in production, not just that a health check passes.

SETUP & USAGE

Try it now — no setup

The whole point of deploying this was that you shouldn't need any of the below just to see it work:

Chat with it: policy-rag-mcp-server.onrender.com/demo
Connect your own MCP client to it: the same server is reachable over the network at .../mcp via streamable-http. From an MCP client that supports remote servers, or via the mcp-remote stdio bridge for clients (like Claude Desktop) that don't dial a URL directly:
```
"policy-rag-remote": {
  "command": "npx",
  "args": ["-y", "mcp-remote", "https://policy-rag-mcp-server.onrender.com/mcp"]
}
```
(Render's free tier spins a service down after ~15 minutes idle; a GitHub Actions workflow pings it every 10 minutes so that mostly doesn't happen. If it ever does catch a cold one anyway, the UI says so — "Waking up server…" — rather than just sitting there looking stuck.)

Run it yourself

Requirements: Docker Desktop, Python 3.13, an Azure AI Foundry resource with gpt-5-mini and text-embedding-3-small deployed, a PostgreSQL target (local Docker or a free Neon project), and (optionally) a free Langfuse Cloud account for tracing.

Copy .env.example to .env and fill in your own Azure, Postgres, and (optionally) Langfuse credentials (the file documents both a local-Docker and a Neon setup).
Start the vector store — docker compose up -d postgres (skip this if pointing at Neon instead).
Ingest the documents (one-off, or whenever the document set changes):
```
pip install -r requirements-ingest.txt
python embed_and_store.py
```
Run the assistant directly from the CLI: python rag_chain.py
Or connect it as a local MCP tool over stdio: build the server image (docker compose build mcp-server), then add it to your MCP client's config (e.g. Claude Desktop's claude_desktop_config.json):
```
"policy-rag": {
  "command": "docker",
  "args": ["compose", "-f", "<path-to-repo>/docker-compose.yml", "run", "--rm", "-i", "mcp-server"]
}
```
Or run the same combined web+MCP service that's actually deployed: pip install -r requirements-web.txt && python app.py, then open http://localhost:8000.

CONCLUSIONS

The pipeline is fully working end to end, and it's live — not just something that runs when demoed locally. Real government PDFs are chunked, embedded, and stored in a real managed vector database; retrieval and generation are grounded and correctly refuse to answer beyond the corpus; the whole capability is exposed as a standards-compliant MCP tool over both stdio and streamable-http, each verified end-to-end against a real MCP client; the server is containerized, portable, and auto-deploys from main with zero manual steps; every query is traced with cost, latency, and full context for observability; and a public rate limit protects the underlying LLM bill now that anyone can reach it. Every layer named as a target skill gap — MCP, LangChain, a real vector database, Docker, Azure AI Foundry, LLMOps tracing, and shipping to production — is demonstrated with working, deployed, tested code rather than a tutorial-only implementation that stops at "runs on my machine."

AUTHOR INFORMATION

Varun Vikas Jaiswal 2026

KEYWORDS

RAG, Retrieval-Augmented Generation, LangChain, MCP, Model Context Protocol, streamable-http, Azure AI Foundry, Azure OpenAI, pgvector, PostgreSQL, Neon, serverless database, vector database, vector similarity search, Docker, Docker Compose, FastAPI, rate limiting, Render, production deployment, CI/CD, Langfuse, LLMOps, observability, Python, AI Engineer