# RAG Blob MCP

A Model Context Protocol (MCP) server that gives a RAG agent search access to a document library stored in Azure Blob Storage, plus a Streamlit app for uploading documents and chatting with them.

- **MCP server** — FastMCP over Streamable HTTP, owns all Azure Blob Storage access, exposes 4 tools and 3 prompt templates, holds an in-memory vector index (OpenAI embeddings) rebuilt from Blob Storage on every startup.
- **Streamlit app** — a Library tab (upload / list / delete documents) and a Chat tab (ask questions, answered by a LangGraph ReAct agent that calls the server's search tool).

## Quick start

### 1. Prerequisites

- Python 3.11+ (developed against 3.13)
- An Azure Storage Account with a Blob container (see [Azure setup](#azure-setup) below if you don't have one)
- An OpenAI API key

### 2. Install

From the repo root:

```bash
python -m venv .venv
.venv\Scripts\activate          # Windows
pip install -r requirements.txt
```

### 3. Configure

Copy `.env.example` (at the repo root) to `.env` and fill in your values:

```
OPENAI_API_KEY=sk-...
AZURE_STORAGE_CONNECTION_STRING=DefaultEndpointsProtocol=https;AccountName=...;AccountKey=...;EndpointSuffix=core.windows.net
AZURE_STORAGE_CONTAINER_NAME=rag-documents
```

Everything else (`OPENAI_MODEL`, `OPENAI_EMBEDDING_MODEL`, `MCP_SERVER_HOST/PORT/URL`) has a working default — see [Configuration](#configuration).

### 4. Run

Two processes, two terminals, both from the repo root:

```bash
# Terminal 1 — MCP server
python server/mcp_server.py
```
```bash
# Terminal 2 — Streamlit app
streamlit run app/streamlit_app.py
```

Open http://localhost:8501. Upload a PDF/DOCX/TXT/MD file in the **Library** tab, then ask a question about it in the **Chat** tab.

## Azure setup

If you don't already have a Storage Account:

1. [Azure Portal](https://portal.azure.com) → **Create a resource → Storage account**. Standard performance, LRS redundancy is fine for personal use.
2. In the new account: **Data storage → Containers → + Container**, name it (e.g. `rag-documents`), access level Private.
3. **Security + networking → Access keys → Show keys**, copy the connection string.
4. Paste it into `.env` as `AZURE_STORAGE_CONNECTION_STRING`, and set `AZURE_STORAGE_CONTAINER_NAME` to the container name you chose.

## Project layout

```
.
├── server/
│   ├── mcp_server.py      # FastMCP server: tools, prompts, index rebuild, __main__ entrypoint
│   ├── blob_store.py      # Azure Blob Storage wrapper
│   ├── indexing.py        # text extraction (pdf/docx/txt/md) + chunking
│   ├── vector_index.py    # in-memory vector store wrapper
│   └── test_*.py          # automated tests (pytest)
├── agent/
│   └── rag_agent.py       # LangGraph ReAct agent used by the Chat tab
├── app/
│   ├── mcp_client.py      # direct MCP tool-call helpers used by the Library tab
│   └── streamlit_app.py   # the UI
├── scripts/
│   └── smoke_test_server.py  # manual end-to-end smoke test against a running server
├── docs_build/            # scripts that generate PROJECT.docx / PROJECT.pdf
└── pytest.ini
```

## Configuration

All variables live in the repo-root `.env`:

| Variable | Default | Purpose |
|---|---|---|
| `OPENAI_API_KEY` | — (required) | Chat model + embeddings |
| `OPENAI_MODEL` | `gpt-4.1` | Chat model for the RAG agent |
| `OPENAI_EMBEDDING_MODEL` | `text-embedding-3-small` | Embedding model for the vector index |
| `AZURE_STORAGE_CONNECTION_STRING` | — (required) | Blob Storage access |
| `AZURE_STORAGE_CONTAINER_NAME` | — (required) | Container the documents live in |
| `MCP_SERVER_HOST` | `127.0.0.1` | Interface the MCP server binds to |
| `MCP_SERVER_PORT` | `8000` | Port the MCP server binds to |
| `MCP_SERVER_URL` | `http://127.0.0.1:8000/mcp` | URL the Streamlit app/agent connect to — **update this too if you change the port** |

## Usage

**Library tab** — upload PDF/DOCX/TXT/MD files (200MB limit, set by Streamlit). Each upload is chunked, embedded, and added to the search index; the document list shows `indexed` / `failed` / `pending` status with chunk counts. Delete removes a document from both Blob Storage and the index.

**Chat tab** — ask a question in plain English. The agent decides when to call the search tool, retrieves relevant chunks, and answers citing the source document by filename. With an empty or irrelevant library it says so rather than guessing.

## Testing

```bash
pytest server/ -v
```

29 tests cover text extraction/chunking, the vector index, the Azure Blob Storage wrapper (mocked), and all 4 MCP tools + prompts (via `fastmcp.Client` in-process, using fakes — no real Azure/OpenAI calls). There's no automated coverage for the Streamlit UI or the live agent/server wiring; see `scripts/smoke_test_server.py` for a manual smoke test against a real running server.

## Known limitations

- **In-memory index, no persistence.** Every server restart re-downloads, re-extracts, and re-embeds every document in the container. Fine for a small personal library; costs real OpenAI API calls and startup time as the library grows.
- **Single-user, local-only.** No auth, no concurrent-write safety, not deployed anywhere.
- Both processes (MCP server, Streamlit app) must be running for the app to work — see the [full project doc](PROJECT.md#known-bugs--limitations) for more.

For the full picture — architecture, design decisions, known bugs, and next steps — see [`PROJECT.md`](PROJECT.md) (also available as [`PROJECT.docx`](PROJECT.docx) / [`PROJECT.pdf`](PROJECT.pdf)).