Skip to main content
Glama
README.md
# Agentic RAG Assistant with FastAPI + MCP

An internal-docs assistant that combines four things recruiters are currently screening for:

- **Agentic AI** — a tool-use loop where the LLM decides whether to search the knowledge base, run a calculation, or answer directly.
- **FastAPI** — a clean REST backend (`/ingest`, `/query`, `/agent/chat`) with auto-generated Swagger docs.
- **RAG** — documents are chunked, indexed, and retrieved by semantic relevance before the LLM answers.
- **MCP (Model Context Protocol)** — the same tools (RAG search + business logic) are exposed as an MCP server so any MCP-compatible client (Claude Desktop, Claude Code, etc.) can use them directly, not just this API.

## Architecture

```
                     ┌────────────────────┐
                     │   FastAPI Service   │
                     │  (app/main.py)      │
                     └─────────┬───────────┘
                               │
                     ┌─────────▼───────────┐
                     │   Agent Loop         │◄──── Groq API (tool use)
                     │  (app/agent.py)      │
                     └─────────┬───────────┘
                               │ calls
                 ┌─────────────┼──────────────┐
                 ▼                             ▼
        ┌────────────────┐           ┌──────────────────┐
        │  RAG Engine      │           │  Business Tools    │
        │  (app/rag.py)    │           │  (app/tools.py)     │
        └────────────────┘           └──────────────────┘
                 ▲                             ▲
                 └─────────────┬───────────────┘
                                │  same tools, exposed via
                     ┌──────────▼───────────┐
                     │   MCP Server           │
                     │  (app/mcp_server.py)   │
                     └────────────────────────┘
```

## Setup

```bash
python -m venv venv
source venv/bin/activate       # Windows: venv\Scripts\activate
pip install -r requirements.txt
cp .env.example .env           # add your GROQ_API_KEY
```

## Run the API

```bash
uvicorn app.main:app --reload --port 8000
```

Open `http://localhost:8000/docs` for interactive Swagger docs.

### Try it

```bash
# Pure retrieval, no LLM call
curl -X POST http://localhost:8000/query \
  -H "Content-Type: application/json" \
  -d '{"query": "how many days of leave do I get?"}'

# Full agentic chat — LLM decides which tool(s) to call
curl -X POST http://localhost:8000/agent/chat \
  -H "Content-Type: application/json" \
  -d '{"message": "I joined in March and have taken 4 days off. How much leave do I have left, and what is the WFH policy?"}'
```

The second call demonstrates multi-tool reasoning: the model calls `rag_search` for the WFH policy AND `calculate_leave_balance` for the math, in one conversation.

## Run as an MCP server

```bash
python -m app.mcp_server
```

Point any MCP host at this script over stdio (e.g. add it to Claude Desktop's `claude_desktop_config.json` as a custom MCP server) and it will expose `rag_search`, `get_current_datetime`, and `calculate_leave_balance` as callable tools.

## What to say about this project in an interview

- Why TF-IDF instead of embeddings by default: keeps the demo runnable with zero API keys and zero external downloads; the `VectorStore` class is written so swapping in FAISS/Chroma + real embeddings is a drop-in change, not a rewrite.
- Why the tools live in one file (`tools.py`) and get exposed twice (agent.py and mcp_server.py): single source of truth, no logic duplication between the HTTP path and the MCP path.
- The agent loop is a manual implementation of the tool-use pattern (not a black-box framework), so you can explain every step: model requests a tool → server executes it → result is fed back → model continues or answers.

## Possible extensions (good "what would you improve" answers)
- Swap TF-IDF for real embeddings (OpenAI/Voyage/local sentence-transformers) + a persistent vector DB.
- Add conversation memory across turns (currently each `/agent/chat` call is stateless).
- Add streaming responses via Server-Sent Events.
- Add authentication (API key or JWT) before deploying publicly.
- Containerize with Docker + docker-compose for one-command startup.

Maintenance

ActivityMaintained
ResponsivenessNo issues