Agentic RAG Assistant with FastAPI + MCP

An internal-docs assistant that combines four things recruiters are currently screening for:

• Agentic AI — a tool-use loop where the LLM decides whether to search the knowledge base, run a calculation, or answer directly.

• FastAPI — a clean REST backend (/ingest, /query, /agent/chat) with auto-generated Swagger docs.

• RAG — documents are chunked, indexed, and retrieved by semantic relevance before the LLM answers.

• MCP (Model Context Protocol) — the same tools (RAG search + business logic) are exposed as an MCP server so any MCP-compatible client (Claude Desktop, Claude Code, etc.) can use them directly, not just this API.

Architecture

                     ┌────────────────────┐
                     │   FastAPI Service   │
                     │  (app/main.py)      │
                     └─────────┬───────────┘
                               │
                     ┌─────────▼───────────┐
                     │   Agent Loop         │◄──── Groq API (tool use)
                     │  (app/agent.py)      │
                     └─────────┬───────────┘
                               │ calls
                 ┌─────────────┼──────────────┐
                 ▼                             ▼
        ┌────────────────┐           ┌──────────────────┐
        │  RAG Engine      │           │  Business Tools    │
        │  (app/rag.py)    │           │  (app/tools.py)     │
        └────────────────┘           └──────────────────┘
                 ▲                             ▲
                 └─────────────┬───────────────┘
                                │  same tools, exposed via
                     ┌──────────▼───────────┐
                     │   MCP Server           │
                     │  (app/mcp_server.py)   │
                     └────────────────────────┘

Related MCP server: mcp-docpilot-server

Setup

python -m venv venv
source venv/bin/activate       # Windows: venv\Scripts\activate
pip install -r requirements.txt
cp .env.example .env           # add your GROQ_API_KEY

Run the API

uvicorn app.main:app --reload --port 8000

Open http://localhost:8000/docs for interactive Swagger docs.

Try it

# Pure retrieval, no LLM call
curl -X POST http://localhost:8000/query \
  -H "Content-Type: application/json" \
  -d '{"query": "how many days of leave do I get?"}'

# Full agentic chat — LLM decides which tool(s) to call
curl -X POST http://localhost:8000/agent/chat \
  -H "Content-Type: application/json" \
  -d '{"message": "I joined in March and have taken 4 days off. How much leave do I have left, and what is the WFH policy?"}'

The second call demonstrates multi-tool reasoning: the model calls rag_search for the WFH policy AND calculate_leave_balance for the math, in one conversation.

Run as an MCP server

python -m app.mcp_server

Point any MCP host at this script over stdio (e.g. add it to Claude Desktop's claude_desktop_config.json as a custom MCP server) and it will expose rag_search, get_current_datetime, and calculate_leave_balance as callable tools.

What to say about this project in an interview

• Why TF-IDF instead of embeddings by default: keeps the demo runnable with zero API keys and zero external downloads; the VectorStore class is written so swapping in FAISS/Chroma + real embeddings is a drop-in change, not a rewrite.

• Why the tools live in one file (tools.py) and get exposed twice (agent.py and mcp_server.py): single source of truth, no logic duplication between the HTTP path and the MCP path.

• The agent loop is a manual implementation of the tool-use pattern (not a black-box framework), so you can explain every step: model requests a tool → server executes it → result is fed back → model continues or answers.

Possible extensions (good "what would you improve" answers)

• Swap TF-IDF for real embeddings (OpenAI/Voyage/local sentence-transformers) + a persistent vector DB.

• Add conversation memory across turns (currently each /agent/chat call is stateless).

• Add streaming responses via Server-Sent Events.

• Add authentication (API key or JWT) before deploying publicly.

• Containerize with Docker + docker-compose for one-command startup.