Codebase Copilot

An AI coding assistant built from scratch to learn and demonstrate four core building blocks of modern AI systems: RAG, Agents, MCP, and Multi-Agent orchestration — applied to a real codebase (tqdm).

Instead of using a framework that hides how these systems work, every layer here is built manually in plain Python, so the internals are fully understood, debugged, and explainable.

What it does

Point it at a codebase (tqdm, in this case) and ask questions like "what is the tqdm class and how does its update method work?" — it retrieves the relevant code, reasons about what it needs, and generates an accurate, grounded answer.

Related MCP server: @lex-tools/codebase-context-dumper

Architecture — four progressive stages

1. RAG (index.py, query.py)

• Parses the codebase using Python's ast module, splitting code into meaningful chunks (whole functions/classes, not arbitrary word-count slices)

• Embeds each chunk locally using sentence-transformers

• Stores embeddings in a local ChromaDB vector database

• Retrieves the most relevant chunks for a question and generates an answer via the Groq API (openai/gpt-oss-120b)

Real bug fixed: naive word-count chunking caused the main tqdm class (spread across a large file) to never surface in search results, since no single chunk represented it well. Fixed by switching to AST-based chunking — splitting by function/class boundaries instead, with large classes further split by individual method.

2. Agent (agent.py)

• Gives the model two tools: search_code and read_file

• The model decides autonomously which tool to use, when, and whether it needs another step before answering — instead of a fixed search-then-answer sequence

• Includes safeguards for real agent failure modes: malformed tool arguments, repeated/looping tool calls, and forced convergence to a final answer within a step budget

3. MCP Server (server.py, test_mcp_client.py)

• Wraps search_code and read_file as a standard Model Context Protocol server, making them accessible to any MCP-compatible client — not just this project's own script

• Verified with a custom MCP client that connects over stdio, lists available tools, and calls them successfully against the live database

4. Multi-Agent System (multi_agent.py)

Three specialized agents coordinated by an orchestrator:

• Retriever — searches the codebase (with a targeted secondary search for specific method names)

• Explainer — writes an answer from retrieved context

• Reviewer — checks the answer for accuracy and completeness against the actual context, and can send it back to the Explainer with specific feedback for revision (up to 2 rounds)

Real bug fixed: the Reviewer initially approved an answer that incorrectly claimed information was "missing," when it was actually present in the codebase — the Retriever just hadn't surfaced it. This exposed a real multi-agent design flaw: a Reviewer can only judge consistency with the context it's given, not whether the Retriever gathered the right context in the first place. Fixed by improving retrieval coverage and adding an explicit check in the Reviewer's prompt for this failure pattern.

Tech stack

• Python — core language

• ChromaDB — local vector database

• sentence-transformers (all-MiniLM-L6-v2) — local embeddings

• Groq API (openai/gpt-oss-120b) — LLM inference

• MCP (Model Context Protocol) — standardized tool exposure

Setup

1. Clone this repo and create a virtual environment:

   python -m venv venv
   source venv/bin/activate   # Windows: venv\Scripts\activate
   pip install -r requirements.txt

2. Clone tqdm's source into a repo folder (or point REPO_PATH in index.py at any other small Python codebase):

   git clone https://github.com/tqdm/tqdm.git repo

3. Add a .env file with your Groq API key:

   GROQ_API_KEY=your_key_here

4. Build the index:

   python index.py

5. Try any of the four stages:

   python query.py          # Week 1: plain RAG
   python agent.py          # Week 2: agent with tools
   python server.py         # Week 3: MCP server (run test_mcp_client.py in a separate terminal to test it)
   python multi_agent.py    # Week 4: multi-agent system

What I learned

Building this project surfaced real engineering problems that don't show up in tutorials:

• Retrieval quality is a hard ceiling on generation quality, no matter how good the LLM is

• Chunking strategy matters more than embedding model choice for code specifically

• Agents need explicit guardrails against looping and malformed tool calls

• A "reviewer" agent is only as good as the context it's reviewing against — multi-agent systems can still fail silently if earlier stages don't surface the right information

Future directions

• Smarter multi-agent flow where the Reviewer's feedback can trigger the Retriever again, not just the Explainer

• Swap the local embedding model for a larger hosted one (e.g. Voyage AI) to compare retrieval quality

• Extend to support editing code, not just answering questions about it