# RepoSage — agentic code-Q&A over a codebase

Ask a repository questions in natural language — *"where is auth handled?"*,
*"what breaks if I change this function?"*, *"show every caller of X"* — and get
answers grounded in real source, with `file:line` citations.

RepoSage exists because generic RAG ("chat with your PDF") fails on code: fixed
token-window chunking cuts functions in half, and pure vector search ignores
the call graph that actually connects code. RepoSage treats **retrieval quality
as the engineering problem** and is exposed as an **MCP server**, so it plugs
straight into Claude Code / Cursor.

## Retrieval pipeline

```
repo ─► AST-aware chunking ─► hybrid index ─► fusion ─► cross-encoder ─► graph ─► cited
        (function/class/        (dense +       (dense +   rerank         expand    answer
         method units,           BM25)          BM25)     (top pool)     (1-hop)
         calls + imports)
```

- **AST-aware chunking** — one chunk per function / method / class (never a
  half-function), plus the `calls` and `imports` each definition makes.
  Zero-dependency (`ast` stdlib); tree-sitter multi-language is a v2 backend.
- **Hybrid retrieval** — dense embeddings **+** BM25 lexical, min-max
  normalized and fused with a tunable `alpha`. Beats either alone on code.
- **Cross-encoder rerank** — retrieve a wide pool via fusion, then reorder it
  with a `(query, code)` cross-encoder for precision (the standard "retrieve
  wide, rerank precise" second stage). Toggleable; see the eval for its
  measured, honest effect.
- **Call-graph expansion** — after search finds the best definition, walk one
  hop along the call graph to pull in what it calls / what calls it. This is
  the signal generic RAG cannot provide, and it's what makes impact questions
  answerable.
- **Graceful degradation** — if `sentence-transformers` isn't installed, dense
  retrieval falls back to a deterministic hashed embedding so the whole system
  still runs end-to-end on a fresh machine.

## MCP tools

| Tool | Purpose |
|------|---------|
| `index_repo(path)` | AST-ingest a repo and build/persist the hybrid index |
| `status()` | Index size + active embedding backend |
| `search_code(query, k, hybrid)` | Ranked definitions with per-signal scores |
| `get_context(query, k, expand_graph)` | Cited context bundle for answering |
| `impact_radius(chunk_id)` | Blast radius — who calls this definition |

## Quickstart

```bash
uv venv --python 3.12 .venv           # standard CPython (not free-threaded)
uv pip install -e .                    # core: mcp + rank-bm25 + numpy
uv pip install -e ".[embeddings]"      # optional: real semantic embeddings
```

Register with Claude Code (from this directory):

```bash
claude mcp add reposage -- .venv/Scripts/python.exe -m reposage.server
```

Then in Claude Code: *"index this repo with reposage, then ask where auth is handled."*

## Evaluation

The differentiator is `eval/` — labeled, auditable question sets with an
**ablation** that shows what each layer buys. Two corpora: **Flask**
(`pallets/flask`, 404 chunks, external — the fair test) and this repo's own
`src/` (53 chunks, dogfood). Every gold label and call edge is verified against
the actual source, not guessed.

```bash
# Flask (clone once, then run):
git clone --depth 1 https://github.com/pallets/flask .corpora/flask
python -m eval.run --dataset flask

python -m eval.run                    # dogfood on ./src
python -m eval.run --repo PATH        # any repo (with a matching dataset)
```

### Primary result — Flask (19 questions, 404 chunks, equal 8-result budget)

| Config | hit@8 | MRR |
|--------|-------|-----|
| vector-only | 0.79 | 0.563 |
| + BM25 fusion | 0.89 | 0.576 |
| **+ cross-encoder rerank** | **1.00** | **0.680** |
| + graph expansion | 0.84 | 0.568 |

By category (hit@8):

| Category | vector | + BM25 | + rerank | + graph |
|----------|--------|--------|----------|---------|
| semantic | 0.88 | 0.88 | **1.00** | 0.75 |
| keyword (exact identifiers) | 1.00 | 1.00 | 1.00 | 1.00 |
| impact | 0.40 | 0.80 | **1.00** | 0.80 |

**What the numbers actually say** (the honest read, not a rigged monotonic table):

- **The cross-encoder reranker is the big win — on a real corpus.** It lifts
  hit@8 from 0.89 → **1.00** and MRR +0.10, helping both semantic (0.88→1.00)
  and impact (0.80→1.00). Crucially, the *same* reranker was a **wash on the
  53-chunk dogfood corpus** (see below) — because a tiny corpus has too few
  distractors for reranking to matter. **The lesson: you cannot fairly evaluate
  a reranker on a toy corpus.** Running both corpora is what surfaced that.
- **BM25 fusion pays off on the queries it should** — it takes impact queries
  from 0.40 → 0.80 (exact symbol names) and lifts overall hit to 0.89. Dense
  handles paraphrase; BM25 handles literal identifiers; fusion gets both.
- **Graph expansion does not improve ranking — it slightly hurts it** (0.89 →
  0.84, semantic 0.88 → 0.75), and that's reported rather than hidden. Fusing
  graph neighbors into the ranked list displaces real hits. The call graph is a
  **context-enrichment / impact-analysis** feature, not a ranking-fusion layer —
  so it's evaluated on its own job instead:

**Call-graph quality (Flask)** — 8 hand-verified in-repo call edges:

| Metric | Value |
|--------|-------|
| caller-recall | **1.00** |
| avg callers/node (noise proxy) | 3.12 |

The AST-derived graph recovers every labeled caller edge — which is what makes
`impact_radius` trustworthy on real code.

### Secondary — dogfood on own `src/` (18 questions, 53 chunks)

| Config | hit@8 | MRR | note |
|--------|-------|-----|------|
| vector-only | 0.78 | 0.452 | |
| + BM25 fusion | 0.89 | 0.615 | |
| + cross-encoder rerank | 0.89 | 0.618 | **flat — corpus too small to test rerank** |
| + graph expansion | 0.94 | 0.622 | |

Kept deliberately: the contrast between "rerank is a wash" here and "rerank wins"
on Flask *is* the finding. Small-corpus metrics are also saturated, so treat
them as directional only.

_Raw metrics: `eval/results_flask.json`, `eval/results.json`._

## Status

v0.1 — pipeline + MCP server working end-to-end; real embeddings
(`all-MiniLM-L6-v2`) + cross-encoder rerank (`ms-marco-MiniLM-L-6-v2`) wired;
4-stage eval ablation on two corpora (Flask + dogfood) with call-graph metrics;
9 pipeline tests passing. On Flask the reranker reaches hit@8 = 1.00. Next: a
code-domain reranker, smarter graph-aware context assembly, and tree-sitter for
non-Python repos.