codebase-mcp

MCP server that indexes one repo's docs, closed issues, and source code across three differential chunking strategies, and exposes hybrid + reranked retrieval as MCP tools.

The chunking comparison is the point of this project — never cut it, even under time pressure. Everything else has a defined fallback below.

Build order

• Scaffold: project layout, Docker Compose (Qdrant + Ollama), CI skeleton

• Ingestion: pull docs / closed issues / source for the target repo → data/raw/

• Chunking (the core contribution) → data/chunks/

  • docs: heading-boundary splitting

  • issues: thread-level (not comment-level)

  • code: AST boundaries via tree-sitter — fallback if short on time: file-level chunks

  • baseline: naive fixed-size, for the comparison

• Indexing: BGE-small embeddings → Qdrant; BM25 index in parallel

• Eval ground truth: mine closed issues with maintainer answers → labelled pairs (do this before tuning retrieval, not after)

• Retrieval: hybrid BM25 + dense, RRF fusion, BGE-reranker-base cross-encoder rerank

• Query router: LLM classifier (docs / issues / code / mixed) → per-source weights - [ ] stretch: distill into a small trained model for millisecond routing — fallback if short on time: keep the LLM classifier

• MCP server (MCP Python SDK) + FastAPI/Swagger front door

• CI/CD: GitHub Actions → GHCR image

Related MCP server: github-rag-mcp

Evaluation

• Recall@k and MRR, reported per source (docs / issues / code)

• All three chunking strategies vs. the naive fixed-size baseline, head to head

• Routing accuracy, reported separately from end-to-end answer quality

• Ablation table: baseline → +hybrid → +rerank → +routing

Stack

Python 3.11 · LangChain/LangGraph · tree-sitter · Qdrant · sentence-transformers (BGE-small-en-v1.5) · rank_bm25 · BGE-reranker-base · Ollama (dev) / Groq (hosted demo) · MCP Python SDK · FastAPI + Swagger · Docker Compose · GitHub Actions → GHCR

Getting started

cp .env.example .env        # fill in TARGET_REPO, GITHUB_TOKEN, GROQ_API_KEY
docker compose --profile dev up -d qdrant ollama
pip install -e ".[dev]"

codebase-mcp ingest --repo owner/name
codebase-mcp chunk --strategy all
codebase-mcp index
codebase-mcp eval-mine
codebase-mcp eval-run --ablation
codebase-mcp serve