repopack

Give an LLM the context of a repository — without sending the whole project over the network. Two ways to do it, one tool, zero runtime dependencies (standard library only):

• pack — squeeze a repository (any language) into a single JSON file you can hand to a model, under a token ceiling you choose.

• serve — index your repositories locally and expose them over MCP, so a model pulls only the snippets a task needs, from all your projects, through one server.

Install

pip install -e .           # or just run: python -m repopack ...

Python 3.9+. No runtime dependencies — that is the point: it works on a bare interpreter.

Related MCP server: roam-code

Pack a repository

# Full pack of the current repository (any path works too)
repopack pack . -o context.json --gzip

# Incremental: only what changed since the last pack
repopack pack . --delta -o delta.json --gzip

# Restore on the other end
repopack unpack context.json -d ./restored

--budget is a ceiling on the whole pack, in estimated tokens — tree, stats and the per-file metadata count against it too, not just file content. The command prints how much of the budget went to content and how much to structure.

How the context is organized

What is left after the structural floor is spent in three layers, in priority order (README/docs → configs/manifests → entrypoints → the rest, boosted by git churn):

1. full — complete content, secrets redacted, up to ~72% of what is left

2. skeleton — imports, signatures and docstrings: the interface survives, the bodies do not. Measured on this codebase it keeps ~22% of the original characters (13–29% per file), so treat it as "a fifth of the cost", not a tenth

3. metadata — path, size, sha256, language (content: null plus a reason)

Besides the files, a pack carries tree (every path), stats, git.recent_commits, todos (harvested TODO/FIXME/HACK) and, in --delta mode, deleted_since_last_pack.

Byte-identical content is packed once. The other copies become pointers (content: null, reason: duplicate_of, plus the path that carries it) — their paths stay in the tree, and unpack resolves the pointer so every file is still restored. The highest ranked copy is the one that keeps the content. Measured on a repository with a SKILL.md copied into three places: duplicated content went from 0.89% to 0.00%, 8,982 characters freed for unique context.

In a large repository the structural floor alone can exceed a small budget — 4,212 files cost about 252k tokens in tree and metadata. When that happens the pack is generated without file content and the tool says so on stderr, naming the floor, instead of quietly returning 26× what you asked for. Trimming the tree to fit would hide which files exist, which is exactly what the pack promises to list.

Serve your repositories over MCP

One server, many repositories — local and remote — instead of one MCP per project.

repopack repo add .                                  # a local repository
repopack repo add https://github.com/user/project    # or a remote (shallow clone, cached)
repopack index                                       # ingest; the second run is a no-op
repopack serve                                       # MCP at http://127.0.0.1:7777/mcp

Running repopack serve inside a git repository registers and indexes it automatically (--no-auto-register opts out). Only repositories that were never indexed are ingested at startup, so restarting the server is not a full re-index.

A repository that is registered but never indexed answers nothing, so it is never silent about it: repo_stats counts it under unindexed, and a search that comes back empty names it and tells you to run repopack index.

Tools exposed

Protocol: revision 2026-07-28 (stateless), with a compatibility shim for clients from the initialize era (2025-03-26 … 2025-11-25).

The code graph

repopack index builds a graph of the repository alongside the text index, and search walks it (Personalized PageRank seeded by the lexical hits). Four kinds of edge, each carrying a different signal:

Whether the walk helps depends on the graph being dense: the ICLR'26 GraphRAG benchmark finds graph retrieval losing to plain RAG and names sparse, fragmented graphs as the cause. So repo_stats reports the diagnosis, not just a total — measured here:

(A snapshot: these move as the repositories do. Run repo_stats for yours.)

Both are an order of magnitude denser than the sparse profile the paper blames, which is the reason the walk is switched on by default here. Without the [ast] extra the call edges disappear — the densest kind — so check graph.ast in repo_stats before reading a thin graph as "this repository has no structure".

Automatic refresh

Remotes are refreshed in the background (--refresh-interval, default 900s, jittered; --no-auto-refresh opts out). When a fetch fails the server keeps serving, and the failure shows up in list_repos and repo_stats — not only in the log. Stale context served as if it were fresh is the failure mode that visibility exists to prevent.

Security

• Binds to 127.0.0.1 by default. Leaving loopback (--host 0.0.0.0) requires a bearer token: if you do not pass one, the server generates it and prints it once. There is no "network without authentication" mode — the index holds the source code of every repository in the workspace.

• The Origin header is validated on every request (403 when invalid), against DNS rebinding.

• Redaction happens at ingest, not on the way out: what is secret never enters the database, so there is no path for it to leave. .env and its variants stay out by default (--include-env opts in, and still redacts).

• get_file never serves anything outside the repository root — the same guard unpack uses.

• Credentials for private repositories come from your environment (ssh-agent, git helper); repopack stores no token and never prompts for a password (GIT_TERMINAL_PROMPT=0).

• Secrets are redacted by pattern (API keys, GitHub/Slack tokens, JWT, private keys, .env variables, URLs with credentials) plus a Shannon-entropy sweep.

• unpack audits each restored file against the sha256 the pack carries, and warns on mismatch. The check is conclusive only for content that is intact by definition — complete, not truncated, not redacted — because a redacted file diverges from its hash by construction.

Redaction is a net, not a guarantee: it covers variable names containing SECRET/TOKEN/KEY/PASSWORD and known key formats, so a line like MY_PLAIN=value123 gets through. That is why the default for .env is to not pack the content at all, rather than trusting the filter.

What this is not

• unpack is not a backup tool. It restores only content that is intact, refuses degraded content by default (naming each file and why), and never overwrites an existing file unless you pass --force. A pack is a context artifact, not a copy of your repository.

• skeleton mode is lossy by design. It keeps imports, signatures and docstrings and drops function bodies. Restoring a skeleton over a real file destroys the original and gives back something that will not even parse — which is why it takes --force and says so.

• Skeleton extraction is regex, not a real AST. It is good enough to keep an interface readable and wrong often enough that you should not treat it as a parser. The [ast] extra does bring real tree-sitter parsing, but it feeds the code graph, not pack --skeleton: installing it makes search better and leaves skeleton output byte-identical.

Honest limitations

• Search is lexical (BM25) fused with a code-graph walk, and still has no embeddings. Measured on this repository with repopack eval (18 questions, --no-graph reproduces the first row):

  ranking	recall@5	MRR	identifier	conceptual
  BM25 only	83.3%	0.596	100% / 0.875	70% / 0.373
  BM25 + graph (default)	94.4%	0.625	100% / 0.823	90% / 0.467

  Read those absolute numbers with the ruler's granularity in mind: 18 questions means one question is worth 5.6 points, and the corpus being measured is this repository, which changes every commit. Measured: adding two files whose docstrings talk about budgets and ceilings moved recall between 88.9% and 94.4% with no ranking change at all. What the gate enforces is therefore a floor one question below the measurement, and the comparison between the two rows — both measured on the same corpus in the same run, which is the part that does not drift.

  The graph costs about +212 ms per query on a 4,212-file repository (68 ms → 280 ms; on a small one it is 4.5 ms → 7.7 ms), and it costs a little position on identifier queries — which is why the lexical top hit is pinned. What it did not fix: a test file that mentions the concept still outranks the implementation for some conceptual questions. "Where is the budget enforced?" now returns packer.py in the top-3, but behind test_rag_search.py. A dense ranker is still absent.

• Diversification (MMR) is implemented and deliberately switched off, because it did not win. Measured across λ from 1.0 down to 0.6, the one apparent gain does not survive a change of k (at k=3 a different λ wins; at k=10 every λ ties at 100% recall while MRR falls monotonically as λ drops) — a peak that moves is one question oscillating, not an effect. The premise did not hold either: across a 3-repository, 14,606-chunk workspace the average Jaccard similarity between returned snippets is 0.04–0.08, and the only identical pairs are 3-token fixtures. Switching it on costs 4 ms → 97 ms per query. select.LAMBDA re-opens the decision, and a test refuses the change unless the ruler backs it.

• Cross-repo search gets noisy with many repositories and a generic query. What holds today is the token ceiling and per-repo attribution.

• IDF is measured across the whole workspace. A term that is common in one project and rare in the others weighs differently than it would in an isolated search.

• The local gate (scripts/check.sh) runs on the Python versions installed on your machine, which is usually fewer than the matrix. The full 3.9–3.14 matrix is exercised by CI on every push; the three lists (classifiers, CI matrix, local gate) are cross-checked by the test suite, so they cannot drift apart in silence.

The index — and the --delta state — live in ~/.cache/repopack/ (honors XDG_CACHE_HOME; REPOPACK_HOME overrides), never inside the repositories you scan. A --delta state left in a repository root by an older version is still read, so your delta survives the upgrade; it is not deleted, because removing a file from someone's repository is more invasive than having created it.

Configuration

Create a .repopackrc (JSON) at the repository root — see .repopackrc.example. Precedence: defaults < .repopackrc < CLI flags.

Roadmap

• Ranking priors by file kind, so a test file stops taking the first slot from the implementation it tests — the one conceptual defect the graph did not fix

• Diversification (MMR) and near-duplicate suppression across repositories

• Optional dense ranker behind a [rag] extra, for conceptual queries

• Task-conditioned packing (--task)

Contributing

See CONTRIBUTING.md. The one rule that is not negotiable: zero runtime dependencies.

License

MIT — see LICENSE.