tbguard
# tbguard
A context and token budget guard for coding agents. It runs as a CLI or as an MCP server over stdio. Before files reach the model, it:
1. **drops noise**: lockfiles, build output, `node_modules`, logs, minified bundles, binaries, generated files, and anything over 1 MB;
2. **cleans what's left**: removes trailing whitespace, collapses runs of blank lines, and folds repeated lines;
3. **enforces a hard cap**: 4k / 8k / 16k tokens, or any integer. The packed output is never larger than the budget;
4. **logs the savings**: every call appends raw vs. final token counts to a JSONL ledger.
It has zero runtime dependencies (Python 3.10+, standard library only). `tiktoken` is used if it is installed.
## Quick start
```bash
pip install -e . # or just run: python -m tbguard ...
tbguard pack src/ docs/ -b small --report > context.txt
tbguard stats
```
As an MCP server (Claude Code, Claude Desktop, or any MCP client):
```json
{
"mcpServers": {
"tbguard": { "command": "python", "args": ["-m", "tbguard", "serve"] }
}
}
```
Tools exposed:
| tool | args | returns |
|---|---|---|
| `guard_context` | `paths[]`, `budget?`, `ignore[]?` | savings summary line + packed context |
| `inspect_file` | `path` | token count, keep/drop and the reason |
| `savings_summary` | none | cumulative totals from the ledger |
## Benchmarks
These are real runs on this machine (Windows 11, Python 3.14, heuristic tokenizer). Times are wall clock and include interpreter start-up.
| input | raw tokens | packed | saved | time |
|---|---:|---:|---:|---:|
| `examples/sample_repo` (FastAPI service + lockfile, log, minified JS, node_modules, dist) | 80,358 | 1,299 | 98.4% | 0.23 s |
| a 740k-token mixed folder (3D assets, scripts, this repo) at `large` | 740,214 | 15,969 | 97.8% | 1.1 s |
The first row is mostly filtering. The second row is mostly the budget doing its job. Run `python examples/make_sample.py` to regenerate the sample.
## Architecture
```
paths ──► collect_files ──► classify_path ──► classify_content ──► strip_noise_lines
(prunes noise (dir names, (NUL bytes, (whitespace,
dirs, >1 MB) globs) @generated, minified) repeats)
│
▼
pack (smallest-first fair share)
│
truncate_to (keep head ⅔ + tail ⅓)
│
packed text + GuardReport ──► ledger.jsonl
```
- **Fair-share packing.** Files are packed smallest first. Each one gets `remaining / files_left` tokens, so a single 50k-token file can't take the whole budget from ten small ones. The output keeps your original order.
- **Head and tail truncation.** When a file must be cut, imports, signatures and the end of the file survive, and the middle is replaced with `... [N lines omitted by tbguard] ...`. The cut point is found by binary search over per-line token prefix sums (O(n log n)), followed by an exact recount.
- **Hard guarantee.** `count_tokens(packed) <= budget`, and the tests check this at five budget sizes.
- **MCP without an SDK.** The server is about 150 lines of newline-delimited JSON-RPC 2.0 (`initialize`, `ping`, `tools/list`, `tools/call`). Tool failures come back in-band as `isError: true`, and bad arguments return `-32602`, as the spec requires.
## Trade-offs and limits
- **Token counts are estimates unless `tiktoken` is installed.** The heuristic averages a characters/4 estimate with a per-word estimate. It is close for code and prose, but it is not exact for any particular model. Use `pip install .[exact]` to get cl100k counts. Different models tokenize differently, so leave some headroom.
- **Priority is input order, not relevance.** There is no embedding or ranking step, by design: it would add dependencies and latency. Put the most important paths first.
- **Truncation works on lines, not syntax.** It can cut through the middle of a function body. Parsing with an AST would fix that, but only for one language at a time.
- **Filtering is opinionated.** For example, `bin/`, `vendor/` and `*.jsonl` are dropped. Override with `--ignore` (to add globs), or edit `NOISE_DIRS` / `NOISE_GLOBS`.
- **The ledger is local only.** It lives at `.tbguard/ledger.jsonl` (override with `TBGUARD_LEDGER`). Nothing is sent anywhere.
## Tests
```bash
python -m unittest discover -s tests -t .
```
23 tests cover filters, the token estimator, the budget guarantee, truncation, starvation, the ledger, MCP protocol handling and the stdio loop.
## License
MIT
TDQS
Scored across 3 tools
Each tool has a clear role: guard_context packs files under a budget, inspect_file previews a single file's token count and drop decision, and savings_summary reports aggregate savings. inspect_file overlaps slightly with guard_context since it previews the guard's behavior, but the descriptions make the boundary clear enough.
All names use snake_case and are readable. Two follow a verb_noun pattern (guard_context, inspect_file) while savings_summary is noun_noun, a minor deviation but not confusing.
Three tools is well-scoped for a narrow token-budget/context-packing utility: inspect, pack, and report savings. No tool feels redundant or missing from the count perspective.
The core workflow is covered: inspect a file, pack a context under budget, and view cumulative savings. Minor gaps exist—no way to configure the token budget or drop patterns, and no per-call savings detail or reset—but agents can work around these.