tokensave

Why tokensave?

AI coding agents waste tokens exploring codebases. Every grep, glob, and file read costs money. On complex tasks, agents spawn multiple Explore sub-agents that scan hundreds of files just to build context.

tokensave gives agents a pre-indexed semantic knowledge graph. Instead of scanning files, the agent queries the graph and gets instant, structured answers -- the right symbols, their relationships, and source code, in one call.

How It Works

┌──────────────────────────────────────────────────────────────┐
│  AI Coding Agent (Claude Code, Codex, Gemini, Cursor, ...)   │
│                                                              │
│  "Implement user authentication"                             │
│        │                                                     │
│        ▼                                                     │
│  ┌─────────────────┐       ┌─────────────────┐               │
│  │  Sub-agent      │ ───── │  Sub-agent      │               │
│  └────────┬────────┘       └─────────┬───────┘               │
└───────────┼──────────────────────────┼───────────────────────┘
            │                          │
            ▼                          ▼
┌──────────────────────────────────────────────────────────────┐
│  tokensave MCP Server                                        │
│  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐           │
│  │   Search    │  │   Callers   │  │   Context   │           │
│  │   "auth"    │  │  "login()"  │  │   for task  │           │
│  └──────┬──────┘  └──────┬──────┘  └──────┬──────┘           │
│         └────────────────┼────────────────┘                  │
│                          ▼                                   │
│              ┌───────────────────────┐                       │
│              │   libSQL Graph DB     │                       │
│              │   • Instant lookups   │                       │
│              │   • FTS5 search       │                       │
│              └───────────────────────┘                       │
└──────────────────────────────────────────────────────────────┘

Without tokensave: Agents use grep, glob, and Read to scan files -- many API calls, high token usage.

With tokensave: Agents query the graph via MCP tools -- instant results, local processing, fewer tokens.

Key Features

Quick Start

1. Install

Homebrew (macOS):

brew install aovestdipaperino/tap/tokensave

Scoop (Windows):

scoop bucket add tokensave https://github.com/aovestdipaperino/scoop-tokensave
scoop install tokensave

Cargo (any platform):

cargo install tokensave                          # full (34 languages, default)
cargo install tokensave --features medium        # medium (20 languages)
cargo install tokensave --no-default-features    # lite (11 languages, smallest binary)

Prebuilt binaries (Linux, Windows, macOS):

Download from the latest release and place the binary in your PATH.

2. Configure your agent

tokensave install                         # auto-detects installed agents
tokensave install --agent claude          # Claude Code
tokensave install --agent codex           # OpenAI Codex CLI
tokensave install --agent gemini          # Gemini CLI
tokensave install --agent opencode        # OpenCode
tokensave install --agent cursor          # Cursor
tokensave install --agent copilot         # GitHub Copilot
tokensave install --agent cline           # Cline
tokensave install --agent roo-code        # Roo Code
tokensave install --agent zed             # Zed
tokensave install --agent antigravity     # Google Antigravity (formerly Windsurf)
tokensave install --agent kilo            # Kilo CLI

Each agent gets its MCP server registered in the native config format. Claude Code additionally gets a PreToolUse hook (blocks wasteful Explore agents), a UserPromptSubmit hook, a Stop hook, prompt rules in CLAUDE.md, and auto-allowed tool permissions.

All changes are idempotent -- safe to run again after upgrading. After agent setup, you'll be offered a global git post-commit hook and the background daemon service.

3. Index your project

cd /path/to/your/project
tokensave init

This creates a .tokensave/ directory with the knowledge graph database. Initialization and sync are separate commands: init is a one-time opt-in per project, while sync only updates projects that were already initialized. This prevents the global git post-commit hook from silently creating databases in repos you never intended to index. After init, use tokensave sync to incrementally update -- only changed files are re-indexed.

MCP server

{
  "mcpServers": {
    "tokensave": {
      "command": "/path/to/tokensave",
      "args": ["serve"]
    }
  }
}

PreToolUse hook

The hook runs tokensave hook-pre-tool-use -- a native Rust command (no bash or jq required). It intercepts Agent tool calls and blocks Explore agents, redirecting Claude to use tokensave MCP tools instead.

CLAUDE.md rules

Appends instructions to ~/.claude/CLAUDE.md that tell Claude to use tokensave tools before reaching for Explore agents or raw file reads.

Crash-Resilient Sync

Tree-sitter grammars are compiled C/C++ code. They occasionally hit an internal assertion or otherwise terminate the process by paths that Rust panic handling cannot intercept. As of v4.3.0, every file is parsed inside a short-lived worker subprocess: if a grammar segfaults, calls abort(), or hits a stack overflow, only the worker dies. The pool respawns it, the offending file is logged and skipped, and sync keeps going.

The worker is a hidden extract-worker subcommand authenticated against the parent via a 256-bit per-spawn token, required as both a TOKENSAVE_WORKER_TOKEN env var and as the first 32 bytes received on stdin. Direct invocation by users fails. Defaults to available_parallelism() workers; opt out with TOKENSAVE_DISABLE_SUBPROCESS=1.

Edit primitives (tokensave_str_replace, tokensave_insert_at, etc.) still run in-process: they target one file at a time where subprocess overhead would dominate, and an extractor crash there is immediately visible to the agent.

Multi-Branch Indexing (Optional)

tokensave can optionally maintain a separate code graph per git branch. When enabled, switching branches never gives you stale results and never re-indexes files you already parsed on another branch. Multi-branch tracking is opt-in -- without it, tokensave uses a single database for all branches.

How it works

When you track a branch, tokensave copies the nearest ancestor DB and syncs only the files that differ. This means tracking a feature branch off main is nearly instant -- it only parses the files you've changed.

CLI commands

tokensave branch add              # track the current branch
tokensave branch list             # see tracked branches and DB sizes
tokensave branch remove <name>    # stop tracking a branch
tokensave branch removeall        # remove all tracked branches except default
tokensave branch gc               # clean up branches deleted from git

Cross-branch MCP tools

Three MCP tools enable cross-branch queries without switching your checkout:

• tokensave_branch_search -- search symbols in another branch's graph

• tokensave_branch_diff -- compare code graphs between two branches: symbols added, removed, and changed (signature differs). Supports file and kind filters.

• tokensave_branch_list -- list tracked branches with DB sizes, parent branch, and sync times

Branch fallback

When the MCP server can't find a database for the current branch, it serves from the nearest ancestor branch's DB and includes a warning in every tool response suggesting you run tokensave branch add.

See docs/BRANCHING-USER-GUIDE.md for the full guide.

48 MCP Tools

The discovery and analysis tools are read-only, safe to call in parallel, and annotated with readOnlyHint. The four edit primitives (the only writers) are scoped to single files and re-index in place. The three core tools (tokensave_context, tokensave_search, tokensave_status) are marked anthropic/alwaysLoad so they bypass the client's tool-search round-trip.

Discovery

Call Graph & Impact

Code Quality

Code-Health Analytics

Five tools surface structural quality signals from the existing graph. The composite score uses a geometric mean over independent dimensions so no single one can be gamed.

Sessions

Snapshot health metrics at the start of an AI coding session, then diff at the end to see what improved or regressed.

Edit Primitives

Four writer tools that let agents modify files without regex or shell-quoting hazards. Each is single-file, anchored, and triggers an in-place re-index after writing so the graph never goes stale.

Git & Workflow

Type System

Porting

Multi-Branch

MCP Resources

Four resources are exposed via resources/list and resources/read:

• tokensave://status -- graph statistics as JSON

• tokensave://files -- indexed file tree grouped by directory

• tokensave://overview -- project summary with language distribution and symbol kinds

• tokensave://branches -- tracked branches with DB sizes and parent info

Token Tracking

tokensave measures the tokens it saves on every MCP tool call. Each tool response includes a tokensave_metrics: before=N after=M line showing how many raw-file tokens were avoided by that specific call.

Cost observability

tokensave cost                     # 7-day cost summary (default)
tokensave cost today               # today only
tokensave cost --by-model          # breakdown by Claude model
tokensave cost --by-task           # breakdown by task category (coding, debugging, exploration, ...)
tokensave cost --export json       # JSON export to stdout
tokensave cost --export csv        # CSV export to stdout

Parses Claude Code session transcripts (~/.claude/projects/**/*.jsonl), classifies each API turn into one of 13 task categories, computes dollar cost using model pricing, and stores results in ~/.tokensave/global.db for fast aggregate queries. Pricing is refreshed from LiteLLM every 24 hours and falls back to an embedded table when offline.

The tokensave status header includes a cost row showing today's spend, 7-day total, and efficiency ratio (tokens saved / total tokens). The tokensave monitor TUI shows a live cost panel alongside the savings feed. At the end of each Claude Code session, the hook_stop handler prints a one-line receipt to the terminal.

Task classification categories: Coding, Debugging, Feature Dev, Refactoring, Testing, Exploration, Planning, Delegation, Git Ops, Build/Deploy, Brainstorming, Conversation, General. Classification is deterministic (pattern matching on tool names and Bash commands), requires no LLM calls, and is adapted from AgentSeal/codeburn.

Live monitor

tokensave monitor

A global TUI that shows MCP tool calls from all projects in real time, via a shared memory-mapped ring buffer at ~/.tokensave/monitor.mmap. Each entry shows the project name, tool name, and token delta. A cost panel at the top shows today's spend, savings, efficiency, and top model (refreshed every 30 seconds).

Session and lifetime counters

tokensave current-counter          # show per-project session counter
tokensave reset-counter            # reset the session counter
tokensave status                   # shows project + global lifetime totals + cost

Worldwide counter

All tokensave users contribute to an anonymous aggregate counter. tokensave status shows both your project total and the worldwide total. The upload sends only a single number (e.g. 4823) with no identifying information. Opt out with tokensave disable-upload-counter.

Background Daemon

The daemon watches all tracked projects for file changes and runs incremental syncs automatically.

tokensave daemon                       # start in foreground
tokensave daemon --enable-autostart    # install as launchd/systemd/Windows Service
tokensave daemon --disable-autostart   # remove autostart service
tokensave daemon --status              # show daemon status

The daemon is upgrade-aware: it snapshots its own binary's mtime and size at startup and checks every 60 seconds. When a package manager replaces the binary (brew upgrade, cargo install, scoop update), the daemon flushes pending syncs and exits. The service manager (launchd KeepAlive, systemd Restart=on-failure, Windows SCM failure actions) automatically relaunches with the new version.

Self-Upgrade

tokensave upgrade                  # upgrade to latest in current channel
tokensave channel                  # show current channel (stable/beta)
tokensave channel beta             # switch to beta channel
tokensave channel stable           # switch back to stable

tokensave upgrade downloads the correct platform binary from GitHub releases, stops the daemon, replaces the binary, and restarts the daemon. Supports stable and beta channels independently.

CLI Reference

tokensave init [path]              # Initialize a new project (full index)
tokensave sync [path]              # Incremental sync (must be initialized first)
tokensave sync --force [path]      # Force a full re-index
tokensave sync --doctor [path]     # Sync and list added/modified/removed files
tokensave status [path]            # Show statistics + cost summary
tokensave status [path] --json     # Show statistics (JSON output)
tokensave status --details         # Include node-kind breakdown
tokensave cost [range]             # Token cost summary (default: 7d)
tokensave cost --by-model          # Cost grouped by model
tokensave cost --by-task           # Cost grouped by task category
tokensave cost --export json|csv   # Export cost data
tokensave query <search> [path]    # Search symbols
tokensave files [--filter dir] [--pattern glob] [--json]   # List indexed files
tokensave affected <files...> [--stdin] [--depth N]        # Find affected test files
tokensave install [--agent NAME]   # Configure agent integration + daemon offer
tokensave reinstall                # Refresh settings for all installed agents
tokensave uninstall [--agent NAME] # Remove agent integration
tokensave serve                    # Start MCP server
tokensave monitor                  # Live TUI showing MCP calls across all projects
tokensave upgrade                  # Self-update to latest version
tokensave channel [stable|beta]    # Show or switch update channel
tokensave doctor [--agent NAME]    # Check installation health
tokensave branch add|list|remove|removeall|gc   # Multi-branch management
tokensave daemon [--enable-autostart|--disable-autostart|--status]
tokensave current-counter          # Show per-project token counter
tokensave reset-counter            # Reset per-project token counter
tokensave disable-upload-counter   # Opt out of worldwide counter uploads
tokensave enable-upload-counter    # Re-enable worldwide counter uploads

tokensave doctor

Run a comprehensive health check of your tokensave installation:

tokensave doctor

Checks: binary location, project index, global DB, user config, daemon status, agent integration (MCP server, hooks, permissions, prompt rules), and network connectivity. If any tool permissions are missing after an upgrade, it tells you to run tokensave install. Use --agent to check a specific agent only.

Doctor also validates that each installed hook uses the correct tokensave subcommand and auto-repairs broken hooks.

How It Works with Claude Code

Once configured, Claude Code automatically uses tokensave instead of reading raw files when it needs to understand your codebase. Three layers reinforce each other:

The result: Claude gets the same code understanding with far fewer tokens. A typical Explore agent reads 20-50 files; tokensave returns the relevant symbols, relationships, and code snippets from its pre-built index.

Network Calls & Privacy

tokensave's core functionality (indexing, search, graph queries, MCP server) is 100% local -- your code never leaves your machine.

The worldwide counter upload sends a single HTTP POST with a JSON body like {"amount": 4823}. No cookies, no tracking, no user ID. The Cloudflare Worker logs the country of your IP address (derived from request headers) for aggregate geographic statistics -- your actual IP address is not stored.

The model pricing refresh fetches a public JSON file from GitHub (raw.githubusercontent.com/BerriAI/litellm/main/model_prices_and_context_window.json) to keep Claude model pricing up to date for tokensave cost. No data is sent -- it is a plain HTTPS GET. The response is cached at ~/.tokensave/pricing.json for 24 hours. If the fetch fails, tokensave uses its compiled-in pricing table.

34 Languages

tokensave supports 34 programming languages organized into three tiers controlled by Cargo feature flags. Each tier includes all languages from the tier below it. As of v4.1.8, Markdown headers are also extracted (in the full tier) as Module nodes with hierarchical Contains edges, so document structure participates in graph queries alongside source code.

Lite (11 languages) -- --no-default-features

Always compiled. The smallest binary for the most popular languages.

Medium (Lite + 9 = 20 languages) -- --features medium

Full (Medium + 14 = 34 languages) -- default

Individual languages can also be cherry-picked without a full tier:

cargo install tokensave --no-default-features --features lang-nix,lang-bash

All extractors share the same depth: functions, classes, methods, fields, imports, call graphs, inheritance chains, docstrings, complexity metrics, decorator/annotation extraction, and cross-file dependency tracking.

tokensave vs CodeGraph

tokensave is a ground-up Rust rewrite of CodeGraph (Node.js/TypeScript). Both build semantic code graphs for AI coding agents, but they diverge significantly in scope and capabilities.

CodeGraph pioneered the approach and remains a solid choice if you prefer npm tooling and only need Claude Code integration. tokensave extends the concept with deeper analysis, more agents, background sync, multi-branch support, and a native binary with no runtime dependencies.

For detailed comparisons against CodeGraph, Dual-Graph (GrapeRoot), code-review-graph, and OpenWolf, see docs/COMPARABLE-TOOLS.md.

Why tokensave Over the Alternatives

Several tools reduce token usage for AI coding agents. Here's why tokensave stands apart.

Single native binary, zero dependencies

Every alternative requires a runtime: Python, Node.js, or both. tokensave ships as a single ~25 MB Rust binary with all 34 tree-sitter grammars bundled. Nothing else to install.

Deepest code intelligence

tokensave works at the symbol level: functions, structs, fields, call edges, type hierarchies, complexity metrics. Alternatives like Dual-Graph (GrapeRoot) work at the file level -- they know which files exist but can't answer "who calls this function?" or "what breaks if I change this struct?" tokensave's 48 specialized MCP tools cover call graph traversal, impact analysis, dead code detection, test mapping, rename preview, type hierarchies, circular dependency detection, complexity ranking, code-health analytics (Gini, DSM, dependency depth, risk-weighted test gaps), atomic edit primitives, and more. The closest competitor (code-review-graph) has 22 tools; others have 5-9.

Broadest agent support

9 AI coding agent integrations with per-agent native configuration formats. No other tool covers as many agents with as deep an integration. Claude Code gets hooks, prompt rules, and auto-allowed tool permissions. Other agents get MCP server registration in their native config format.

Multi-branch indexing

The only tool in this space with optional per-branch graph databases and cross-branch diff and search. When enabled, switching branches is instant -- no re-indexing required.

Per-call token tracking

The only tool that reports exactly how many tokens each individual MCP tool call saved, plus a live TUI monitor across all projects and lifetime counters.

Fully open source

MIT-licensed Rust, auditable end to end. Dual-Graph's core engine (graperoot on PyPI) is proprietary -- you can't see what it does with your code graph. OpenWolf is AGPL-3.0, which requires derivative works to be open-sourced.

Background daemon with upgrade awareness

The only tool that runs as a system service (launchd/systemd/Windows SCM), persists across sessions and reboots, and automatically restarts when the binary is upgraded by any package manager.

Performance

Full-index benchmark on a 1,782-file mixed Rust/Java/Scala codebase (57K nodes, 103K edges):

Troubleshooting

"tokensave not initialized"

The .tokensave/ directory doesn't exist in your project.

tokensave init

MCP server not connecting

The AI agent doesn't see tokensave tools.

1. Ensure the agent config includes the tokensave MCP server (run tokensave doctor)

2. Restart the agent completely

3. Check that tokensave is in your PATH: which tokensave

Missing symbols in search

• Run tokensave sync to update the index

• Check that the language is supported (see table above)

• Verify the file isn't excluded by .gitignore

Indexing is slow

Large projects take longer on the first full index.

• Subsequent runs use incremental sync and are much faster

• Use tokensave sync (not --force) for day-to-day updates

• The background daemon handles sync automatically

Origin

This project is a Rust port of the original CodeGraph TypeScript implementation by @colbymchenry. The port maintains the same architecture and MCP tool interface while leveraging Rust for performance and native tree-sitter bindings.

Building

cargo build --release                          # full (34 languages, default)
cargo build --release --features medium        # medium (20 languages)
cargo build --release --no-default-features    # lite (11 languages)

cargo test                                     # run all tests (requires full)
cargo check --no-default-features              # verify lite compiles
cargo clippy --all

Star History

License

MIT License -- see LICENSE for details.

tokensave.dev