# TokenFluid

> Let AI work on your code safely. It gets the right context, works inside a controlled local workspace, validates changes, and only approved patches touch your real repository.

TokenFluid is a local-first AI development runtime. It gives AI coding assistants:

1. **Precise repository context** — token-budgeted context packs, FTS5 search, symbol extraction for 32 languages.
2. **Safe isolated workspaces** — a persistent local copy of your repo where the AI can experiment.
3. **Deterministic Git control** — all Git operations go through a safe `GitManager` that blocks destructive commands.
4. **Build/test validation** — run commands inside the workspace with a blocklist of destructive operations.
5. **Patch approval** — the AI prepares patches; only a human can apply them to the real repo.
6. **Full rollback history** — every attempt is a commit; roll back to any point.

**No cloud. No telemetry. No Docker. No remote execution. Your code stays on your machine.**

## Quick start

```bash
# Install
pip install -e .

# Index your repo (incremental, fast)
tokenfluid index /path/to/your/repo

# Build a context pack for a task
tokenfluid pack /path/to/your/repo "add forgot password feature" --max-tokens 8000

# Initialize a safe workspace (one-time per repo)
tokenfluid workspace init /path/to/your/repo

# Start a task
tokenfluid task start /path/to/your/repo "rename hello to greet"

# (The AI or you modifies files in the workspace copy)
# Workspace: /path/to/your/repo/.tokenfluid/workspaces/<id>/working-copy/

# Snapshot the changes
tokenfluid task snapshot /path/to/your/repo 1 --summary "renamed"

# Generate a patch (checks if it applies cleanly to the real repo)
tokenfluid task patch /path/to/your/repo 1

# Review the diff
tokenfluid task diff /path/to/your/repo 1

# Approve and apply to the real repo (HUMAN-CONTROLLED)
tokenfluid task approve /path/to/your/repo 1 --commit --commit-message "rename hello to greet"
```

## CLI commands

| Command | Description |
|---------|-------------|
| `tokenfluid index <repo>` | Index a repo. `--full`, `--stats`. |
| `tokenfluid status <repo>` | Show repo index stats + workspace + tasks. |
| `tokenfluid search <query> --repo <repo>` | Full-text search with ranked, explainable results. |
| `tokenfluid pack <repo> <task>` | Build a token-budgeted context pack. |
| `tokenfluid related <repo> <file>` | Find files related to a given file. |
| `tokenfluid memory add\|search\|list\|delete` | Manage saved memories. |
| `tokenfluid eval <repo> <cases.json>` | Run an evaluation benchmark. |
| `tokenfluid doctor <repo>` | Check environment + setup. |
| `tokenfluid plugins` | List all 32 language plugins. |
| `tokenfluid graph <repo> --kind imports\|calls\|inheritance` | Show dependency graphs. |
| `tokenfluid config <repo>` | Show or modify layered config. |
| `tokenfluid init <repo>` | Create `.tokenfluid/` with default config. |
| `tokenfluid export <repo>` / `tokenfluid import <repo>` | Export/import metadata. |
| `tokenfluid workspace init\|status\|list\|refresh\|reset\|remove <repo>` | Manage safe workspaces. |
| `tokenfluid task start\|status\|list\|snapshot\|diff\|patch\|rollback\|approve\|discard\|run\|build\|test\|lint` | Manage AI coding tasks. |
| `tokenfluid benchmark run\|report` | Run local benchmarks. |
| `tokenfluid serve --repo <repo>` | Run as a stdio MCP server. |

## MCP tools

TokenFluid exposes 17 tools over stdio MCP:

**Search & context:** `search_code`, `get_file`, `get_symbol`, `get_related_files`, `build_context_pack`, `recall_memory`, `save_memory`, `repo_status`

**Workspace & tasks (safe subset):** `workspace_status`, `task_start`, `task_status`, `task_snapshot`, `task_diff`, `task_patch`, `task_run`, `task_rollback`, `task_discard`

**NOT exposed via MCP:** `task_approve` — approval is human-controlled (CLI only).

See [`docs/mcp-setup.md`](docs/mcp-setup.md) and [`docs/mcp-workspace-tools.md`](docs/mcp-workspace-tools.md).

## Safety model

- **AI does not control Git.** All Git operations go through `GitManager` which blocks `reset`, `clean`, `push`, `pull`, `rebase`, `merge`, `stash`, etc.
- **Real repo is untouched until approval.** The AI works in a workspace copy. Only `tokenfluid task approve` (CLI, human-run) applies patches to the real repo.
- **Approval is human-controlled.** `task_approve` is NOT exposed as an MCP tool.
- **Workspace is local.** No Docker, no containers, no remote execution.
- **Path traversal protection** on every MCP tool that accepts a `rel_path`.
- **Secret files** (`.env`, `*.pem`, `id_rsa`) never have content indexed.
- **Build/test runner** blocks destructive commands (`rm -rf`, `git push`, `format`, etc.).

See [`docs/git-safety.md`](docs/git-safety.md), [`docs/workspace.md`](docs/workspace.md), [`docs/patch-approval.md`](docs/patch-approval.md), and [`docs/security.md`](docs/security.md).

## Privacy statement

TokenFluid indexes repositories locally. By default, code and memory are stored only on your machine in `.tokenfluid/`. This package does not call external LLM APIs or upload source code. No telemetry, no analytics, no auto-update pings.

## Limitations

- **Lexical only.** No embeddings or semantic search. Ranking is BM25 + token overlap + synonyms.
- **No multi-repo federation.** One SQLite DB per repo.
- **Workspace is a full copy.** For very large repos (>100k files), workspace init takes time and disk.
- **No web UI.** CLI + MCP only, by design.
- **Patch approval is manual.** The AI cannot auto-apply patches — that's the safety model.

## Documentation

- [`docs/architecture.md`](docs/architecture.md) — module map, data flow, schema.
- [`docs/workspace.md`](docs/workspace.md) — workspace engine.
- [`docs/git-safety.md`](docs/git-safety.md) — GitManager safety model.
- [`docs/task-workflow.md`](docs/task-workflow.md) — task lifecycle.
- [`docs/patch-approval.md`](docs/patch-approval.md) — patch approval safety.
- [`docs/benchmarks.md`](docs/benchmarks.md) — local benchmark suite.
- [`docs/mcp-workspace-tools.md`](docs/mcp-workspace-tools.md) — MCP workspace tools.
- [`docs/mcp-setup.md`](docs/mcp-setup.md) — generic MCP client config.
- [`docs/security.md`](docs/security.md) — threat model and controls.
- [`CHANGELOG.md`](CHANGELOG.md) — versioned changes.

## License

MIT. See [`LICENSE`](LICENSE).
"# tokenfluid-mcp"