Tokenectomy Razor

Fast, deterministic log surgery and secret redaction for AI coding agents — purge 90%+ framework noise, redact credentials with O(N) ReDoS immunity, sub-millisecond latency. Written in safe Rust.

Tokenectomy (noun): token + -ectomy (surgical removal) — the precise excision of wasteful tokens from LLM context windows.

High-Performance Log Surgery & Secret Redaction Engine for AI Coding Agents

• Official MCP Registry: mcp-name: io.github.Tokenectomy-Labs/razor

📋 Table of Contents

• What It Does

• Key Features

• Benchmarks

• Installation

• MCP Integration

• Usage by Use Case

• Advanced Features

• Comparison

• FAQ

• Roadmap

• Security

• Contributing

Related MCP server: uacos

What It Does

Tokenectomy Razor is an autonomous, machine-to-machine (M2M) Model Context Protocol (MCP) server and stream processing engine written in safe Rust. It intercepts error logs from AI agents, strips 90%+ of framework noise, automatically redacts secrets (JWTs, API keys, database credentials), and caches sanitized contexts with a 24-hour TTL—all without sending raw data to external services.

In 30 Seconds

The Problem:

• AI agents waste tokens on framework noise (node_modules, site-packages, .cargo/registry)

• Sensitive credentials accidentally leak into LLM logs (AWS keys, database URLs, API tokens)

• Repeated identical errors cost money for every retry

The Solution:

Raw Error Log (38K tokens + secrets)
    ↓
[Redact secrets locally] → [Filter framework frames] → [Extract user code]
    ↓
Sanitized Context (2K tokens, no secrets) → Safe to send to LLM

Real Example

Before:

$ cat error.log | head -20
Error in /home/user/.cargo/registry/src-xxx/tokio-1.35/src/runtime/mod.rs:12345
  at /home/user/.cargo/registry/src-yyy/serde/src/lib.rs:456
  Database connection failed: postgresql://admin:secretpass@db.example.com:5432/mydb
  JWT Auth token: eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIn0...
  [... 500+ more framework frames ...]

After:

$ cat error.log | razor --scrub
Error in /home/user/src/main.rs:42
  at /home/user/src/utils.rs:18
  Database connection failed: [CONNECTION_STRING_REDACTED]
  JWT Auth token: [JWT_REDACTED]

Benefits:

• ✅ 95% smaller context (2K vs 38K tokens) → Save money on LLM API calls

• ✅ Zero secrets in logs → Sleep better at night

• ✅ Identical errors cached → Second retry costs $0

Technical Highlights

                  ┌──────────────────────────────────────────────┐
   Agent Error     │              TOKENECTOMY RAZOR               │    Sanitized Context
   Dump (38K toks) │  - Polyglot Stack Frame Filter               │ ──►  (2K toks) ──► LLM
  ────────────────►│  - Deterministic Secret Redactor (O(N))      │
                   │  - SHA-256 Idempotency Cache (24h TTL)       │
                   └──────────────────────────────────────────────┘

• Deep Polyglot Trace Surgery: In-memory parsing across Rust, Python, TypeScript/JavaScript, Go, Java/Kotlin (Spring Boot 3, Tomcat, Hibernate, Netty, Undertow, HikariCP), C/C++ (AddressSanitizer, GDB, glibc), and PHP. Surgically filters noisy framework internals and runtime boilerplate while isolating genuine user application code frames.

• Static AST Code Analysis Engine (analyze_code): High-throughput static AST analysis detecting unclosed handles, resource leaks, and security vulnerabilities with bounded execution limits (50K AST nodes, 10 MB file limit) and precise LSP UTF-16 coordinates.

• Glama Grade A TDQS Compliance: 100% Tool Definition Quality Score with explicit schema boundaries, runtime preconditions, and full disclosure across all MCP tools.

• AI Gateway Reverse Proxy (--proxy): Transparently intercepts prompt streams on 127.0.0.1:8080, performing real-time token excision and credential sanitization before upstream forwarding to OpenAI, Anthropic, or Ollama.

• Zero-Knowledge Secret Redaction: Linear-time deterministic regex engine strips JWTs, API tokens, cloud access keys, connection strings, and private keys prior to network transmission. All processing happens locally.

• SHA-256 Idempotency Cache: Stores deterministic responses with a 24-hour TTL. Repeated CI/CD or agent loop failures incur zero upstream API cost.

• Path Traversal Containment: All MCP filesystem access is canonicalized and locked to the workspace root boundary (CWD). No ../ escapes or symlink breakouts.

• M2M Protocol Compliance: Native JSON-RPC 2.0 stdio server compliant with the official Model Context Protocol specification.

Verifiable Benchmarks

Performance metrics are hardware-grounded and reproducible via standalone benchmark suites:

Understanding the Benchmarks

Reproduce locally:

cargo test --release --test stress_benchmark -- --nocapture

Installation

Method 1: Instant via npx (Recommended for MCP Clients)

No Rust toolchain, native compilation, or manual path setup required:

npx -y tokenectomy-razor --mcp

Or install globally via npm:

npm install -g tokenectomy-razor

Method 2: Cargo (crates.io)

cargo install tokenectomy

Method 3: Precompiled Native Binaries (GitHub Releases)

Download zero-dependency, precompiled standalone binaries directly from GitHub Releases:

• Linux: tokenectomy-linux-x86_64 (glibc), tokenectomy-linux-x86_64-musl, tokenectomy-linux-aarch64

• macOS: tokenectomy-darwin-arm64 (Apple Silicon M1/M2/M3/M4), tokenectomy-darwin-x86_64 (Intel)

• Windows: tokenectomy-windows-x86_64.exe

Method 4: Build from Source

git clone https://github.com/Tokenectomy-Labs/Tokenectomy.git
cd Tokenectomy
cargo build --release
sudo cp target/release/razor /usr/local/bin/razor

Method 5: Multi-Arch Container (GHCR)

docker pull ghcr.io/tokenectomy-labs/razor:latest
docker run -it ghcr.io/tokenectomy-labs/razor:latest --help

Model Context Protocol (MCP) Integration

Configure Tokenectomy Razor as an autonomous background server across major AI agent environments:

Claude Desktop

Add to ~/Library/Application Support/Claude/claude_desktop_config.json (macOS) or %APPDATA%\Claude\claude_desktop_config.json (Windows):

{
  "mcpServers": {
    "tokenectomy": {
      "command": "npx",
      "args": ["-y", "tokenectomy-razor", "--mcp"]
    }
  }
}

Cursor

Add to .cursor/mcp.json in your project root:

{
  "mcpServers": {
    "tokenectomy": {
      "command": "npx",
      "args": ["-y", "tokenectomy-razor", "--mcp"]
    }
  }
}

VS Code (Native MCP / GitHub Copilot Agent / Continue)

For VS Code with native MCP support, create or edit .vscode/mcp.json in your workspace:

{
  "mcpServers": {
    "tokenectomy": {
      "command": "npx",
      "args": ["-y", "tokenectomy-razor", "--mcp"]
    }
  }
}

Or use the ultra-low latency native binary (if installed via cargo install tokenectomy):

{
  "mcpServers": {
    "tokenectomy": {
      "command": "razor",
      "args": ["--mcp"]
    }
  }
}

VS Code + Cline

Open Cline Settings in VS Code (or edit cline_mcp_settings.json):

• macOS: ~/Library/Application Support/Code/User/globalStorage/saoudrizwan.claude-dev/settings/cline_mcp_settings.json

• Linux: ~/.config/Code/User/globalStorage/saoudrizwan.claude-dev/settings/cline_mcp_settings.json

• Windows: %APPDATA%\Code\User\globalStorage\saoudrizwan.claude-dev\settings\cline_mcp_settings.json

{
  "mcpServers": {
    "tokenectomy": {
      "command": "npx",
      "args": ["-y", "tokenectomy-razor", "--mcp"],
      "disabled": false,
      "autoApprove": [
        "get_error_context",
        "search_stack_overflow",
        "analyze_code"
      ]
    }
  }
}

VS Code + Roo Code

In Roo Code Settings (or edit cline_mcp_settings.json in Roo storage):

• Linux: ~/.config/Code/User/globalStorage/rooveterinaryinc.roo-cline/settings/cline_mcp_settings.json

• macOS: ~/Library/Application Support/Code/User/globalStorage/rooveterinaryinc.roo-cline/settings/cline_mcp_settings.json

{
  "mcpServers": {
    "tokenectomy": {
      "command": "npx",
      "args": ["-y", "tokenectomy-razor", "--mcp"],
      "disabled": false,
      "autoApprove": ["get_error_context", "analyze_code"]
    }
  }
}

Windsurf (Codeium)

Add to ~/.codeium/windsurf/mcp_config.json:

{
  "mcpServers": {
    "tokenectomy": {
      "command": "npx",
      "args": ["-y", "tokenectomy-razor", "--mcp"]
    }
  }
}

Google Antigravity CLI

agy mcp add tokenectomy-razor -- npx -y tokenectomy-razor --mcp

Exposed MCP Tools

Quick Start

I use Claude Desktop

# 1. Add to claude_desktop_config.json (see MCP Integration section above)
# 2. When Claude encounters errors, it automatically uses "get_error_context" tool
# 3. Errors stay sanitized without any additional setup

I use GitHub Actions

# Add to your workflow (.github/workflows/build.yml)
- name: Sanitize Build Failure Log
  if: failure()
  uses: daffa2555/tokenectomy-action@v1
  with:
    log-file: 'build.log'
    output-file: 'sanitized.log'

# Now you can safely share sanitized.log without leak concerns

Parameters:

I want max privacy (air-gapped environment)

# All redaction happens locally—no network calls except to your LLM
cargo install tokenectomy

# Process logs without any cloud services
echo $ERROR_LOG | razor --scrub --local-only

# Or from a file:
razor --scrub --file /var/log/app/error.log > sanitized.log

Advanced Usage

Standalone CLI

In addition to M2M agent mode, Razor provides CLI commands for terminal piping and local shell scripting:

# Scrub framework frames and output clean log
npm test 2>&1 | razor --scrub > sanitized.log

# Sanitize a specific log file
razor --scrub --file /var/log/app/error.log > sanitized.log

# CLI diagnosis with specific AI provider
razor --file error.log --provider openai
razor --file error.log --provider anthropic
razor --file error.log --local-only

AI Gateway Reverse Proxy Mode

Tokenectomy Razor can operate as a transparent local HTTP reverse proxy. It sits between client applications and upstream LLM providers (OpenAI, Anthropic, Ollama, OpenRouter), performing real-time token excision and credential sanitization before upstream forwarding.

Local Development (Default Loopback):

# Forward to OpenAI
razor --proxy --proxy-bind 127.0.0.1:8080 --upstream-url https://api.openai.com/v1

# Forward to local Ollama instance
razor --proxy --proxy-bind 127.0.0.1:8080 --upstream-url http://127.0.0.1:11434/v1

Point any standard SDK or IDE client to the local proxy:

export OPENAI_BASE_URL="http://127.0.0.1:8080/v1"
# Now all API calls are automatically sanitized

Production Proxy Hardening

Binding to external interfaces (0.0.0.0) requires explicit token authorization:

razor --proxy --proxy-bind 0.0.0.0:8080 --upstream-url https://api.openai.com/v1 --allow-remote --proxy-token "YOUR_SECURE_TOKEN"

Resource limits enforced: MAX_HEADER_SIZE (64 KB), MAX_BODY_SIZE (10 MB), client/upstream timeouts (30s / 60s), and a 128-connection concurrency cap.

Real-Time FinOps Economics Dashboard & Metrics

Tokenectomy Razor features an embedded zero-dependency real-time FinOps dashboard and metrics exporter:

• Interactive UI: Open http://127.0.0.1:8080/dashboard in any browser to inspect live token savings, dollars saved (blended LLM pricing), total requests, and active security redactions.

• Prometheus / JSON Metrics: Poll GET http://127.0.0.1:8080/v1/metrics for programmatic FinOps telemetry integration.

• Health Check: GET http://127.0.0.1:8080/health returns gateway operational status.

Configuration

Configuration values can be set via ~/.tokenectomy.toml:

default_provider = "openai"  # openai | anthropic | ollama | mock
openai_api_key = "sk-..."
anthropic_api_key = "sk-ant-..."
ollama_base_url = "http://localhost:11434"
context_lines = 10
max_context_chars = 10000

Supported Ecosystems

Note: Currently shipped with robust extractors for Rust, Python, TypeScript/JavaScript, and Go. Java/Kotlin, C/C++, and PHP support is coming in v1.2. See #1 for progress tracking.

How Tokenectomy Compares

When to Use Tokenectomy:

• ✅ You use AI coding agents (Claude, Cursor, Cline, etc.)

• ✅ You care about privacy & local-first processing

• ✅ You want to reduce LLM token costs

• ✅ You're worried about secret leakage in logs

When to Use Something Else:

• ❌ You only need static secret scanning → use truffleHog, detect-secrets

• ❌ You need real-time monitoring dashboards → use Datadog, New Relic, Splunk

• ❌ Your error logs are naturally <100 tokens → overhead not worth it

• ❌ You're fully air-gapped → Actually Tokenectomy is perfect! (100% local processing)

Frequently Asked Questions

Q: Does Tokenectomy send my logs to external servers?

A: No. All redaction, parsing, and filtering happens locally on your machine. The only network call is to your chosen LLM (OpenAI, Anthropic, Ollama) after sanitization is complete. See SECURITY.md for the zero-knowledge guarantee.

Q: What secrets does Tokenectomy redact?

A: GitHub PATs, AWS keys, OpenAI/Anthropic API keys, JWTs, database connection strings (PostgreSQL, MySQL, MongoDB, Redis), private SSH keys, Slack/Discord webhooks, and more. Full list in src/redact.rs.

Q: What if my secret doesn't match the redaction patterns?

A: File an issue with an example (sanitized). We'll add the pattern. For now, you can add custom patterns in ~/.tokenectomy.toml (feature coming in v1.3).

Q: Is Tokenectomy safe for production?

A: Yes. Written in safe Rust (zero unsafe code in security paths), ReDoS-immune, and audited via RustSec. See SECURITY.md for full details.

Q: Can I use Tokenectomy offline?

A: Yes—except Stack Overflow search. Use --local-only flag to disable all network access (except your LLM).

Q: How do I remove Tokenectomy?

A: Simply uninstall:

npm uninstall -g tokenectomy-razor
# OR
cargo uninstall tokenectomy

Zero config cleanup needed—no files left behind.

Q: Can I use Tokenectomy in my CI/CD pipeline?

A: Yes! Use the GitHub Marketplace action (see Quick Start section) or the Docker container. Works with GitHub Actions, GitLab CI, Jenkins, etc.

Q: What's the difference between Razor (OSS) and Sentinel (Commercial)?

A: Razor is the free, community version with all essential features. Sentinel adds advanced capabilities like tree-sitter AST healing, anti-hallucination guards, and time-machine undo. See Edition Comparison below.

Edition Comparison

Interested in Sentinel? View pricing & features

Roadmap

Security & Reliability Invariants

• Zero-Knowledge Processing: All scanning and redaction occurs on local hardware before data leaves the system boundary.

• ReDoS Immunity: All pattern matchers utilize finite automaton evaluation with linear time guarantees. Verified in benchmarks.

• Path Traversal Isolation: File operations are strictly locked within workspace boundaries via WorkspaceBoundary security module.

• Memory Safety: Implemented in safe Rust with bounded stream readers (.take()) preventing resource exhaustion attacks.

• Audit Verification: Continuous dependency auditing maintained via RustSec advisory databases.

Vulnerability Disclosure: See SECURITY.md for responsible disclosure procedures.

Contributing

Found a bug? Have a feature request? Want to add support for a new language?

1. Issues: github.com/Tokenectomy-Labs/Tokenectomy/issues

2. Pull Requests: Fork, create a feature branch, and submit a PR with tests

3. Security: See SECURITY.md for private vulnerability disclosure

See CONTRIBUTING.md for detailed contribution guidelines.

Resources

• 📖 Documentation — Full guides, API reference, and integration tutorials

• 📋 Changelog — Release history and notable changes

• 🤝 Contributing — How to contribute, development workflow, and testing

• 🔒 Security Policy — Vulnerability disclosure and audit details

• 🏗️ Architecture — Internal design and system architecture

License

MIT License. See LICENSE for full terms.

Made with ❤️ by @daffa2555

Questions? Open an issue or start a discussion on GitHub.