Context Guardian

MCP server that cuts cloud LLM costs 36-42% by indexing context locally and giving agents precision retrieval tools instead of raw context dumps.

Works with any MCP-compatible agent (Claude Code, Cursor, Aider, Cline, Factory Droid). Also works as a transparent proxy for agents that use OpenAI/Anthropic APIs directly.

How It Works

1. Your agent sends a massive prompt (code + logs + configs) to the cloud

2. Context Guardian intercepts it, chunks and indexes it locally using Ollama

3. Instead of the raw dump, the cloud model gets tools to search the indexed content

4. The model retrieves only what it needs → fewer tokens processed → lower cost

The local LLM (Qwen 3.5 4B) handles classification and embedding. The cloud model does the reasoning. You pay for focused retrieval, not haystack scanning.

Related MCP server: MCP of MCPs

Quick Start (MCP Mode)

# Install
npm install -g context-guardian-mcp

# Start the MCP server
context-guardian mcp

# Add to your agent's MCP config:
#   URL: http://localhost:9120/mcp

Factory Droid

droid mcp add context-guardian http://localhost:9120/mcp --type http

Claude Code / Cursor / Cline

Add to your MCP settings:

{
  "mcpServers": {
    "context-guardian": {
      "url": "http://localhost:9120/mcp"
    }
  }
}

Quick Start (Proxy Mode)

# Start the proxy
context-guardian start

# Point your agent to the proxy instead of OpenAI/Anthropic:
export OPENAI_BASE_URL=http://localhost:9119/v1

The proxy intercepts requests above the token threshold, rewrites them with RAG tools, runs a multi-round tool loop with the cloud, and returns the final response. Requests below the threshold pass through unchanged.

Requirements

• Node.js 20+

• Ollama running locally with:

  • ollama pull qwen3.5:4b (intent extraction + classification)

  • ollama pull nomic-embed-text (embeddings)

MCP Tools Exposed

Benchmark Results

Measured on a real codebase (WebLLM/Bonsai-WebGPU R&D repository — public), 3 scenarios (75K-98K tokens each), 3 repeats, live Claude Opus via Factory Droid.

Key findings:

• Zero accuracy loss when the cloud model uses tools autonomously (unguided MCP mode)

• 36-42% cost reduction measured in actual billed tokens (Anthropic pricing)

• On investigation tasks (finding bugs in 75K+ token logs), cost savings reach 50-67%

• On dense, mostly-relevant code contexts, savings are minimal (~2-14%) — CG correctly adds less value when context is already focused

• Adds 15-25 seconds per request (MCP indexing + tool calls). Negligible for 20-30 minute agent sessions

Methodology

• Real files, not synthetic noise. Context is actual GPU kernel code, audit documents, and analysis reports

• Facts scattered at random positions (not head/tail) to avoid primacy/recency bias

• Ground truth is regex-checked against specific values in the codebase

• 95% confidence intervals computed via Student's t-distribution

• Reproducible: npm run bench:oss (local, no API key needed) or npm run bench:realworld (requires Droid CLI)

• Full results in benchmark/realworld-3way-results.json

CLI Commands

context-guardian start          # Start proxy server (port 9119)
context-guardian mcp            # Start MCP server (port 9120)
context-guardian init           # Generate default config file
context-guardian sessions       # List recent proxy sessions
context-guardian compact        # Manual session compaction
context-guardian eval --prompt  # A/B evaluation of a prompt
context-guardian dry-run --file # Preview rewrite without cloud call
context-guardian check          # Verify Ollama + model availability

Configuration

Create .context-guardian.json with context-guardian init, or pass CLI flags:

context-guardian start \
  --port 9119 \
  --threshold 8000 \      # tokens above this trigger interception
  --budget 4000 \         # max tokens in rewritten request
  --model qwen3.5:4b \    # local LLM model
  --endpoint http://localhost:11434  # Ollama URL

Architecture

Agent (Claude Code, Cursor, Aider...)
  │
  ├─[MCP mode]─→ Context Guardian MCP Server ──→ Ollama (local)
  │               index_content / grep / search    ↓
  │               ←── tool results ────────────── chunks + embeddings
  │
  └─[Proxy mode]→ Context Guardian Proxy ──→ Cloud API (OpenAI/Anthropic)
                   intercept → rewrite → tool loop → response

Local processing: Chunking, classification (7 types), embedding (nomic-embed-text), cosine similarity search, entity extraction, session memory with SQLite persistence.

Cloud forwarding: OpenAI and Anthropic format detection, multi-round tool call loop (up to 10 rounds), streaming SSE synthesis, auto-compaction of tool loop context.

License

MIT

Author

Kuldeep Singh — LinkedIn