Context-Optimizer-MCP
README.md
# Context-Optimizer-MCP
> **Cuts AI prompt context by 90–99% and reduces API costs by $70–$140 per 1,000 queries.**
A local-first **Model Context Protocol (MCP) server suite** that gives AI coding assistants persistent long-term memory and high-speed codebase discovery — eliminating context amnesia and token waste across every session.
---
## The Problem
Modern AI coding assistants (Claude, Cursor, Copilot) suffer from two compounding problems:
- **Context amnesia** — Every new session starts from zero. Architectural decisions, past mistakes, and established patterns must be re-explained each time.
- **Token waste** — To answer a simple question, the AI blindly reads thousands of lines of source code, burning tokens on irrelevant logic before finding anything useful.
Context-Optimizer-MCP solves both.
---
## Benchmark Results
Tested against this codebase (21 source files, **58,808 raw tokens**) across 15 diverse query types — from narrow configuration lookups to broad architectural questions.
| Metric | Value |
| :--- | :--- |
| Minimum context reduction | 97.4% |
| Median context reduction | **99.81%** |
| Maximum context reduction | 99.9% |
Run it yourself:
```bash
python benchmark.py
```
---
## Architecture
```
┌─────────────────────────────────────────────┐
│ AI Agent (Claude / Cursor) │
└───────────────┬─────────────────────────────┘
│ MCP Protocol
┌───────────┴────────────┐
│ │
┌───▼──────────┐ ┌────────▼────────┐
│ Memory │ │ Discovery │
│ MCP Server │ │ MCP Server │
│ │ │ │
│ Stores and │ │ AST + Regex │
│ retrieves │ │ codebase scan │
│ decisions, │ │ → endpoints, │
│ mistakes, │ │ queries, │
│ observations │ │ tech debt │
└──────┬───────┘ └────────┬────────┘
│ │
└──────────┬──────────┘
│
┌───────▼────────┐ ┌──────────────────┐
│ SQLite DB │◄────────►│ ai-memory.yaml │
│ mcp_memory.db │ cli.py │ (Git-tracked) │
└───────┬────────┘ └──────────────────┘
│
┌───────▼────────┐
│ FastAPI + │
│ React Dashboard │
└────────────────┘
```
---
## Core Components
### Memory MCP Server
Persistent SQLite-backed memory for AI agents. Stores decisions, mistakes, and observations with full lifecycle management.
- **Semantic deduplication** — Embeds incoming memories (Gemini `text-embedding-004` / OpenAI `text-embedding-3-small`) and runs cosine similarity in RAM. Similarity ≥ 0.85 triggers a merge instead of a new insert, incrementing the existing memory's confidence score. Falls back to exact-string matching when no API key is present.
- **Staleness tracking** — Classifies memories as `fresh` (<30 days), `warming` (30–90 days), or `stale` (>90 days) based on last validation timestamp. Auto-migrates older databases on startup.
- **Memory pruning** — `mem_prune` deletes unreinforced one-off entries (confidence == 1.0) older than N days, with dry-run mode on by default.
### Discovery MCP Server
Scans codebases structurally using AST parsing and regex — extracts API endpoints, database queries, class/function maps, and `# TODO` debt markers without reading implementation logic line-by-line.
Produces a lightweight "blueprint" of the project that the AI can query in ~150 tokens instead of reading the full source.
### Context Engine (FastAPI)
The search backend bridging agents and storage.
- **Dual-mode semantic search** — Uses vector embeddings when API keys are present; falls back to TF-IDF + cosine similarity for fully offline, zero-setup retrieval.
- **Context compression** — Retrieves, ranks, and compresses relevant memories and code structures before passing them to the LLM.
### React Dashboard
Local UI for auditing the AI's memory state.
- Trigger codebase scans manually
- Search memories with Google-style queries
- Verify/refresh `warming` and `stale` memory cards with a one-click checkmark (✓)
- View AST blueprints of the current project structure
---
## Key Design Decisions
**Cross-agent portability.** Memory is stored in open SQLite — no vendor lock-in. Switch from Claude to Gemini tomorrow; the new agent inherits the full project history instantly.
**Git-friendly memory sync.** The binary `.db` file is not committed directly. `cli.py export` converts it to a human-readable `ai-memory.yaml`. Teams commit the YAML, and `cli.py import --merge` rebuilds the database on each machine using the semantic dedup engine to resolve conflicts rather than overwriting.
**Zero mandatory dependencies.** No API key required to run. Semantic search degrades gracefully to TF-IDF offline mode. The whole system works on an air-gapped machine.
---
## Installation
**Requirements:** Python 3.10+, Node.js (only needed if rebuilding the dashboard; precompiled build included)
```bash
# 1. Clone and install
git clone https://github.com/your-username/Context-Optimizer-MCP.git
cd Context-Optimizer-MCP
pip install -r requirements.txt
# 2. Configure environment (API keys optional)
cp .env.template .env
# Add GEMINI_API_KEY or OPENAI_API_KEY to enable semantic search
# Leave blank for offline TF-IDF mode
# 3. Import memory from Git history
python cli.py import
# 4. Start the dashboard
python context_engine/server.py
# Open http://127.0.0.1:8000
```
---
## Connecting to AI Clients
### Cursor
Settings → Cursor Settings → Features → MCP → **+ Add New MCP Server**
| Field | Value |
| :--- | :--- |
| Name | `memory-server` |
| Type | `command` |
| Command | `python -u "C:/path/to/Context-Optimizer-MCP/mcp_servers/memory_server.py"` |
Repeat for `discovery-server` using `discovery_server.py`.
### Claude Desktop
Add to `claude_desktop_config.json`:
```json
{
"mcpServers": {
"codebase-memory": {
"command": "python",
"args": ["C:/path/to/Context-Optimizer-MCP/mcp_servers/memory_server.py"]
},
"codebase-discovery": {
"command": "python",
"args": ["C:/path/to/Context-Optimizer-MCP/mcp_servers/discovery_server.py"]
}
}
}
```
---
## CLI Reference
```bash
# Export SQLite → YAML (for Git)
python cli.py export
# Rebuild SQLite from YAML
python cli.py import
# Merge YAML into existing DB (semantic dedup on conflicts)
python cli.py import --merge
# Preview stale memories eligible for pruning (dry run)
python cli.py prune --days 90 --confidence 1.0
# Execute pruning
python cli.py prune --days 90 --confidence 1.0 --execute
```
---
## Tests
```bash
python -m unittest tests/test_memory_discovery.py
```
10 integration tests covering deduplication logic, YAML sync, staleness scoring, pruning, and benchmark validation. All passing.
---
## Tech Stack
Python · FastAPI · SQLite · React · Vite · Model Context Protocol (MCP) · Google Gemini Embeddings · OpenAI Embeddings · TF-IDF / Cosine Similarity
---
## License
MIT
This server cannot be deployed
Maintenance
ActivityStale
ResponsivenessNo issues