⚠️ NOTICE — The rest of this repository (code, comments, scripts, docs, commits, issues) is written in French. Only this README is in English.

✅ SWAP DONE — 2026-10-24. IcyCache was renamed from CacheForge. The universal multi-upstream gateway is now the single active service on :8080, and the historical server holds the MCP layer.

• src/server.py → the gateway (18 upstreams in config/icycache.gateway.json: OmniRoute, LM Studio, Ollama, OpenAI, Anthropic, NVIDIA, Groq, Cerebras, OpenRouter, Kilo, Nous, Copilot, AI Gateway, HuggingFace, Google AI Studio, LLM7.io, Mistral, Cloudflare).

• src/mcp_server.py → the MCP layer (SSE :8081, Streamable HTTP :8084, stdio).

• Both start together via tray.py; :8080 is the single entry point for all LLM traffic.

• Legacy entry point kept: python src/server.py --port 8090 for isolated testing.

DELTA-002 — upstream resilience + opt-in semantic cache (2026-10-26)

• BUG-001 — 502 stuck after a 400: the httpx client is now drained, recycled, and the request retried once (src/upstream_client.py). The gateway recovers instead of staying stuck.

• Number normalization: "two plus two" → 2+2. 11 new rules (1111 → 1122).

• Semantic cache: implemented (src/icy_semantic.py) but off by default (semantic_enabled: false).

• Tool registry: the 100 verified MCP tool anchors (DELTA-001) are registered and asserted at startup.

Available MCP transports (all three active at once)

Transport	Endpoint	Target client
stdio	python mcp_stdio.py	Claude Desktop, IDEs, any stdio client
SSE (legacy)	http://127.0.0.1:8081/sse	Hermes, older clients
Streamable HTTP (standard)	http://127.0.0.1:8084/mcp	Recent MCP clients, MCP Inspector

In stdio mode, go through mcp_stdio.py: it reserves stdout for JSON-RPC. (Nothing is written to stdout anymore, but the wrapper remains the safety net.)

🔥 IcyCache — Universal LLM Gateway with local cache

"Forge the cache, not the answer." HTTP gateway that sits between any client and any upstream (local or cloud), caches responses and serves them back instantly — no CPU regeneration.

📖 Full configuration guide : docs/CONFIGURATION.md — multi-upstream routing, per-client setup (Hermes, Claude Desktop, Cursor, VS Code, Ollama, direct SDK), MISS→HIT proof, troubleshooting, firewall enforcement.

Related MCP server: qualixar/superlocalmemory

The problem

IcyCache attacks all four: normalization before hashing, SQLite cache with TTL, exportable metrics, and a single entry point for all LLM traffic.

Semantic cache status: implemented but opt-in (semantic_enabled: false by default). It needs an embedding endpoint (LM Studio on :1234) and a Qdrant instance; when either is missing the cache degrades silently to a miss instead of failing the request. See config/semantic_cache.json.

Target architecture

   Any client                 (Hermes, Claude Code, scripts, IDE, SDK, CLI)
   base_url = http://127.0.0.1:8080
              │
              ▼
┌───────────────────────────────────────────────────────────────────────┐
│  ICYCACHE GATEWAY  127.0.0.1:8080   (src/server.py)                   │
│                                                                       │
│  ROUTING      URL prefix  /<upstream>/v1/...                          │
│               header      X-IcyCache-Upstream: lmstudio               │
│               body        "upstream": "ollama"                        │
│               default     gateway.default_upstream (config)           │
│                                                                       │
│  NORMALIZATION NFKD → lowercase → Aho-Corasick → accents → punctuation│
│  CACHE KEY     SHA256 of the NORMALIZED payload (+ upstream + endpoint)│
│  DECISION      HIT → instant response | MISS → forward + save         │
│  TOKENS        REAL usage from the upstream. Never an estimate.       │
│  OBSERVABLE    WebSocket /ws/events : live deltas + final             │
│  MCP (facade)  stats / clear / search / realtime / tool-cache         │
└───────────────────────────────────────────────────────────────────────┘
        │                    ▲                              │
        ▼                    │ same path back                │
 { OmniRoute | LM Studio | Ollama | OpenAI | Anthropic | … }
        │
        ▼
   SQLite db/cache.db   (events: upstream, real tokens, latencies, ttft)

Rule: everything that leaves a client and comes back to it goes through IcyCache. OmniRoute is no longer the entry point: it is one upstream among others.

Components

Routing: pick your upstream

No implicit default in the code: without an explicit source, the request is rejected with 400 listing the available upstreams. An unknown upstream is rejected with 404.

HTTP routes (gateway)

Confirmation headers: X-IcyCache-Confirm (unified naming). The old X-CacheForge-Confirm is still accepted by the gateway for compatibility.

Tokens: the rule of the real

events columns: upstream, client, prompt_tokens, completion_tokens, cached_tokens, cache_write_tokens, latency_ms, ttft_ms.

MCP tools (facade)

MCP server (src/mcp_server.py) — 11 tools

Gateway server (src/mcp_gateway.py) — 11 tools

The 7 observability tools (get_cache_stats, clear_cache, normalize_preview, search_cached_prompts, get_realtime_tokens, list_upstreams, tool_cache_stats) plus the 4 relay tools (proxy_tool, proxy_tools, list_proxy_servers, list_proxy_tools).

MCP is a facade: it observes and drives. It is not in the request path (an MCP client calls its tools deliberately — you cannot intercept an OpenAI/Anthropic stream with it). The request path is the HTTP gateway.

Agent loops: icy_core.llm_loop_check() detects the same tool called again with the same arguments and the same results (Anthropic and OpenAI formats). Threshold: 3 repetitions = warning.

Tool calling: safety rules

The tool proxy (src/tool_proxy.py) relays calls to target MCP servers. Three non-negotiable rules:

1. An MCP error is never cached and never returns ok: true. An isError: true becomes ok: false with the real message (no opaque ExceptionGroup).

2. Write tools are never cached. Replaying a cached response must have no side effect; otherwise a write is silently skipped.

   • cache_tools present → strict allowlist (only tools listed as true are cached).

   • otherwise cache: true → anti-write heuristic (create_, write_, delete_, push_, … excluded).

   • otherwise → no cache.

3. MCP sessions are persistent (one process per server, not per call). Without this, every tool call respawned a process: 21s measured on npx, 12s on uvx.

Example

// via any MCP client
{"server": "github", "tool": "search_repositories", "args": {"query": "icycache"}}
// -> {"ok": true, "cached": false, "latency_ms": 4867, "result": {...}}

Target server configuration

config/icycache.json → tool_proxy.servers. Two portability tokens:

On Windows, .cmd commands (like npx) are resolved automatically: CreateProcess cannot launch an npx.cmd script without a shell.

Enforcement: making bypass impossible

# 1. Install the rules (ELEVATED PowerShell)
powershell -ExecutionPolicy Bypass -File tools\enforce_gateway.ps1 -Action apply `
    -TargetProgram "C:\Users\Tech\AppData\Local\hermes\hermes-agent\apps\desktop\release\win-unpacked\Hermes.exe" `
    -AllowPort 8080 -BlockPorts 20128,1234,11434

# 2. The proof: can the client still reach an upstream directly?
python tools\verify_enforcement.py --program "D:\IcyCache\venv\Scripts\python.exe" --allow-port 8080

Strategy (Windows Firewall, per program): block upstream local ports including on 127.0.0.1, block all remote traffic, explicitly allow 127.0.0.1:8080. The verify_enforcement.py program runs with the client binary (rules are per program) and checks that only IcyCache remains reachable.

Environment variables (unified naming)

Installation

The registry publication is pending. The repository, the PyPI distribution and the npm package all carry the same name — icycache — and the packages are built, tagged and ready to publish (v0.3.2). The names were free on both registries, and icycache-mcp is the second binary the npm package exposes.

The registry commands below are the final ones, but they return a 404 until the first publish. Install from source in the meantime: pip install "icycache @ git+https://github.com/MlkBlk-IT/icycache.git".

uv tool install icycache          # or: pip install icycache

This gives you the icycache console script:

icycache serve          # the gateway, http://127.0.0.1:8080
icycache mcp            # the MCP layer (SSE :8081 / HTTP :8084 / stdio)
icycache doctor         # diagnostic: paths, config, upstreams, database

Installed packages never write into site-packages: db/ and logs/ go to %LOCALAPPDATA%\IcyCache (Windows) or ~/.local/share/IcyCache. Full guide: docs/INSTALL.md.

npm (Node-only clients)

The npm package is icycache — the same name as the repository. It exposes two binaries:

npx -y icycache --help              # the CLI (serve / mcp / doctor)
npx -y -p icycache icycache-mcp     # MCP server on stdio

From source (Windows)

# 1. Environment (Python 3.12.10)
python -m venv venv
.\venv\Scripts\Activate.ps1
pip install -e ".[dev]"

# 2. Manual start
.\start_icycache.ps1

# 3. Start at login (Task Scheduler, or Startup fallback without admin)
.\install_autostart.ps1

Build the packages yourself

python -m build          # -> dist/icycache-<version>-py3-none-any.whl + .tar.gz

Building and publishing this package: see PUBLISHING.md.

Tests

# Isolated suite (mock upstream, temp DB, no port occupied)
.\venv\Scripts\python.exe -m pytest tests\test_gateway.py -v

# Isolated integration suite (temp DB + fake upstream :8190/:8191)
.\venv\Scripts\python.exe tests\run_isolated.py

# Offline PrefixProxy test (fake provider, no external call)
.\venv\Scripts\python.exe tests\test_prefix_proxy.py

# Real smoke test (LM Studio / OmniRoute, port 8090, temp DB)
.\venv\Scripts\python.exe tools\smoke_gateway.py --port 8090

# DELTA-002 suite (semantic cache, upstream recovery, normalization anchors)
.\venv\Scripts\python.exe -m pytest tests\test_semantic_cache.py tests\test_upstream_recovery.py tests\test_anchors.py -v

The smoke test touches neither the production service (:8080) nor the dashboard. tests\run_isolated.py uses a temporary database (ICYCACHE_DB) and never touches the production database.

License

MIT