IcyLoom

One MCP server. One SQLite file. One ledger that tells you what it actually saved.

A local gateway that puts an LLM cache, a compressed context drawer and an MCP tool relay behind one MCP server, one configuration file and one SQLite database.

IcyLoom is a standalone project: no imports, links or paths to any other project. The code is written from scratch; the only files taken from elsewhere are data (normalisation rules, a tool registry), because the cache hash depends on them.

The numbers are real. A cache hit is counted as avoided, a miss is counted as paid with the upstream's own usage figures — never estimated. The 35 % threshold is read straight from the ledger.

Overview: the unified registry, five live KPIs, calls per bucket, latency, outcomes, models, upstreams.

Quick start

Three commands, and it works:

# 1. check that everything works, like a real client
.\.venv\Scripts\python.exe -m icyloom check

# 2. plug IcyLoom into your MCP client (writes the config on your machine)
.\.venv\Scripts\python.exe -m icyloom client --target cursor --install

# 3. read the config to paste (Claude Code, Claude Desktop, Cursor, Windsurf, Codex)
.\.venv\Scripts\python.exe -m icyloom client --target claude

icyloom check plays the role of the client: it starts the server, connects, lists the 13 tools and calls them (ctx_store, ctx_search, ctx_verify, cache, ledger, proxy_info). It tests the three transports (stdio, streamable-HTTP, SSE) and prints 21/21 vérifications réussies when all is well. If HTTP or SSE fails, the stack is not running (icyloom serve); the stdio mode, on the other hand, is self-contained — it is the one Claude Code and Cursor use.

For the HTTP gateway (your LLM client pointing base_url at IcyLoom) and the web console:

.\.venv\Scripts\python.exe -m icyloom serve      # 8150 / 8151 / 8152 / 8153

D:\ICY-LOOM
  icyloom.yaml      single configuration (Pydantic)
  data\             externalised data (JSON)
  var\icyloom.db    single SQLite database (WAL)

  HTTP gateway :8150                      MCP server (one build function)
    /v1/chat/completions                    stdio             dedicated process
    /v1/messages                            streamable-HTTP   :8151
    /v1/models, /v1/embeddings              SSE               :8152
    /{upstream}/v1/...                      13 tools, one surface
    cache -> routing -> upstream

  admin API + dashboard :8153
  gateway, MCP and admin all write to one database and one ledger

Related MCP server: mcplex

Status (verified 2026-10-28)

This section is the honest summary. Details and raw numbers are in PROGRESS.md.

The "50 percent tokens saved" printed by tests/proof_e2e.py is true by construction: that test sends every request twice. It is not a traffic figure and must not be quoted as one.

Getting started

cd D:\ICY-LOOM
.\.venv\Scripts\python.exe -m icyloom doctor        # diagnostics
.\.venv\Scripts\python.exe -m icyloom serve         # gateway + MCP + admin
.\.venv\Scripts\python.exe -m icyloom mcp --transport stdio   # for an MCP client
.\.venv\Scripts\python.exe -m icyloom gateway       # gateway only (8150)
.\.venv\Scripts\python.exe -m icyloom admin         # admin API only (8153)

An MCP client connects with a single entry, for example:

{ "mcpServers": { "icyloom": { "command": "D:\\ICY-LOOM\\.venv\\Scripts\\python.exe",
                              "args": ["-m", "icyloom", "mcp", "--transport", "stdio"] } } }

An LLM client points its base_url at http://127.0.0.1:8150/v1.

The tool cache

IcyLoom relays calls to downstream MCP servers and can cache their results. The rules:

1. Deny by default. Only tools set to true in a server's cache_tools are ever replayed from the cache. There is no guessing from the tool name. A server with cache_enabled: true and no cache_tools caches nothing.

2. State epoch per server. Any call that is not cacheable (a possible write) advances that server's epoch. The epoch is part of the cache key, so a read cached before a write is never served after it. The epoch also carries a boot token: the cache does not survive a relay restart, because the server may have changed while the relay was down.

3. Errors are never cached. A result flagged is_error is not stored.

4. Schema-guided canonicalisation. A near-identical call is recognised only according to the tool's inputSchema: integer, number and boolean parameters, and path-like string parameters. Without a schema, only an exactly identical call matches. Never: case changes, whitespace folding inside strings, reordering of lists. Schemas are loaded when list_tools is called.

5. The loop guard counts cache hits. A loop answered from the cache is still a loop: the 5th identical call within the window is blocked.

6. A batch stops at the first error. The remaining calls are reported as skipped_after_error.

tool_proxy:
  servers:
    - name: fetch
      command: uvx
      args: ["mcp-server-fetch@2026.10.10"]
      cache_enabled: true
      ttl_seconds: 3600          # protects against changes made outside the relay
      cache_tools:
        fetch: true              # explicit allowlist: nothing else is cached

The TTL is the only protection against a downstream server whose state changes outside the relay. Choose it per server.

Loading tools fast

Two things make the relay quick to use, both verified by tools\proof_tools.py:

1. One live session per downstream server. The server process stays alive between calls instead of being respawned for every call. The session lives in its own thread and its own event loop, because the MCP SDK opens a stdio session inside an anyio task group that must be entered and left in the same task. Measured on a local server: the first call costs ~840 ms (process start), the next ones ~21 ms — 40× faster.

2. The tool catalogue is cached and searchable. proxy_info(what="tools", server=…) lists the tools of a server and keeps the result (with the inputSchema) in memory; proxy_info(what="search", query=…) then finds a tool by name or description — "git" finds git_log, "file" finds read_file. The catalogue is also loaded automatically on the first call to an unknown tool, because the schema is what makes the near-identical recognition possible.

.\.venv\Scripts\python.exe tools\proof_tools.py   # 14 checks: near-hit, no false hit, reuse, search

The schema-guided merge is deliberately conservative: limit: 10 and limit: "10" are one and the same call (the schema declares an integer), but " 8150 " and "8150" are not merged (a string is a value — the downstream tool answered "unknown port" for the padded one).

The console (dashboard)

A web console, served by the admin API, shows all metrics with matching charts:

http://127.0.0.1:8153/dashboard      (alias: http://127.0.0.1:8153/)

.\.venv\Scripts\python.exe -m icyloom dashboard          # URL + current state
.\.venv\Scripts\python.exe -m icyloom dashboard --serve  # serve the console

One console, two consoles fused

The console is the merge of the IcyCache console and the Loom dash, rebuilt on the unified registry v_token_ledger and drawn in the rough-sketch style (warm paper, brown ink, hand-drawn strokes, hatched fills, stickers, doodles).

It is an application shell, not a long scrollable page: a side panel on the left switches the view, and the content area only ever renders the view you are on.

The side panel carries the eight views (each with its own live counter), the system actions — Pause / Refresh / Export / Compact, which really act — and a live status card (calls, hit rate, tokens avoided, gain, ledger rows, database size). A toolbar above the content shows the current view, the LIVE state, the measured request rate and the event count; the control bar holds the analysis window (1 h / 24 h / 7 d / 14 d — it really changes the query), a Reconnect button and the live database pills.

Every button is wired to a real action: Pause freezes the figures (the polling stops, the pill turns PAUSED), Refresh and Reconnect re-query the admin API, Export downloads the exact JSON the page is drawing, Compact shrinks the layout for a laptop screen. Keyboard shortcuts: 1…8 switch view, R refreshes, P pauses, E exports, C compacts.

The eight views: Activity (3D mountain + topo map), Economy (curve, gauge, split), Cache (entries, ranking), Guide (path of a request). Context, Tools and Audit complete the set — see docs/screenshots/.

Everything is computed in real time; no value is frozen in the page:

• /api/dashboard is polled every 2 s; the server keeps the snapshot fresh in a background thread, so the answer is instant (≈ 10 ms) even on a busy base;

• requests/min is computed between two real samples and smoothed;

• each value is interpolated toward its new target, so counters scroll instead of jumping;

• the window and the pause button apply immediately;

• every timestamp is UTC — one single clock, from the SQLite defaults to the charts.

The data comes from a single read-only route:

GET /api/dashboard   ->  { kpi, config, registry, surface, series, breakdown, tables, context, recent, db }

registry is the unified v_token_ledger (cache + compression + tool tax, all in tokens), surface is the day × hour traffic grid behind the 3D mountain and the map, series holds the time series (per hour over 24 h, per day over 14 d). GET /api/gateway relays the gateway's live state (circuits, upstreams) from port 8150 so the page stays on a single origin.

The console is self-contained: one HTML file, hand-written SVG charts and a hand-written sketch engine (no rough.js, no framework), no external dependency, no CDN, no network call except /api/dashboard. The tokens shown are the real tokens reported by the upstream.

Reference screenshots (1440 × 960) live in docs/screenshots/: overview, activity, economy, cache, context, tools, audit, guide. The logo is src/icyloom/static/logo.svg — the same file is served by the console at http://127.0.0.1:8153/assets/logo.svg.

Demo data set (fake data, isolated stacks, ports 8150-8154):

.\.venv\Scripts\python.exe tools\seed_demo.py     # continuous traffic, about 4 req/s
.\.venv\Scripts\python.exe tools\check_js.py       # JavaScript syntax check of the console

The functional test campaign

Re-run on 2026-10-28 with the current code: 88/88 checks green (report in var\campaign-report.json).

A hermetic campaign that exercises every layer on an isolated stack (ports 8160-8164, temporary database), with entirely different inputs: 24 distinct prompts, 6 distinct documents, 3 distinct markers, 3 distinct tool calls:

.\.venv\Scripts\python.exe tools\campaign.py       # report + var\campaign-report.json

It covers the cache (variants, anti-collision, key scope), the ledger (real tokens, gain), the context drawer (byte-exact, invariants, discriminating search, isolated sessions), the tool relay, the gateway formats (OpenAI, Anthropic, SSE streaming), MCP HTTP/SSE parity, the safeguards (confirmation, concurrency) and the consistency of the console with the database. Output: one verdict per check and a non-zero exit code at the first failure.

Two blocks play the role of an agent doing real work:

• N, agentic work: the agent looks up its spec in the drawer, files a real document, asks 8 really different requests (plan, risks, test, talk, threat, rollback, measure, strategy), iterates one request ("v1" then "v2"), then calls 6 distinct downstream tools and closes with an invariants check.

• O, anti-mixing: interleaved requests and tools, purge, replay.

To prove that a hit returns the right answer, the campaign uses tools/echo_upstream.py: the upstream returns the fingerprint of the request it received (echo:<sha256>). An answer served from the cache therefore carries the fingerprint of its own request, never another one's.

The 13 tools

Every tool carries ToolAnnotations (read-only, destructive, idempotent, open-world). A destructive action without confirm=true is refused with no side effect.

Capability map: tools and routes

(Capabilities from spec section 5. ✔ = covered and tested · ◐ = partly covered · ✘ = not implemented, announced)

Cache and network

Context

The ledger

A single ledger_entries table records everything that costs and everything that saves: LLM calls (real upstream tokens), cache hits, context compression, tool calls. The 35 % threshold is readable there continuously:

tokens saved / (tokens paid + tokens saved)

The tokens of an LLM cache hit are the real tokens measured on the original call: they are never estimated. Compression tokens come from the local counter (tiktoken if available, otherwise an estimator), and the counter used is recorded in every entry.

Tool path: a tool-cache hit is ledgered with saved_latency_ms only; tool_tokens_saved is 0. The tool cache saves latency, not tokens.

Known limitations

• No measured token savings on tool calls. See the Status section.

• Compression quality is not proven. In a prototype run on a real model (carried over from earlier notes, details in PROGRESS.md), estimated prompt tokens dropped by about 84 to 92 percent, but the model answered incorrectly: the anchor keeps only a header, the invariants and the first 800 characters, so a key line in the middle of a log was lost. Compression therefore stays opt-in; "strict" guarantees that the invariants are kept, not that the text the model sees is unchanged.

• The gateway cache uses one slot per key shared by JSON and streaming responses, so a streamed request can miss a hit created by the JSON one.

• An upstream ReadTimeout during streaming is not caught in gateway.py (around line 578) and can end a client run.

• The tool-cache epoch is verified on a synthetic server; a real server with both reads and writes has not been tested yet.

• Downstream state that changes outside the relay is only covered by the TTL.

Design decisions

1. One MCP server, two HTTP processes. mcp>=1.2,<2 exposes one transport per instance: HTTP and SSE are therefore two instances built by the same function (build_server), which guarantees an identical tool surface. stdio stays a dedicated process.

2. Own environment variables. ICYLOOM_CONFIG, ICYLOOM_HOME, ICYLOOM_STATE_DIR. The historical names from other projects are not reused: the project is independent.

3. Ports 8150-8153 strictly reserved, bound to 127.0.0.1; tests run on 8154-8159.

4. One transaction per request. Cache, LLM journal and ledger are written in one transaction: three successive transactions cost about 20 ms per request, more than the gain from the cache.

5. Circuit breaker really re-armed: after reset_seconds the state becomes half-open and one probe is allowed.

6. Upstream failover: a failing upstream no longer permanently excludes the others; the priority is never mutated during a call.

7. Compression off by default, strict profile, never lossy implicitly.

8. Fail-closed sandbox: without an allowed directory, no command is run.

9. Tool cache: deny by default, with a per-server state epoch (see The tool cache).

10. No git repository for now: the licence is still undecided, so the project is not published. The safety net is a folder copy.

Cache hash compatibility

The normalisation engine follows the reference semantics (NFKD, lowercase, Aho-Corasick rules, number-words, diacritics, sentence punctuation). Operators are kept (12+2 is not 1+22), and volatile fields (stream, user, metadata) are excluded from the key.

Measured on the historical cache (read-only): 70 % of normalised texts coincide. The remaining 30 % were written by an earlier version of the engine that removed :, /, +, * and did not convert numbers to words. Those entries are already incompatible with the reference engine; honouring them would break compatibility with the current engine. tests/test_normalize.py measures this agreement and checks the equivalence between the fast engine and an independent naive implementation.

Tests

.\.venv\Scripts\python.exe -m pytest tests\ -q --ignore=tests\proof_e2e.py   # 102 tests
.\.venv\Scripts\python.exe tests\proof_e2e.py                              # end-to-end proof
.\.venv\Scripts\python.exe tests\test_cache_correctness.py                 # stale-hit measurement (prints a report)
.\.venv\Scripts\python.exe tests\real_mcp_fetch.py                         # real MCP server; needs uvx and Internet

• proof_e2e.py starts a fake upstream on 8154, launches the stack on 8150-8153, checks the 4 services, verifies the tool list over HTTP and SSE, then measures latency over 200 requests (direct / MISS / HIT) and the ledger gain. Its "50 %" gain is by construction (every request is sent twice).

• test_cache_correctness.py runs a stateful fake tool server under interleaved reads and writes and compares every cached answer with the server's true state. Run it as a script to get the numbers, or through pytest for strict thresholds (zero stale hits).

• real_mcp_fetch.py drives the real mcp-server-fetch through the relay: schema capture, MISS then HIT, integer canonicalisation, no merge across different URLs, a real error never replayed.

• Tests use a temporary database and never touch var\icyloom.db. Set ICYLOOM_TEST_PORT to run the suite while a stack occupies 8154.

Layout

D:\ICY-LOOM\
├── icyloom.yaml              single configuration
├── pyproject.toml            package (requires-python >=3.11,<3.14)
├── LICENSE                   MIT
├── README.md
├── PROGRESS.md               what is proven, what is not, what is next
├── MERGE_PLAN.md             the merge plan (IcyCache + Loom -> IcyLoom)
├── RESEARCH.md               the research notes behind the choices
├── ICYLOOM_PROMPT.md         the original brief
├── docs\
│   ├── logo.png              the mark (source: src\icyloom\static\logo.svg)
│   └── screenshots\          the eight console views
├── data\
│   ├── normalization_rules.json   1122 rules (data)
│   └── tools_registry.json        100 anchors (data)
├── src\icyloom\
│   ├── cli.py            doctor / paths / version / config / serve / ...
│   ├── config.py         Pydantic models of the single configuration
│   ├── paths.py          path resolution (no hard-coded path)
│   ├── logsetup.py       logs on stderr, UTF-8 forced
│   ├── db.py             single database: schema, WAL, additive migrations
│   ├── normalize.py      normalisation + cache key
│   ├── cache.py          LLM cache (TTL, LRU)
│   ├── upstream.py       multi-upstream routing
│   ├── pool.py           HTTP pool: drain -> recycle -> retry
│   ├── breaker.py        circuit breaker with real re-arming
│   ├── gateway.py        OpenAI/Anthropic HTTP gateway
│   ├── admin.py          admin API + metrics + console + /assets/logo.svg
│   ├── dashboard.py      metrics aggregate (read-only snapshot)
│   ├── static\           web console (HTML + SVG, self-contained) + logo.svg
│   ├── mcpserver.py      single FastMCP instance + 13 tools
│   ├── toolproxy.py      tool relay + cache (deny-by-default, epoch) + loop guard
│   ├── ledger.py         single economic journal
│   ├── tokenizer.py      local token counter (tiktoken or estimator)
│   ├── migrate.py        additive migration (sources read-only)
│   ├── runner.py         combined launcher
│   └── context\          drawer, anchors, invariants, verification, sandbox...
├── tools\
│   ├── seed_demo.py      demo data set (real traffic, isolated stack)
│   ├── spread_demo.py    deterministic time-spreading of the demo events
│   ├── campaign.py       hermetic functional campaign (88 checks)
│   ├── check_js.py       JavaScript syntax check of the console
│   ├── fake_mcp_server.py  fake downstream MCP server (relay target)
│   ├── run_fake_upstream.py fake OpenAI/Anthropic upstream
│   └── proof_tools.py, report_live.py, report_tools.py, diag_norm.py, ...
└── tests\                 test suite, end-to-end proof, correctness harness, real-server test

Project map

Everything points at everything else:

The console is the living version of this README: it reads the same v_token_ledger the ledger section describes, through the same GET /api/dashboard route.

License

MIT — see LICENSE.

Copyright (c) 2026 Malik Belkacemi.