Loom

⚠️ Note: The rest of this repository (code, comments, scripts, docs, commits, issues) is written in French. Only this README is in English.

⚠️ WORK IN PROGRESS — NOT READY FOR PRODUCTION (updated 2026-10-26)

Loom is under repair and being re-measured. Do not use in production.

• No token savings have been demonstrated to date. The old figures (saved_pct 94–99%, bench-report.json, bench_results.json, REPORT_FINAL.md) measured the anchor size alone: they are invalid.

• Current real measurement (LM Studio, small local models, small documents): Loom costs more tokens than the baseline (fixed anchor overhead + tool schemas). The large-context test with a cloud model has not been done yet.

• Decision criterion frozen before the bench: same success rate with ≥ 35% fewer total tokens (in+out), retrieval and tool tax included. Otherwise: pivot or stop.

• Code state (measured 2026-10-25, see PROGRESS.md): 83 passed / 19 failed / 1 collection error across 103 test functions in 20 files. Most failures come from a code refactor that was not followed by a test update, plus 4 real anchor regressions (port kind, rare kinds, dates). verify, redact, audit and sandbox exist and their tests pass.

• The server exposes 17 MCP tools — this section used to list 13. The tool table below is now the measured list.

"Weave the context without cutting the thread." A portable MCP server that keeps the LLM context small by moving it out of the prompt — never by summarizing it.

Related MCP server: context-hub-mcp

Dashboard

Dashboard in --demo mode (python dash/server.py --demo, port set by LOOM_DASH_PORT, default 8775). The figures on screen (for example "85% saved") are synthetic demo data, not measurements — see the work-in-progress notice above.

Installation

The registry publication is pending. The repository, the PyPI distribution and the npm package all carry the same name — mcp-loom-mcp — and the packages are built, tagged and ready to publish (v0.1.3). The names mcp-loom and loom-mcp belong to other projects on those registries — on PyPI a server for Loom's internal GraphQL API, on npm an unrelated DuckDB proxy — and a registry never releases a name. That is why the repository was renamed from Mcp-Loom (the old URL redirects automatically).

The registry commands below are the final ones, but they return a 404 until the first publish. Install from source in the meantime: pip install "mcp-loom-mcp @ git+https://github.com/MlkBlk-IT/mcp-loom-mcp.git".

# PyPI
uv tool install mcp-loom-mcp          # or: pip install mcp-loom-mcp

# npm
npx -y mcp-loom-mcp

The package installs the loom-mcp console script (the server MCP clients spawn) and the loom admin CLI. Point your MCP client at loom-mcp:

{
  "mcpServers": {
    "loom": {
      "command": "loom-mcp"
    }
  }
}

From source

pip install "mcp-loom-mcp @ git+https://github.com/MlkBlk-IT/mcp-loom-mcp.git"

From source, with the dev extras

git clone https://github.com/MlkBlk-IT/mcp-loom-mcp.git
cd mcp-loom-mcp
python -m venv .venv
.\.venv\Scripts\Activate.ps1        # Windows
pip install -e ".[dev]"

Admin CLI

The same operations are available on the command line, built on top of the MCP tools:

loom health      # drawer health status
loom stats       # token stats for a session
loom vacuum      # compact the SQLite drawer
loom search      # FTS5 search over stored chunks

Transports

The HTTP transport (streamable-http) and token authentication are not yet tested end to end.

Building and publishing this package: see PUBLISHING.md.

The problem

An LLM agent burns its tokens on bulky context: raw logs, 100 files read, tool outputs. Loom takes that volume out of the prompt and files it in a local drawer. The model keeps a short, dense anchor plus stable IDs. When it needs the detail, it asks Loom back — byte for byte.

Loom moves nothing else. That is the whole thesis of the project.

Current state

v0.1.1 — working core, honest test state. 17 MCP tools exposed, a reproducible benchmark on a 21-file corpus, and a test suite that is not green: 83 passed / 19 failed / 1 collection error (measured 2026-10-25 — PROGRESS.md). Phase 2 (HTTP transport, rate-limiting and audit wired to the last 5 tools) in progress.

Architecture

LLM context
     │
     │  ctx_store / ctx_store_file / ctx_store_cmd
     ▼
┌──────────────────────────────────────────┐
│  ANCHORING      anchors.py + compress.py │  ← extracted invariants, condensed anchor
│  REDUCTION      profile.py               │  ← strict / balanced / aggressive
│  GUARANTEE      verify.py                │  ← safety net: nothing disappears
│  ENCRYPTION     crypto.py                │  ← Fernet (column) / SQLCipher (DB)
│  REDACTION      redact.py                │  ← secrets masked before storage
│  TRUST          sandbox.py + ratelimit.py│  ← isolated execution + rate limiting
│  TRACEABILITY   audit.py                 │  ← structured JSONL logs
└──────────────────────────────────────────┘
     │
     │  SQLite + FTS5 (store.py)
     ▼
   LOCAL DRAWER  (~/.loom/loom.db)

Modules

Exposed MCP tools (17)

Administration operations (purge, vacuum, health, list_sessions) are also available as CLI: python -m loom <cmd>.

Profiles

Metrics — and what they actually mean

This is the most important part of the project: badly named metrics make a benchmark lie. Here is the exact meaning of each measure.

Proven feasibility — measured benchmark

Corpus: 21 real files (6.87 MB — code review, config, incident, doc, analysis, session, migration) × 3 sizes × 3 profiles = 63 measurements, 1,405,483 tokens processed per profile. Tokenizer tiktoken:cl100k_base. Corpus reproducible via scripts/make_dataset.py.

Averages per profile (21 files)

retrievable = 1.0 and critical_missing_retrievable = 0 across all 63 measurements.

Breakdown by size

Reading: latency is dominated by invariant extraction (regex), not by anchor size — hence near-identical latencies across profiles on the same file. On a 1.2 M-token file, the strict anchor drops to ~25% visible coverage but 100% of the content stays retrievable: exactly the intended trade-off.

Reproduce

.venv/Scripts/python.exe bench_dataset.py   # → bench_results.json (63 measurements)

Proven artifacts

Test suite

The suite is not green. Measured on 2026-10-25 — full breakdown in PROGRESS.md:

$ .venv/Scripts/python.exe -m pytest -q --ignore=tests/test_allowed_dirs.py
103 test functions in 20 files  ->  83 passed, 19 failed

• 1 collection error — tests/test_allowed_dirs.py imports loom.server._path_allowed, a helper that was replaced by an inline LOOM_ALLOWED_DIRS check. The test is stale, the code is newer.

• 19 failures — 15 come from a code refactor that was not followed by a test update (renamed APIs, removed kwargs, the tool list growing from 13 to 17), and 4 are real anchor regressions (port kind no longer produced, rare kinds no longer reaching the strict anchor, dates still extracted as numbers).

• verify, redact, audit and sandbox exist and their tests pass.

The safety-net subset (green)

3 real bugs found by the tests (they would never have surfaced otherwise):

1. FastMCP 4.x no longer accepts readOnlyHint= as a kwarg → worked around with annotations=ToolAnnotations(...).

2. chunk_id typed int while MCP clients send strings → tolerant str | int signature.

3. Chunk splitting was losing bytes (strip("\n")) → broke the "byte-identical" promise. Fixed, with safety net + parameterized test cases.

Configuration (environment variables)

Known limitations

• ctx_store_cmd is not sandboxed on every platform: it refuses to run if no sandbox is available (fail-closed).

• Redaction breaks byte-identity of masked zones — an accepted trade-off, reported in the report.

• No full database encryption on Windows: SQLCipher has no prebuilt wheel (C build failure) → fallback to column-level Fernet, with the FTS5 index remaining in clear for search.

• chmod 600 is ignored on NTFS → protect the drawer directory manually.

• HTTP transport (streamable-http) and token authentication are not yet tested end to end.

• Rate-limiting and audit are wired to 8 of the 17 tools; 9 remain to be wired.

Methodology

• Safety net first: the invariant extractor and verifier were written before the compression logic. If the verifier cannot prove something survived, the tool refuses to compress.

• Deterministic extraction: regex and heuristics only, never an LLM. A verifier that depends on an LLM cannot prove coverage.

• Honest measurement: every report gives before/after tokens counted by a real tokenizer, never estimated.

• Audit before action: reproducible benchmark, versioned corpus, per-session report in docs/.

• Portable by construction: SQLite + FTS5, no proprietary SDK, no paid API.

What this project demonstrates

1. Moving ≠ summarizing — lossy compression is a choice, not a technical necessity. Loom proves it with retrievable = 1.0 across 63 measurements.

2. The token budget is an architecture problem, not a prompt problem — the right answer is an indexed local drawer, not a shorter prompt.

3. Honest metrics do more work than flattering metrics — naming coverage "visible in the anchor" instead of "covered" prevents a benchmark that lies.

4. An AI project can be 100% CPU and 100% local — Python + SQLite, no GPU, no paid API.

Author

Malik — IT Support student (DEP), self-taught Focus: Sysadmin + Cyber + AI + Cloud

Ownership and license

Project origin and evidence · License

⚡ Built on CPU. Proven on disk. ⚡

Original project and documentation by Malik Belkacemi · MLKBLK-IT · © 2026