KOPENG
by djy89
README.md
# KOPENG
[](https://github.com/djy89/kopeng/actions/workflows/ci.yml)    
**Memory that curates itself instead of growing into landfill.**
KOPENG is persistent, self-curating, local-first memory that you own — built first for AI coding agents (Claude Code, Codex CLI) and usable by any MCP-compatible assistant. It learns from how you actually work, recalls the right context on every prompt, and — unlike append-only RAG — cleans up after itself: duplicates collapse, stale facts decay, contradictions get routed to you for review. Automated archives and merges (dreaming, promotion, maintenance, curation) are snapshot-first and audited; memories a dream creates are audited on creation and undone by archival; operator edits through `PUT /api/memories/:id` are revision-snapshotted and reversible; direct archive/unarchive (admin-gated) creates neither a revision nor an audit row. All inference is 100% local: no per-query API cost, and your codebase context never leaves the box.
## Install
One prerequisite: **Node.js 20+**. That's it — no Docker, no database server, no cloud account, no API keys, no admin rights.
```bash
npx @djy89/kopeng@latest init
```
On **Windows**, also install the [Visual C++ Redistributable](https://aka.ms/vs/17/release/vc_redist.x64.exe) first — the local embedding runtime is a native module that will not load without it (KOPENG degrades to keyword-only search rather than failing, but you want the semantic half).
The installer is dry-run-first: it shows you exactly what it will do, then waits for your yes. On your next Claude Code prompt, recall just fires.
**What it puts on your machine** (and what `npx @djy89/kopeng uninstall` removes):
| Item | Where |
| --- | --- |
| KOPENG server + CLI (prebuilt — nothing compiles) | `~/.kopeng/app` |
| Your memory data: SQLite database(s), logs, and `.env` config with a generated admin key | `~/.kopeng/data`, `~/.kopeng/logs`, `~/.kopeng/.env` |
| Embedding model (all-MiniLM-L6-v2, ~30 MB) and Reranker model (ms-marco-MiniLM-L-6-v2, downloaded on first search) | `~/.kopeng/models` |
| A user-level autostart entry (Scheduled Task / systemd --user unit / LaunchAgent) | your user session, recorded in `~/.kopeng/autostart.json` |
| MCP registration + 5 fail-open Claude Code hooks, merged minimally into your existing config (backed up first) | `~/.claude.json`, `~/.claude/settings.json` |
| Learning-profile flags (learn from your tools, suggest memories, dream nightly — on by default; the installer explains each and `--profile minimal` turns them off) | `~/.kopeng/.env` |
| License acceptance record (which license text you accepted, when, and how) | `~/.kopeng/license-acceptance.json` |
`npx @djy89/kopeng uninstall` reverses all of it and keeps your memory data unless you pass `--purge`. Nothing phones home — there is no telemetry to opt out of.
Prefer to run from source, wire configs by hand, or install as a real system service? The clone-and-`npm run wire` path lives in **[SETUP.md](SETUP.md)**, along with Codex CLI wiring and troubleshooting.
## Watch it think
```bash
npx @djy89/kopeng viz
```
<!-- SCREENSHOT/GIF HERE: live tab + graph tab while a Claude Code session runs -->
KOPENG ships a six-tab dashboard that makes the whole system observable while you work: a **live feed** of tool-use observations streaming in real time, a **memory graph** you can traverse, **ops panels** (confidence distribution, decay, dream history, corpus health), the **dream review queue** where you accept or reject the Librarian's proposed consolidations, **session replay**, and **slots**. This is the difference between "trust me, it's learning" and watching it learn. Runs at `http://localhost:8780` in the foreground — Ctrl-C stops it.
## Why KOPENG
Most "agent memory" is append-only store-and-retrieve. It remembers; it never prunes. The corpus drifts — duplicates pile up, stale facts outrank current ones, contradictory memories ("we use X" / "we switched to Y") both keep surfacing — and the operator becomes the garbage collector. KOPENG adds the missing half: **curation.**
**It curates, not just recalls — the dreaming Librarian.** An operator-gated nightly consolidation pass collapses duplicates, decays stale memories, and routes contradictions and supersessions for your review. The engine is deterministic-first: every mutation is deterministic code; the optional local LLM only _classifies_ pairs and is structurally locked out of the write path. Supersession is a temporal chain, not a deletion — the history stays.
**It learns passively from real tool use, at zero LLM cost.** Six heuristic detectors turn observed behavior — repeated commands, error-then-fix patterns, hot files, cross-session sequences — into confidence-scored memories. You never have to remember to save anything.
**Automated curation is audited and reversible.** Every archive, merge, supersession, contradiction marking, and crystallization that dreaming, promotion, discovery maintenance, or scope curation applies, and discovery maintenance's promotion of a cross-scope duplicate to `global`, runs snapshot-first: snapshot to `memory_revisions` → mutate → append a `dream_audit_log` row, and if the audit append fails the mutation is compensated (undone) rather than left standing unaudited. Memories a dream creates have no prior state to snapshot; they are audited on creation and undone by archival. Operator edits through `PUT /api/memories/:id` are revision-snapshotted but not audited — the revision alone is their reversibility record; alias re-scopes (`npm run migrate:scope-aliases`) write through that same route, so they too are revision-snapshotted only. Direct archive/unarchive (`PATCH /api/memories/:id`, `POST /api/memories/:id/archive`, `DELETE /api/slots/:slot_key`) and slot edits write neither a revision nor an audit row; archive and unarchive undo each other. Roll back any memory to any of its revisions via one endpoint (`POST /api/memories/:id/rollback`), which snapshots the current state first, so a rollback is itself reversible. The revision is a whole-row snapshot — content, embedding, tags, confidence, scope, type, the decay clock, and the `is_locked` Hard Anchor — so a rollback reverts protection state along with everything else, except the decay clock: a rollback refreshes it (and unarchives the row) so the restored memory is not immediately re-archived. Scope of the claim, honestly: what is reversible is *memory* state, and three automated writes land in place with no revision and no audit row — discovery's reinforcement of a re-observed memory (confidence bump, decay-clock reset, evidence history), the dream engine's clearing of a routed contradiction flag, and reinforcement-on-access, where every memory surfaced by recall, search, or `GET /api/memories/:id` gets its observation count +1 and its decay clock reset. Deletes are archives, never row removal — except the two admin revision-purge routes, which are the deliberate redaction escape hatch and do destroy history.
**Fully local.** Embeddings and reranking run as quantized ONNX in-process; the optional reasoner is Ollama on your own GPU. No cloud model sits in the retrieval path or the consolidation path.
## How it works
|Layer|What it does|
|---|---|
|**Retrieval**|Semantic search with a keyword (FTS5) lane and an optional cross-encoder rerank. Results are ordered by relevance only: we tested a confidence-weighted ordering against planted ground truth and removed it because it lost ([`docs/proof/rp5-confidence-policy.md`](docs/proof/rp5-confidence-policy.md)). Confidence still filters (`min_confidence`) and decay still drives archival; neither reorders results. The keyword lane is RRF-fused on the hook recall path; on `/search` it fires only when every query word matches, so natural-language searches are effectively semantic-only today ([`docs/proof/ranking-confidence-proof.md`](docs/proof/ranking-confidence-proof.md)). The fast hook recall path skips reranking.|
|**Dreaming / consolidation**|Deterministic-first nightly pass: duplicate collapse, durability-scaled decay, contradiction routing, supersession chains. Snapshot-first, audited, reversible. Off by default.|
|**Auto-discovery**|Tool-use observations → 6 zero-cost detectors → confidence scoring → semantic dedup → memories.|
|**Static surfacing**|Per-prompt injection of relevant tools, skills, and project conventions from your `~/.claude` indexes.|
|**Observability**|Live SSE event stream + the six-tab dashboard above.|
|**Optional reasoner**|Local Ollama pair classifier — classify/extract only, never writes. Absent or down, behavior degrades byte-for-byte to deterministic-only.|
|**Storage**|SQLite (supported preview backend) + in-process ONNX models. Optional, feature-flagged: Neo4j, Redis, MinIO.|
|**Interfaces**|19 MCP tools (thin stdio client) + Fastify REST API on `127.0.0.1:3200`.|
Two entry points, one backend: `src/server.ts` is the real server (owns the database, index, and services); `src/index.ts` is the MCP stdio client that proxies tool calls to it. The REST contract never changes across backend swaps.
Memory types: `user`, `feedback`, `project`, `reference`, `discovery`. Scopes: `global`, `project:<name>`, `client:<name>`, with a `PRIMARY_SCOPE` routing rule so nothing silently lands in `global`.
Three commands answer the three operational questions, any time: `wire` (is it connected?), `doctor` (is the whole install correct?), `canary` (does recall actually work, end to end?).
## Preview status
KOPENG is a **developer preview**: a self-hosted memory system for a single expert developer, on one machine, bound to loopback (`127.0.0.1:3200`).
- **Learning is on by default, auto-apply is not.** The recommended profile turns on passive learning and nightly dreaming; the installer (`init`) and `wire` explain each setting and offer `minimal` to turn them off. The reasoner and every `auto_accept_*` control stay OFF — dream output queues for your review.
- **Auto-apply is hard-restricted in code** to exactly two change classes (exact duplicates and decay) — both also OFF by default. Everything else queues for operator review.
- **Reliability comes from engineering rigor, not scale:** a zero-LLM pinned-clock replay regression net, adversarial reviews run against a copy of real data, idempotent consolidation passes, and fail-open behavior everywhere a hook or service could stall. The full suite (`npm test`) runs against in-memory SQLite with no server needed.
- **Security posture, in one paragraph:** operator mutations require a generated admin key; observation ingestion uses its own separate optional key; reads are public by design on loopback. **Remote deployment is unsupported** — a non-loopback bind refuses to boot without both keys, and even then requires an outer boundary (VPN or authenticating reverse proxy). Full threat model: [SECURITY.md](SECURITY.md).
> Renamed 2026-07 from its previous codename. Everything uses `kopeng` — hook env vars are `KOPENG_*`, client data lives in `~/.kopeng/`.
## Evals
Retrieval quality is measurable, not vibes: `npm run eval` reports P@K, R@K, MRR, and NDCG@K, with baseline-vs-reranked comparisons, and the consolidation layer has its own zero-LLM regression and effectiveness harnesses (`npm run dream:replay`, `npm run dream:effectiveness`). One deliberate exception to local-only, clearly flagged: `npm run eval:seed` sends selected memory content to the Anthropic API to draft eval queries — the only shipped command that egresses corpus data. The runtime itself never does.
## License
KOPENG is **source-available** under the [KOPENG Personal Use License](LICENSE): free for **individuals** — personal, hobby, and educational use, and your own independent work as a freelancer or solo consultant. Any use by or inside a business or other organisation, or by more than one person, including internal use, needs a commercial license. Organisations get one free 30-day evaluation window (non-production, one installation per evaluator); after that, get in touch via [github.com/djy89](https://github.com/djy89).
## Contact
Built by **djy89** — a single-maintainer project.
- More of my work: [djy89.net](https://djy89.net/)
- Bugs, questions, design discussion: open a [GitHub issue](https://github.com/djy89/kopeng/issues) — public discussion preferred; it helps the next person.
- Anything else: `hello@kopeng.net`
- Security vulnerabilities: use GitHub's private vulnerability reporting per [SECURITY.md](SECURITY.md) — not issues, not email.
This server cannot be deployed
Maintenance
ActivityMaintained
ResponsivenessNo issues