thoth-mem

Persistent memory your coding agents can share — and you can audit.

Local, SQLite-first memory for OpenCode, Codex, Claude Code, and Pi. Six focused MCP tools. Zero model calls required.

Overview • Install • How it works • Use • Tools • Benchmarks • Runtime • Troubleshooting • Development

Why thoth-mem

Coding agents lose the decisions that matter between sessions: why an approach was chosen, which failure already occurred, what the next safe action is, and which evidence supports the current answer. Static instruction files help with rules, but they do not provide temporal history, scoped retrieval, or attributable provenance.

thoth-mem gives every supported harness one durable local memory without turning memory into an opaque second agent.

Related MCP server: LumenCore

Install

Requirements: Node.js >=22.12.0 and a supported harness (OpenCode stable >=2.2.6). Pi setup has no version allowlist or upper version limit: it checks package-manager capabilities and verifies the installed extension and resources. New Pi releases do not require a version override. The reproducible SDK test baseline is @earendil-works/pi-coding-agent 1.0.1; passing setup is not certification of every runtime behavior on future releases.

thoth-mem installs memory tooling and lifecycle integration—not agents or subagents. Add --plan --json to any managed npx setup command to preview its changes without writing.

How it works

flowchart LR
  subgraph Hosts[Native harnesses]
    O[OpenCode]
    C[Codex]
    A[Claude Code]
    P[Pi]
  end

  O & C & A & P --> H[Lifecycle adapters]
  H --> S[MemoryService]
  S --> L[(Immutable SQLite ledger)]
  S --> F[(Rebuildable FTS5 index)]
  L & F --> R[Bounded progressive context]
  R --> O & C & A & P

1. Native adapters map each harness into the same project and root-session contract.

2. Evidence is immutable; session events receive a database-ordered sequence.

3. Observations stay outside durable memory until a verified review accepts them and an explicit promotion materializes the proposed memory.

4. Recall combines project isolation, temporal truth, lexical ranking, and a strict character budget.

5. Every harness receives the same bounded context without introducing another model into the loop.

Pi delivers recovery once as a persisted hidden message on the first root prompt after startup, resume, new, fork, or successful compaction. Reload skips delivery when the active branch already has recovery after its latest compaction. OpenCode V2 writes the keyed thoth-mem-recovery session instruction entry before the first root request, with durable delivery state preventing rewrites on restart, resume, or compaction. Prompt text and frozen request baselines are unchanged. If memory is unavailable, both hosts deliver locally verified identity first and retry on later root prompts until one verified recovery block is delivered; OpenCode's one replacement becomes an appended instruction update.

OpenCode setup writes an exact-version entry under plugins (or an absolute package-root directory for explicit local setup), and registers the packaged Skill without copying it. Since OpenCode skips install scripts, its thin plugin uses literal Node for validated local runtimes or exact-version npx when the SQLite binding is absent.

Automatic capture is limited to verified root prompts and checkpoint evidence, including available host compaction text. It excludes assistant reasoning, arbitrary tool traffic, delegated/subagent output, filesystem sweeps, and full transcripts. Private blocks and recognizable credentials are filtered before content-dependent keys or persistence; do not submit secrets.

Use the memory

Optional agent instructions

Optionally paste this snippet into your AGENTS.md or CLAUDE.md for proactive memory use. thoth-mem never writes it or edits those files.

## Memory (thoth-mem)
- Before exploring or planning any non-trivial task, run `mem_recall` with the task's key terms. Don't wait to be asked.
- When a decision is accepted, a root cause is verified, an approach fails, or a non-obvious convention is found, save it with `mem_save` right away.
- If a recovered handoff's pending action is done, close it: save the outcome under the same `topic_key`.
- Before ending meaningful unfinished work, save one handoff (`topic_key: handoff/<workstream>`).
- Skip memory for trivial or purely conversational turns.

Check adoption

Run thoth-mem adoption --project <exact-key-or-alias> [--json] [--data-dir <dir>] against an existing database for a read-only baseline. It never creates or migrates a database. It reports open handoffs with ages in seconds, replaced handoffs (superseded by another handoff), and closed handoffs (superseded by a non-handoff under the same topic key). Session totals include zero-save sessions and per-session counts of distinct agent explicit_save/handoff evidence IDs; automatic captures and unattributed evidence are excluded. Recall usage is not measured.

Retrieve progressively

mem_recall mode=compact
        ↓
mem_recall mode=context  or  mem_context
        ↓
mem_get only for selected stable IDs

Choose tools by intent

The MCP server exposes exactly these six tools. Pi and OpenCode expose every tool except mem_session as a native model-facing tool; mem_session is integration-owned and called only by the lifecycle adapter through its MCP child. OpenCode additionally exposes the read-only native thoth_mem_root_identity tool; it is session metadata, not a memory operation. Generic MCP connections attribute sessionless saves to a server-assigned connection session, not a verified native root session.

Tool text is valid JSON capped at 20,000 characters and matches structuredContent. Oversized results clip long strings, then drop trailing array items if needed, with payload_truncated first in warnings; irreducible results return output_too_large. Narrow requests when needed. Pi never slices tool text: unexpected over-cap text becomes a thoth-mem.pi.output_truncated JSON notice with narrowing guidance.

In Pi, memory tool cards render through the Thoth render kit when one is available, honoring the pi-thoth-theme tool density and its expanded/collapsed layouts. Without a compatible kit, Pi keeps the built-in frames; no extra dependency is required.

• Use mem_project with action=timeline when you need to understand how promoted knowledge changed, rather than which memories best match a query.

• mem_get returns the selected record once; lineage is empty unless history: true, then contains only predecessors, newest first. Evidence has no lineage; mem_project action=history still starts with the selected record.

• At checkpoint_pre_compact or finalize, the core validates a submitted summary or deterministically derives or reuses one from the session's evidence, without model or network calls. Pi, Claude Code, and OpenCode also forward available host compaction text as checkpoint evidence without requiring prior pre-compaction; recognized headings yield a cited harness summary, never promoted memory.

• Derived summaries omit a compaction's objective, next-action, pending, and blocker claims after a later substantive root prompt (at least 40 code points after whitespace collapse) or support for a memory-derived claim. Decisions, completed work, and changed surfaces remain eligible; persisted summaries at their activity watermark are not regenerated.

• Sessions have no lifecycle state and always accept new activity. All summaries share one versioned lineage per session: a new summary supersedes the current one when its coverage end is at least the current one's; a lower end is rejected as late. Derivation reuses a current summary that already covers the latest activity.

• Pi uses per-run lifecycle keys so quitting after resume derives a summary covering new activity at exit. Codex and Claude Code may replay an earlier finalize receipt on resume; lazy derivation at the next recovery covers new activity when that session is selected as the previous session. See native lifecycle for this known limitation.

• Automatic recovery, including after compaction, shows the newest open handoff, then the previous session's summary, then other project memories within the existing caps. It never shows the current session's own summary; explicit session reads still can.

• Observation candidates remain outside memory and FTS until a verified root review accepts them and a separate explicit promotion materializes their exact proposed memory.

LongMemEval-S

The public results use the immutable cleaned LongMemEval-S corpus and its 470 eligible non-abstention questions. The runtime remains lexical and local: no embeddings, models, or evaluation-time network calls.

• RecallAny@5: questions with at least one gold session in the first five results.

• Recall@5: fractional coverage across all gold sessions.

• RecallAll@5: questions whose every gold session appears in the first five.

• NDCG@10 / MRR: ranking quality and first-gold position.

• Retrieval p95: environment-sensitive; compare latency only within the same report.

The first four rows come from the immutable Top-5 lexical comparison. The current-default row comes from the passing stable optimization round, which preserved complete ordered output while reducing p95 by 28.3% from its stable baseline. Every listed run records zero errors and zero model, LLM, or evaluation-time network calls.

The dataset is pinned to revision 98d7416c24c778c2fee6e6f3006e7a073259d48f and SHA-256 d6f21ea9d60a0d56f34a05b609c79c88a451d2ae03597821ea3d5a9678c3a442. Evaluation runs offline through the real built MemoryService, with one isolated SQLite database per question. Gold IDs and oracle data never enter indexed text or ranking.

Protocol sources: LongMemEval repository, official cleaned dataset, and pinned dataset revision.

Runtime data and migration

All harnesses resolve one data directory in this order: an explicit command value, THOTH_MEM_DATA_DIR, strict provider configuration, then ~/.thoth-mem. The database is always memory.sqlite inside the selected directory.

Git repositories keep a stable project ID in the shared Git common directory: worktrees share memory, while independent clones remain separate. Folder renames do not change the display name; use thoth-mem project rename --project <exact-key-or-alias> --name <display-name> [--data-dir <dir>] to change it explicitly.

The provider file lives below XDG_CONFIG_HOME/thoth-mem/config.json when XDG configuration is set, or below ~/.config/thoth-mem/config.json otherwise. Malformed, unreadable, schema-invalid, or missing-runtime configuration fails closed.

Hosts automatically migrate supported older schemas to revision 12 at startup; thoth-mem migrate completes pending migrations manually and reports the changes. On large databases, backups and migration can exceed host timeouts; use that manual path instead. Revision 12 creates or verifies memory.sqlite.pre-v12.bak, rechecks the revision and live source under an immediate write-excluding lock, removes session state and end-time columns, and merges existing summaries into one version lineage per session while keeping IDs. Integrity and foreign-key checks must pass; a mismatched backup or source drift fails closed before mutation.

Older databases first apply the revision-10 Pi harness migration (memory.sqlite.pre-v10.bak) and revision-11 empty-session cleanup (memory.sqlite.pre-v11.bak) as needed. That cleanup removes only unreferenced non-import host sessions and their lifecycle receipts that cite neither evidence nor a summary; all other sessions, receipts, evidence, and memories are preserved.

Stop every process that may hold the target database, then import the conventional ~/.thoth/thoth.db:

thoth-mem import-legacy

For a nonstandard source or explicit mapping:

thoth-mem import-legacy --source ./legacy.sqlite --map ./mapping.json --data-dir ./current-memory
thoth-mem import-legacy --json

For a reviewable two-step workflow, write a plan, inspect it, then apply it:

thoth-mem import-legacy plan --source ./legacy.sqlite --target ./current-memory/memory.sqlite --plan ./import-plan.json
thoth-mem import-legacy apply --plan ./import-plan.json --report ./import-report.json

Add --map ./mapping.json to plan for explicit mapping. apply uses the target recorded in the plan; existing plan and report output files are refused.

The importer fingerprints its inputs, creates a verified backup and isolated candidate when needed, and publishes only after integrity checks pass. Keep the legacy database, verified backup, and recovery bundle until the migrated runtime has been independently validated.

Troubleshooting

If memory stays unavailable after an upgrade, look for these host diagnostics:

• Pi: thoth-mem degraded: pi_mcp_startup_timeout — the memory server did not start in time; if memory stays unavailable, run `thoth-mem migrate` (at most once per session). Other failures retain codes such as pi_lifecycle_request_failed.

• OpenCode logs: [thoth-mem] OpenCode bridge degraded: <code>, where <code> is opencode_mcp_startup_timeout, opencode_mcp_request_failed, or lifecycle_request_failed_or_timed_out. A timeout alone does not mean a migration is pending; if memory stays unavailable, run thoth-mem migrate.

• Codex / Claude Code hook stderr: thoth-mem public plugin: Lifecycle failed: SQLite migration revision A→B failed: …, including the observed failure and the thoth-mem migrate recommendation.

A timeout alone does not prove a migration is pending. To complete any pending migrations outside host time limits, run:

npx --yes thoth-mem@latest migrate

With an installed command, use thoth-mem migrate; with a package checkout, use node <package>/dist/index.js migrate. Add --data-dir <dir> to select the directory containing your existing memory.sqlite; otherwise the runtime data selection above applies. A missing database is an error, not a new database.

Output such as Migration: 11 → 12 shows the revision change; Backup created: or Backup verified: names the retained backup. Sessions removed: and Lifecycle receipts removed: count only the empty-session cleanup, not evidence or memories; Duration: is elapsed milliseconds. Add --json for the thoth-mem.migrate.v1 envelope with revisions, backups, removal counts, durationMs, and alreadyCurrent. It is safe to rerun: an already-current database reports a no-op (alreadyCurrent: true) without changing the source database or its sidecars.

Keep memory.sqlite.pre-vN.bak backups until recovery is verified. To restore one, first stop all hosts and thoth-mem processes, move the current memory.sqlite and any memory.sqlite-wal / memory.sqlite-shm files aside, then copy the chosen backup to memory.sqlite. Restoring loses changes made after that backup; an older schema will migrate again on the next writable open. Prefer a clean host exit (in Pi, /quit) over killing its terminal.

Killed processes may leave memory.sqlite.pre-vN.bak.<pid>.<uuid>.tmp files. They are unpublished backup candidates, not retained backups; delete them only when no thoth-mem process is running.

Development

Clone, build, local host wiring, verification, benchmark reproduction, and repository layout live in the development guide.

Task-specific engineering, persistence, privacy, lifecycle, and testing guidance starts at the agent context index.