knowledge-memory

English | 日本語

Local-first, source-grounded memory infrastructure for AI agents.

knowledge-memory gives AI agents a long-term memory without losing the source, time, lifecycle state and correction history of what it stores. Human approval is the boundary between proposed and accepted knowledge, and agents receive scope- and state-filtered search results and contexts over MCP, with explicit limits.

Markdown files are the source of truth. The SQLite search index can be rebuilt from them. The default search and MCP paths need no GPU and no external LLM API.

Why

In many "AI memory" and RAG setups, retrieved information tends to lose its context:

• it is unclear when the information dates from

• what it was based on is unclear

• old and new decisions get mixed together

• what an AI proposed gets mixed with what a person confirmed

• once something is indexed, it is unclear when deletion or expiry takes effect

• private and work sources are separated only by a logical filter

knowledge-memory puts provenance, lifecycle, bounded retrieval and human approval at the center of the design, not only search accuracy.

Beating general-purpose RAG on retrieval quality is not a goal. The difference is that an AI and a person can check where a retrieved record came from, when, and in what state.

Related MCP server: akatsuki

Core principles

• Source-grounded — active records carry at least one source reference; imported records carry their retrieval time

• Local-first — the vault and the SQLite index live on the user's machine. Search, record retrieval and MCP do not connect to external services

• Rebuildable index — the SQLite index is not the original; it can be rebuilt from the vault

• Human-in-the-loop — AI-generated records start as candidates

• Explicit lifecycle — candidate / active / superseded / retracted

• Bounded context — search results and task contexts have explicit limits; bounded MCP mode also limits each whole response

• Fail closed — a stale index or an incomplete fetch is not treated as success

• Boundary aware — keep personal and work data in separate vaults, databases and processes, not only separate scopes

Architecture

Notion / CSV / inbox text
          │
          ▼
   raw snapshot (outside the vault)
          │
          ▼
┌──────── Markdown vault ────────┐
│  candidate ──human review──► active
│  active ──► superseded / retracted
└────────────────────────────────┘
          │  km index (rebuildable)
          ▼
    SQLite FTS5 index
       │        │
       ▼        ▼
      CLI   stdio MCP ──► AI agent

Candidates and active records live in the same vault and are told apart by status. When Notion is the original, the snapshot and the Markdown derived from it are kept apart. The SQLite index can be rebuilt from the vault at any time.

What it does

• YAML frontmatter validation (km validate)

• SQLite FTS5 trigram search, plus literal search for terms shorter than 3 characters (km search)

• state- and scope-aware search, stale-index detection

• source-grounded record retrieval (km get)

• task context generation (km context)

• read-only stdio MCP server, with a bounded mode that enforces response limits (km-mcp)

• read-only Notion fetch and import, CSV import (km fetch-notion / km import-notion / km import-csv)

• explicit synchronization and availability state (km-sync)

• candidate review workflow (km-inbox; non-LLM extraction by default, optionally a local model on loopback)

• backup / restore / recovery check / measurement (km-ops)

• question-set evaluation (km evaluate)

Operator tools, for when you need them:

• pre-deployment checks of profile boundaries (km-profile)

• serialization of sync, index and MCP, with status and post-restore reconciliation (python -m knowledge_memory.runtime)

• retention planning and explicit deletion of sync evidence (km-retention)

Experimental:

• hybrid search with character n-grams or an optional embedding model (km-hybrid, opt-in; default search stays literal)

• an MCP entry point that enforces retrieval limits for evaluation (km-eval-budget)

Quick start

Requires Python 3.11+ and SQLite FTS5 with the trigram tokenizer (SQLite 3.34 or later). Only the pip install step downloads packages. The commands below use only the synthetic data in this repository.

python -m venv .venv
.venv/bin/python -m pip install -e '.[mcp]'

.venv/bin/km validate --vault examples/vault
.venv/bin/km index --vault examples/vault --db .state/demo.sqlite3
.venv/bin/km search --db .state/demo.sqlite3 --scope demo 育種
.venv/bin/km get --db .state/demo.sqlite3 --scope demo note-search-example

The synthetic notes are written in Japanese; 育種 means "breeding".

Five-minute demo

To run validate → index → search / get → context → MCP in one go (no external API, GPU or credential; everything happens in a temporary directory that is removed afterwards):

.venv/bin/python examples/demo.py

See docs/demo.md for what each step shows and what it does not.

Tests

.venv/bin/python -m unittest discover -s tests -v

CI runs on Python 3.11, 3.12 and 3.13.

MCP

A stdio MCP server is included.

Default mode:

• search_memory

• get_memory

• build_context

Bounded mode:

• search_memory_bounded

• get_memory_chunk

• build_context_bounded

.venv/bin/km-mcp --vault examples/vault --db .state/demo.sqlite3 --scope demo
.venv/bin/km-mcp --vault examples/vault --db .state/demo.sqlite3 --scope demo \
  --bounded --response-max-chars 12000 --response-max-bytes 24000

The vault, database and scope are fixed when the server process starts. Candidates are visible only when the server is started with --include-candidates; tool arguments from the model cannot widen that. Arbitrary file reads and shell execution are not exposed as memory tools. Bounded mode does not advertise the default tool names and never falls back to full-text retrieval under those names. How to register the server differs by MCP client.

A working MCP connection does not mean an agent will choose to search and retrieve records on its own. Check the SDK connection and, separately, whether your client actually searches, retrieves the record and answers with sources. See "Checking that an agent actually uses the server" in docs/mcp.md and the acceptance guide.

Record model

A record distinguishes at least:

• stable ID

• source

• retrieval / edit / event / verification time

• lifecycle status

• claim kind

• scope

active means "used by normal search"; it does not mean "true".

Details: docs/schema.md

Notion ingestion

Nothing is written to Notion. Only read APIs are called (retrieving pages, blocks and data sources, and querying data sources).

Notion
  ↓  km fetch-notion (read-only API)
raw snapshot (outside the vault)
  ↓  km import-notion / km-sync
candidate Markdown (inside the vault)
  ↓  km index
search index (candidates are excluded by default; use --include-candidates explicitly)
  ↓
human review (making a record active is an explicit human step)

Imports do not treat incomplete pagination or inconsistent cursors as success. Material that can no longer be fetched keeps its original text but is put on hold and removed from search. Credentials, snapshots and real vaults are stored outside this repository.

What it does not guarantee

• It does not verify whether source content is true

• scope is not authentication or authorization

• A search score is not a confidence value

• Zero hits do not prove that the information does not exist

• An MCP connection alone does not guarantee that an agent will use the memory

• The server cannot guarantee that an agent will not follow instructions contained in retrieved text; the client must treat it as data

• It is not a multi-user distributed database. It is designed for a single user and small vaults

• It is not a way to store a private vault on GitHub

Verification status

• Automated tests and CI use synthetic fixtures and the official MCP Python SDK stdio client

• The evaluation results in this repository are for a synthetic corpus. They say nothing about search or answer quality on real data

• Split retrieval for very large Notion data sources (over the 10,000-item API limit) is not implemented

• Local-model use in km-inbox and embedding models in hybrid search come with no quality guarantee for any particular model

Repository data policy

This repository contains synthetic fixtures only. It does not contain private vaults, imported articles, raw snapshots, credentials, work data, real Notion IDs, or private evaluation questions and results.

Reporting and contribution rules: SECURITY.md, CONTRIBUTING.md

Documentation

Several documents under docs/ are still written in Japanese or mix Japanese and English.

Getting started

• Five-minute demo — the main path on synthetic data

Core design

• Architecture — design decisions and functional boundaries

• Record schema — frontmatter, states, sources and times

• MCP — tool contract, bounded mode, checking agent usage

Ingestion and operations

• Notion import settings and conversion

• Synchronization and data source sync

• Acceptance — what to check after setting it up

• Operations — backup, restore, recovery checks, measurement

• Inbox review — AI proposals and human approval

Advanced / experimental

• Profile boundaries — keeping personal, work and public sources apart

• Managed runtime and evidence retention

• Hybrid search — opt-in search comparison and evaluation

Maintainers

• Releasing and updating — merging changes, release checklist, tags and GitHub Releases

Status

v0.1, the first public release. Clean install, security/privacy and accurate documentation take priority over new features. Direction and contribution priorities: docs/roadmap.md.

License

Apache License 2.0