Open Knowledge Fabric (OKF)

A personal knowledge base that an AI assistant reads and writes through one MCP server. The notes are plain markdown files with YAML frontmatter, in a git repo you own. Between the assistant and those files sits the Librarian: ordinary, deterministic Python with no LLM in it. It assigns ids, files notes, validates every write against a schema, enforces each note's lifecycle, and searches.

The assistant does the judgment: what to write, what a new note makes false, how to rerank what search returns. The Librarian does the bookkeeping and refuses bad writes. Search runs on your machine: word scoring plus a small local embedding model (bge-small), so your text never leaves the box.

Built by Sumit for his own notes, and run daily against several hundred of them. This repo has the code and a fictional example corpus; no real notes are included.

The note design: three layers

Every note (a node) has three layers, read in this order:

1. Metadata. Title, type, project, tags, and asks: up to eight questions a stranger would type when this note is the answer. Search indexes asks like the summary, so a note can be found by someone who has forgotten its words.

2. Summary: the current state. Up to eight - bullets. Bullet 1 says what the note is. Each other bullet states one point the note currently makes and ends with → § <heading>, the section of the body that holds the detail. When a point changes, its bullet says so: (changed 2026-09-02: was 2pm). The summary is rewritten whenever the note changes.

3. Body: the long form. It is never overwritten. Corrections are appended as dated blocks, and a body replacement that would drop existing text is refused unless the caller passes an explicit flag. The body is the history; the summary is the answer of record.

---
id: okf_20260902_064500_d002
type: decision
title: Wholesale orders close at noon the day before
summary: |-
  - The current wholesale cut-off: cafés order by noon for the next morning's delivery.
  - Noon gives the evening shift the full mix before it starts → § Why
  - Replaces the 2pm cut-off (changed 2026-09-02: was 2pm) → § Why
status: active
maturity: established
canonical: true
project: bakery
asks:
- what time do cafés have to order by
- can a café order after lunch
supersedes: okf_20260714_070000_d001
---
# Wholesale orders close at noon the day before
## Why
…

Replacing a note does not delete it. The new note supersedes the old one; the Librarian marks the old note superseded and links both ways. Search then leaves superseded notes out and tells you history exists.

Related MCP server: Cortex

Quickstart (5 minutes, over stdio)

You need Python 3.11+ and git.

git clone https://github.com/sumit-anfy/open-knowledge-fabric.git
cd open-knowledge-fabric
python3.11 -m venv .venv
.venv/bin/pip install -r requirements.txt
.venv/bin/python -m librarian.validate examples/corpus     # OK — 0 error(s)

The server runs against the bundled example corpus by default: 37 notes from a fictional baker, Rowan, covering every note type. To keep the repo clean while you experiment, work on a copy:

cp -r examples/corpus ~/okf-demo

Claude Code:

claude mcp add --transport stdio --env OKF_ROOT=$HOME/okf-demo okf -- "$PWD/.venv/bin/python" "$PWD/server.py"

Claude Desktop: add this to claude_desktop_config.json (Settings → Developer → Edit Config), with absolute paths, then restart Claude Desktop:

{
  "mcpServers": {
    "okf": {
      "command": "/absolute/path/to/open-knowledge-fabric/.venv/bin/python",
      "args": ["/absolute/path/to/open-knowledge-fabric/server.py"],
      "env": { "OKF_ROOT": "/Users/you/okf-demo" }
    }
  }
}

Then ask Claude things like:

• "Brief me: what time do the cafés have to order by, and why did it change?"

• "Who owns the bakery?"

• "Create a note in garden: the beans failed because they went in too early."

• "Run the corpus audit."

The first search downloads the embedding model (about 64 MB) into a local cache. Until it is ready, and wherever fastembed is not installed, search falls back to words only and says so (semantic: false).

To start your own corpus, copy examples/corpus/schemas/ into an empty folder, edit PROJECT_REGISTRY in schemas/type-registry.md to list your own homes, git init it, and point OKF_ROOT at it. Set OKF_GIT_COMMIT=1 to have every write committed. All settings are in .env.example.

Run the tests (about 30 seconds). The write tests run on throwaway copies of the example corpus:

.venv/bin/pip install -r requirements-dev.txt
.venv/bin/python -m pytest

The tools

A session note must end with a close-out (Keep / Conflicts / Your call / Tasks), and every ruling it lists must cite the note it is filed in. The server refuses a session note that skips either.

Architecture

Claude (Desktop, Code, claude.ai)
   │  MCP: stdio, or HTTP behind TLS with a bearer token or GitHub OAuth
server.py           thin FastMCP wrapper: ten tools, auth, no logic
   │
librarian/          the deterministic core (no LLM, no network)
   validate.py      schema + registry, read live from the corpus's own schema notes
   store.py         the write path: lock, validate as a set, atomic replace,
                    state machine, supersession, body guard, git commit/push
   context.py       brief(): hybrid ranking, intent-aware type priors, aliases
   embed.py         local bge-small embeddings (fastembed, ONNX), cached in ~/.okf
   sections.py      long notes searched section by section
   summary.py       the bullet-summary format and its checks
   cluster.py       after a write: "which neighbours does this make false?"
   redundancy.py    after a rule is written: "does a canonical note already say this?"
   completeness.py  session rulings must cite the note they are filed in
   audit.py         weekly triage, link suggestions, purge candidates
   findability.py   weekly run of held-out stranger questions, logged as a note
   │
your corpus         markdown + YAML frontmatter, its own git repo
   schemas/         node.schema.md and type-registry.md are themselves notes
okf_web/            Starlette graph explorer and markdown editor, password login
deploy/             systemd units, Caddy, a pull-and-test auto-deploy timer

Some design choices:

• The vocabulary lives in the corpus. Note types, folders and project homes are YAML blocks inside schemas/type-registry.md. The validator reads them live; changing your homes is an edit to a note, not to the code.

• Fail closed. No registry means no writes. HTTP without auth refuses to start. A check that cannot run reports a failure, not a pass.

• Search only adds. The embedding layer and the aliases can reorder results or add matches, but never remove a word match. Every score is broken down in the output (score_lexical, score_semantic, prior), so you can see why a note ranked where it did.

• Live data stays in its own app. Tasks, spreadsheets and calendars are resource notes that point at them, never copies that go stale.

Retrieval numbers (example corpus only)

python tests/retrieval_report.py prints these. They are measured on the 37-note example corpus with the benchmark in tests/retrieval_benchmark.yaml (48 questions) and the held-out bank in tests/findability_seed.yaml (20 notes, two questions each, written without reusing the note's own words).

Read these as a smoke test, not a quality claim: the same person wrote the corpus and the questions, and 37 notes is an easy haystack. The useful signal is the gap between the two columns on held-out wording. That gap is what the embedding layer is for.

What is not built

• No LLM inside. The server never calls a model. Summaries, asks, and the "what does this make false?" review are written by the assistant that calls the tools.

• No importers. Nothing reads Notion, Obsidian or Apple Notes.

• Single owner. One corpus, one owner. Auth decides who gets in, with no per-note permissions, no sharing, and no multi-user editing.

• No packaging. Run from a checkout; there is no pip install okf and no Docker image.

• No encryption at rest. Notes are plain files. Protect the disk and the repo.

• Edges are only links. relates_to links are plain lists. The typed-edge vocabulary in the schema (supports, contradicts, …) is not used yet.

• Tested on macOS and Linux only. It has not been run on Windows.

• Scheduled work is external. The weekly audit is meant to be run by a scheduled assistant task calling audit_corpus; this repo ships no scheduler beyond the findability run inside the HTTP service.

Licence

MIT.