Skip to main content
Glama
CROW: LLM Wiki

CROW: LLM Wiki

Official
README.md
<p align="center">
  <img src="https://raw.githubusercontent.com/crow-hq/llm-wiki-v2/main/docs/assets/crow-banner.jpg" alt="CROW: a pixel-art crow next to the word CROW" width="100%">
</p>

<p align="center">
  <img src="https://raw.githubusercontent.com/crow-hq/llm-wiki-v2/main/docs/assets/crow-stats.svg" alt="An LLM wiki with no scaling limit. CROW vs classic on 11 corpora (2,019 documents, 656 questions): 99.5% right note read vs 79%; 1.6× faster per ingest (18.0s vs 29.2s); 63% fewer LLM tokens (4,861 vs 13,227); 0.59× the cost per ingest with the writer at list price." width="100%">
</p>

<p align="center">
  <img src="https://raw.githubusercontent.com/crow-hq/llm-wiki-v2/main/docs/assets/demo.gif" alt="Twenty-two documents dropped on the brain of an empty wiki are filed in folders the wiki opens, and the brain grows with them (shown five times faster); a question about a damaged parcel gets an answer citing two notes, and one of them opens" width="100%">
</p>

# LLM Wiki v2

Give it your documents. It files them into a tidy wiki of Markdown notes and
answers your questions, citing the notes it used. It runs on your computer,
with any AI provider: OpenRouter, OpenAI, Gemini, Ollama (free and local) or
any OpenAI-compatible server. It works in **CROW** mode by default: a small
classifier decides where things go, so the wiki stays fast and cheap as it grows.

## Start in 2 steps

Rather not use a terminal? [Crow](https://github.com/crow-hq/crow-desktop) is this library as a desktop app, with
downloads for macOS, Windows and Linux. The steps below install the library itself, with its web page and command line.

**1. Install it** with Python 3.11 or later (in a virtual environment, if your system asks for one):

```bash
pip install llmwiki2
```

No Python, or you'd rather keep it apart from your other packages? Use [uv](https://docs.astral.sh/uv/getting-started/installation/),
which also fetches Python: `uv tool install llmwiki2`. The same install is the Python library too:
`from llmw2 import Wiki`, see the [library guide](https://github.com/crow-hq/llm-wiki-v2/blob/main/docs/library/index.md).

**2. Start it:**

```bash
llmwiki2
```

Your browser opens. Pick a provider, paste its key, press **Test connection**,
then **Save and start**. That's it.

**Just looking?** `llmwiki2 demo` opens an example wiki, the notes of a fictional coffee
roastery, with no key needed to browse it and its brain. It is copied to `~/llm-wiki-demo`,
so you can add to it or delete it. To try it without installing anything but uv:
`uvx llmwiki2 demo`.

<p align="center">
  <img src="https://raw.githubusercontent.com/crow-hq/llm-wiki-v2/main/docs/assets/screenshot-settings.png" alt="The first-run page: pick a provider, paste its key, choose a model" width="90%">
</p>

> [!TIP]
> **No key?** Pick **Ollama**: free, runs on your computer, no account
> ([download it](https://ollama.com/download), then `ollama pull gemma4`).
> Ollama runs the model locally; CROW's classifier is separate and remote by default.
> It has its own provider (Settings → Advanced): OpenRouter, TypeSafe or a local
> server. Without one the wiki uses classic mode, where the model decides by itself.
> One [OpenRouter key](https://openrouter.ai/keys) works for every model, and for CROW too.

## What you can do

| | |
|---|---|
| **Add documents** | Drop text, Markdown, PDF, Word, PowerPoint, Excel or OpenDocument files anywhere on the page. Each one lands in the right folder, merged with what the wiki already knows; a spreadsheet gives a note per sheet. |
| **Ask questions** | Type a question. The answer cites the notes it used: click one to read it. |
| **Give Claude a memory** | From the terminal only: `llmwiki2 setup`, then talk to Claude Code or Codex in plain words. With the page: **Memory** → **Start**, then paste the `claude mcp add …` or `codex mcp add …` line it shows. Either way they recall from your wiki and file what they learn in it ([below](#claudes-memory-mcp)). |
| **See the brain** | A live map of your folders, notes and links. |
| **Delete** | Hover a note or folder in the sidebar and press the bin, or **Delete note** at the foot of a note. Links to it go from the other notes. **Empty the wiki** on the home page starts over. |
| **Move and describe** | **Move** at the foot of a note files it in another folder; every link to and from it follows. The pencil beside a folder in the sidebar edits its description, the line the librarian routes by. |
| **Areas** | **Areas** at the top: create a top-level folder, confirm or dismiss the areas proposed to the wiki, rename one or merge it into another. The wiki never opens or proposes an area by itself. |
| **Keep your files** | Plain Markdown in `~/llm-wiki`: open it in any editor, keep it in git. |

## Claude's memory (MCP)

<!-- mcp-name: io.github.crow-hq/llm-wiki-v2 -->

The wiki can be the long-term memory of Claude Code, Codex or any MCP client, documents already filed included.
It comes with the `mcp` extra (the Docker image has it).

### From the terminal only (Claude Code, Codex)

No page, no server to keep running: Claude Code starts the memory by itself.

<p align="center">
  <img src="https://raw.githubusercontent.com/crow-hq/llm-wiki-v2/main/docs/assets/demo-memory.gif" alt="In a terminal, llmwiki2 setup shows the CROW logo and is answered 2, then Enter at every question: the memory is on and Claude Code is connected. In Claude Code, a remember about a supplier's minimum order is merged into the supplier's note, whose Markdown file beside it shows the new line; in a new session, a question in Italian is answered in Italian through recall, citing that note (the models' work sped up)" width="100%">
</p>

**1. Install it:**

```bash
uv tool install 'llmwiki2[mcp]'      # or: pip install 'llmwiki2[mcp]'
```

**2. Set it up:**

```bash
llmwiki2 setup
```

To the first question answer **2** (as the memory of an agentic CLI); after that, Enter takes the default every time:

| It asks | Pick |
|---|---|
| Which models do the work | **1** (the default): Claude Code or Codex do it, on their plan, no key. **2** (hybrid) is quicker and needs an OpenRouter key, or Ollama on your computer |
| Connect Claude Code? | **Y**. With several configurations (`~/.claude`, `~/.claude-work`), it asks which: Enter for all |
| Add the lines to CLAUDE.md? | **Y**: they make Claude use the memory when you talk normally |

**3. Talk to it**, in any language, in Claude Code or Codex, anywhere:

> ricorda che Rossi ci paga a 45 giorni · what did we decide about pricing? · add ~/Downloads/offer.pdf to the
> wiki · move the pricing note to suppliers

The first time it uses each tool, Claude Code asks your permission: choose "don't ask again" for the ones you trust.
To see the brain or browse the notes, run `llmwiki2`: the page opens on the same wiki, while Claude Code keeps using it.
Run `llmwiki2 setup` again to change anything. Windows says it blocked `llmwiki2.exe`? See
[Something wrong?](#something-wrong).

### With the page

Open **Memory**, press **Start**, and connect a client:

```bash
claude mcp add --transport http --scope user llm-wiki http://127.0.0.1:8000/mcp
codex mcp add llm-wiki --url http://127.0.0.1:8000/mcp
```

### What it can do

The client gets its tools: `recall` answers from your notes and documents, citing them; `remember` files what it
learns, in the right folder, merged and linked like any note (CROW); `search` finds notes by their words with no model
call; `read_note` reads a note in full; `folders`, `move_note`, `describe_folder`, `rename_area`, `merge_areas` and
`delete_note` reorganize the wiki (the last two only when you ask); `rebuild` (with `confirm`) files every note again on
the same folders, from the memory's own raw copies. Started from the terminal (`llmwiki2 mcp`) it also files documents
from your disk: `remember_file` and `sync_folder`. Which models do the work is a choice under Memory or in `setup`:

- **This wiki's settings** (the default on the page). OpenRouter or a local model, with Jev or Laya deciding, file the memories; the
  client only calls the memory.
- **Claude or GPT for everything** (the default in `setup`). The memory hands the client every model call it makes, while CROW runs as always.
  Claude Code decides with a Haiku subagent and writes with its own model; Codex uses its own. It runs on the client's
  plan, with no key in the wiki. The requests carry your documents' text to the client: it is told to read it as data
  and to answer without tools, but that is an instruction, not a sandbox (see Security in the memory guide).
- **Hybrid.** The client's model writes, and the wiki's classifier (Jev through OpenRouter, or Ollama or Laya on your
  computer) decides: fewer round trips than the above, and no Haiku subagents.

Docker, the tools and security: [docs/wiki/memory.md](https://github.com/crow-hq/llm-wiki-v2/blob/main/docs/wiki/memory.md).

## Why not RAG?

Because a wiki stays readable. RAG keeps your documents as embeddings, an opaque
vector index no one can read or fix. Here every document becomes a Markdown note,
filed in a folder that says what it holds and linked to related notes: you can open
it, correct it and keep it in git, and every answer cites the notes it used. And
knowledge accumulates: a new source updates the note on its subject instead of
being rediscovered at every question.

CROW keeps it that way as the wiki grows. In [Karpathy's LLM Wiki](https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f)
the model finds pages by reading an index, which works at moderate scale (about a
hundred sources); beyond that the gist suggests adding search, hybrid lexical and
vector. In CROW a small classifier works with the folders instead: filing picks the
folder among every folder of the wiki in one choice, and answering finds the notes by
search and lets the classifier reorder them, so the wiki can grow without vectors and stay
something a person can read. The LLM lays out the folders: in a batch, before its documents are filed, and
for a document filed alone, by splitting a folder that has grown. In the [paper](https://doi.org/10.5281/zenodo.22898752), retrieval kept its
accuracy across a twentyfold growth of the wiki. A separate test on a private wiki
of about 500 notes pointed the same way: with CROW the right note was found nearly
as often at the end as early on, while with the LLM taking the decisions it was
found markedly less often as the wiki grew. On 1,000 Enron mails filed in their owners' folders, CROW read the right
note for every question at each step of the growth while classic went from 76% to 60%. The classifier weighs at most
24 candidate notes per decision, so its tokens per document nearly stopped growing after the first few hundred documents;
CROW's cost per document rose 1.4 times over the 1,000, classic's 1.6 times, and at 800 to 1,000 documents CROW cost 0.41
times classic's with the writer at its list price ([benchmarks/README.md](https://github.com/crow-hq/llm-wiki-v2/blob/main/benchmarks/README.md#crow-as-the-wiki-grows)).

## FAQ

**Does everything run on my computer?** It can, with no key and nothing leaving it.
[Crow](https://github.com/crow-hq/crow-desktop), the desktop app, does it out of the box. With the library, Ollama
serves the writer and the classifier too: a typed classifier such as `tev1` behind Ollama 0.35 or later, or Laya
(see [everything on your computer](https://github.com/crow-hq/llm-wiki-v2/blob/main/docs/wiki/providers.md#everything-on-your-computer-with-small-models)).
The library's default classifier is remote: Jev, through OpenRouter or TypeSafe. Without a classifier the wiki runs in
classic mode, where the LLM takes every decision.

**What is Jev?** A typed classifier by TypeSafe AI. Given a text, it answers typed
questions (pick one of these options; is this statement true?) with probabilities
instead of prose. CROW asks it every filing and retrieval decision and keeps the
LLM for writing. CROW does not depend on it: any classifier that returns those
probabilities can take its place. The authors have no ties to TypeSafe AI.

**Aren't these your own benchmarks?** Yes. The paper measures one corpus of 525
personal documents, mostly in Italian, with one LLM and one classifier version, and
says so among its limitations; its rates carry 95% intervals. The corpora and
scripts behind these measurements are public in [benchmarks/](https://github.com/crow-hq/llm-wiki-v2/tree/main/benchmarks),
and every script has a `--dry-run`, so you can run them with your own models.
The numbers of the banner come from eleven corpora (handbooks, mail, laws in two languages, contracts, tax manuals,
Wikibooks pages and personal notes, 2,019 documents), each filed in CROW and in classic mode with the same writer, and
656 questions written blind from single documents: the right note was read 99.5% of the time against 79.4%, filing was 1.6
times as fast with 63% fewer LLM tokens. The cost depends on the writer's price: on the cheapest providers, where the writer
costs about what the classifier does, it came out the same; with the writer at its list price CROW costs 0.59 times classic's.
The table per corpus is in [benchmarks/README.md](https://github.com/crow-hq/llm-wiki-v2/blob/main/benchmarks/README.md#crow-against-classic-on-eleven-corpora).
The filing measures of the CROW routing use, besides the repo's own synthetic coffee
corpus, the [Enron email corpus](https://www.cs.cmu.edu/~enron/), the
[GitLab handbook](https://gitlab.com/gitlab-com/content-sites/handbook) and the
[PostHog handbook](https://github.com/PostHog/posthog.com): they are linked, not copied.
On the PostHog handbook, CROW files pages closer to their real folders than `main` (its CROW routing and its classic
mode) with both models measured, at 0.86 to 0.87 times the cost of main's CROW and 0.45 to 0.65 times that of classic,
with retrieval between 93% and 98% for all: the table, what each column measures and how to run it again are in
[benchmarks/README.md](https://github.com/crow-hq/llm-wiki-v2/blob/main/benchmarks/README.md#results-of-060-crow-against-crow-050-and-classic).

**Why AGPL, and why a CLA?** The AGPL keeps the code and every change to it open,
also when it runs as a service. The CLA keeps the rights clear between the two
authors and everyone who contributes; you keep the copyright in your work.

## Something wrong?

- **"no API key yet"**: open ⚙ on the page, or run `llmwiki2 setup` in the terminal.
- **Test connection fails**: the message is the provider's own, usually a wrong key or model name.
- **A PDF adds nothing**: it is a scan with no text layer; run OCR on it first.
- **Another folder for the wiki**: `llmwiki2 --bundle <folder>`. Settings live in `~/.config/llm-wiki/config.json`.
- **Windows blocks `llmwiki2.exe`** ("an application control policy has blocked this file", common on work PCs): run
  it through Python, which the policy lets through. Installed with pip: `python -m llmwiki2 setup`. With uv, in
  PowerShell: `& "$(uv tool dir)\llmwiki2\Scripts\python.exe" -m llmwiki2 setup`. Every command works this way
  (`-m llmwiki2 serve`, `-m llmwiki2 demo`), and `setup` connects Claude Code through that same Python.

## For developers

<details>
<summary><b>Run it as a server with Docker</b></summary>

```bash
git clone https://github.com/crow-hq/llm-wiki-v2 && cd llm-wiki-v2
cp .env.example .env          # choose OKF_LLM_PROVIDER and set its key
docker compose up -d --build  # the wiki on :8000 (OKF_PORT to change)
```

With Docker the settings come from `.env`; the page shows them but cannot change them.

</details>

<details>
<summary><b>HTTP API</b></summary>

**The web UI** at http://localhost:8000 shows the folder tree, renders each note
(with its See-also links and sources), answers questions with clickable
citations, and shows the tokens spent by the LLM and by the classifier. Drop
text (`.txt`, `.md`), PDF, Word (`.docx`), PowerPoint (`.pptx`), Excel (`.xlsx`) or
OpenDocument (`.odt`, `.ods`) files anywhere on the page and each one is catalogued
automatically; PDFs need a text layer (scanned PDFs need OCR first). A spreadsheet gives
one note per sheet, which describes it; its rows stay in the raw copy as CSV, and
questions get the rows they need. Old binary formats (`.doc`, `.xls`, `.ppt`) are not
read: save them as `.docx`, `.xlsx` or `.pptx`.

The same through the API:

```bash
curl -X POST localhost:8000/ingest -H 'content-type: application/json' \
     -d '{"text": "Acme refunds orders within 30 days of delivery.", "title": "Refund policy"}'
curl -X POST 'localhost:8000/upload?filename=minutes.pdf' --data-binary @minutes.pdf
curl -X POST localhost:8000/ask -H 'content-type: application/json' \
     -d '{"question": "How long do customers have to ask for a refund?"}'
```

The wiki is plain Markdown files you can read, edit and keep in git (`./wiki`
with Docker). With Docker the CLI runs in the same image: `docker compose run --rm wiki llmwiki2 check`.
To find notes with no model call: `llmwiki2 search "burner service" -k 5` (`--where KEY=VALUE`, `--json`). To file a note elsewhere or reword a folder's description from the terminal: `llmwiki2 move finance/pricing.md /sourcing` and `llmwiki2 describe /finance "Money in and out."`.
`llmwiki2 rebuild [folder] --yes` deletes every note, raw copy and archived version and files the documents again on the folders the wiki has (the documents of `folder`, or with no folder the wiki's own raw copies), with no plan and no new folder; without `--yes` it only says what it would do and exits with 2. A note the limit on top-level folders sent to the likeliest existing folder instead of a new one is marked `overflow` in its frontmatter, logged in `log.md` as "Deviated note", and listed with the root it went to and its probabilities by `sync`, `ingest` and `rebuild`.
In CROW mode, `llmwiki2 audit [--min-confidence 0.7] [--apply]` lists the notes the classifier would file in another branch today (it asks the same single choice among every folder, and proposes only a move to another branch, never deeper into the note's own folder and never up; no LLM call; it moves them only with `--apply`; `--min-confidence` is compared with that choice's top probability, a scale that has not been recalibrated), and `llmwiki2 health [--min-filed 5] [--max-share 0.3]` shows, from `log.md`, the folders that keep needing new subfolders or the LLM fallback (folders laid out by a plan or a split are not counted as new subfolders). Every new note carries a stable `id` in its frontmatter (`wiki.find_note(id)`).

| Endpoint | What it does |
|---|---|
| `GET /` | the web UI |
| `POST /ingest` `{text, title?, resource?, origin?}` | file a source (an `origin` `{source, account?, id, link?, version?, modified?}` makes a repeat with the same content or version return `"unchanged"`); returns the note, folders created, links and token usage |
| `POST /upload?filename=…` (raw file body) | file a text, Markdown, PDF, Office or OpenDocument file (a spreadsheet: its other sheets in `parts`) |
| `POST /ask` `{question}` | answer with citations; returns the notes read and token usage |
| `GET /search?q=…&k=8` | the notes that best match a query (path, title, summary, rank), with no model call: a search box |
| `GET /tree` · `GET /note?path=…` | folder tree · one note (frontmatter and Markdown body) |
| `GET /check` · `GET /graph` | lint problems · the wiki as nodes and links (JSON) |
| `POST /note/move` `{path, folder}` · `POST /folder/describe` `{path, description}` | move a note to another folder, links following (returns the note as `GET /note`) · change a folder's description |
| `GET /proposals?status=open\|all` · `POST /proposals` `{label, description, notes}` · `POST /proposals/{id}/confirm` `{over_limit?}` · `POST /proposals/{id}/dismiss` | the queue of top-level areas users propose · propose one · open it and move its notes · close it |
| `POST /areas` `{name, description, over_limit?}` · `POST /folder/rename` `{path, name}` · `POST /folder/merge` `{source, target}` | create a top-level folder, refused over `max_top_folders` unless `over_limit` · rename one · move all of one into another and delete it (links following) |
| `POST /audit` `{min_confidence?, apply?}` · `GET /taxonomy` | CROW: notes the classifier would file in another branch (moved if `apply`) · per folder and month, new subfolders and fallbacks (from the log; folders laid out by a plan or a split not counted) |
| `DELETE /note?path=…` · `DELETE /folder?path=/…` · `DELETE /wiki` | delete a note · a folder with all it holds · everything; links to them go, and raw copies no remaining note cites |
| `GET /usage` · `GET /health` | token totals per ledger and model · status |
| `GET /settings` · `POST /settings` · `POST /settings/test` | provider, model and key (masked) · save them · try them; changes from this machine only |
| `POST /mcp` | the wiki as an MCP memory (Streamable HTTP; `recall`, `remember`, `search`, `read_note`, `rebuild`, `reply`, `create_area`, `propose_area`, `proposals`, `confirm_area`), with the `mcp` extra. Its tools work while the settings turn it on (`OKF_MCP`, or **Memory** → **Start**), and say how to turn it on otherwise; without the extra, 501 and what to install |
| `POST /settings/memory` `{mcp?, mcp_models?}` | turn the memory on or off, and choose whose models do its work; no key needed, from this machine only |

Every request that changes the wiki may carry an `X-Author` header: the lines it adds to `log.md` end with ` · by <author>`. The author is declared, not verified: there is no authentication, so behind a reverse proxy that authenticates your users, have the proxy set the header. The CLI declares your login name.

</details>

<details>
<summary><b>Use it as a Python library and from the command line</b></summary>

```bash
pip install llmwiki2
```

```python
from llmw2 import Wiki

wiki = Wiki.from_env(bundle="./wiki")        # OKF_* variables and the provider's key, e.g. OPENROUTER_API_KEY
wiki.init()

result = wiki.ingest(open("meeting.md").read(), title="Pricing meeting")
wiki.ingest_file(open("minutes.pdf", "rb").read(), "minutes.pdf")   # also .docx, .pptx, .xlsx, .odt, .ods
print(result.action, result.note, result.created_folders, result.related)

answer = wiki.ask("What did we decide about enterprise pricing?")
print(answer.text, answer.citations)

print(wiki.usage.to_dict())                  # {"llm": …, "classifier": …, "by_model": …}
```

The same operations are on the command line:

```bash
llmwiki2 --bundle ./wiki init
llmwiki2 --bundle ./wiki ingest meeting.md     # or a .txt, .pdf, .docx, .pptx, .xlsx, .odt, .ods
llmwiki2 --bundle ./wiki sync ~/OneDrive       # file a folder and keep the wiki in step with it
llmwiki2 --bundle ./wiki --mode classic ask "What did we decide about pricing?"
llmwiki2 --bundle ./wiki check        # lint: OKF conformance, index drift, broken links
llmwiki2 --bundle ./wiki serve        # the HTTP API and page; plain `llmwiki2` also opens the browser
llmwiki2 setup                        # provider, key and model, tested and saved; or the memory of Claude Code/Codex
llmwiki2 --bundle ./wiki mcp          # the wiki as an MCP memory over stdio, started by the client (claude mcp add … -- llmwiki2 mcp)
llmwiki2 demo                         # an example wiki, copied to ~/llm-wiki-demo and opened
```

To keep the wiki in step with a source, hand `sync` a connector. Each document
keeps its `Origin` (where the original lives), so an unchanged file costs no model
call and a deleted one is marked in the raw copy while its notes stay:

```python
from pathlib import Path
from llmw2 import LocalFolderSource, sync

report = sync(wiki, LocalFolderSource(Path("~/OneDrive")))   # a folder, e.g. one rclone keeps in sync
print(report.to_dict())                                      # created, merged, unchanged, removed, skipped, errors
```

A connector for another service is any object with `name`, `account`,
`changes(cursor) -> ChangeBatch` and `fetch(change) -> bytes` (`SourceConnector`);
the cursor of the last sync is kept in `.llmw2/sync/` inside the wiki. A folder is handed over in pages of 200 changes and the
cursor is saved after each page, so a sync of thousands of documents that stops resumes where its last page ended.

For a wiki you rebuild or audit often, `classifier.cache` (`OKF_CLASSIFIER_CACHE=true`) keeps the classifier's answers in the
bundle, as hashes and answers only: a rebuild, a resumed sync or a repeated audit then asks it nothing it was asked before. It
does not save anything on a sync of new documents. See the [library guide](docs/library/index.md).

**Your own top-level areas.** Make them with `create_folder` before the first sync, each with a description (the
classifier routes by it); to keep it idempotent, `if wiki.tree().find("/finance") is None: wiki.create_folder("/", "finance", "...")`.
Folders made by hand count in `max_top_folders`: set it to the number of your areas and the wiki is closed (no new
top-level folder, the notes go to the likeliest area); a higher value leaves it open. `delete_folder` deletes the notes
in it. For different permissions, use one wiki per domain.

The wiki never opens or proposes an area by itself: proposals come from you, and the list of deviated notes helps you
compose them. `propose_area(label, description, notes)` queues one (the notes by id or path; the queue is kept in the
wiki, in `.llmw2/proposals.json`, and nothing moves yet), `proposals()` lists the open ones, `dismiss_proposal(id)` closes one,
and `confirm_proposal(id, *, over_limit=False)` opens the root (counted in the limit: refused when the wiki has as many, unless
`over_limit`) and moves into it only the proposed notes still there, each with `moved_from` (the folder it came from) and
without its `overflow` mark, with no rebuild and no other note touched. `rename_root(path, new_name)` renames a top-level
folder and `merge_roots(source, target)` moves all of one into another and deletes it; links follow, descriptions stay,
and each is one line of `log.md`. The same from the terminal (`llmwiki2 proposals [add|confirm|dismiss]`, `rename-root`,
`merge-roots`), the HTTP API and the **Areas** page (create an area, confirm or dismiss the open proposals, rename, merge;
proposals are added through the API, the CLI or the memory, not from the page), and from chat with the memory tools
`create_area`, `propose_area`, `proposals` and `confirm_area(id, yes)`.

Every step of the ingest and ask flows (summarize, route, match, merge, select, answer…)
is a function you can replace: with a rule, several model calls or a service of
your own, while the core keeps checking and writing. The guide to the whole
library is in [docs/library/](https://github.com/crow-hq/llm-wiki-v2/blob/main/docs/library/index.md),
the steps in [docs/library/steps.md](https://github.com/crow-hq/llm-wiki-v2/blob/main/docs/library/steps.md),
and runnable programs, some with no key, in [examples/](https://github.com/crow-hq/llm-wiki-v2/tree/main/examples).
From 1.0.0 the library follows semantic versioning; what that covers and what it does not is in the
[Compatibility](https://github.com/crow-hq/llm-wiki-v2/blob/main/docs/library/index.md#compatibility) section of the guide.

</details>

<details>
<summary><b>How it works: classic and CROW</b></summary>

```
ingest:  summarize → route → find match → merge or (new folder →) write → relate → link, index, log
         (CROW: before a batch window of 50, a plan of its folders; after a single document, a split of a folder that grew)
ask:     select notes (search; CROW reorders them with the classifier) → answer from them, with [path.md] citations
```

| Step | Classic mode (`OKF_MODE=classic`) | CROW mode (`OKF_MODE=crow`) |
|---|---|---|
| Summarize the source into a note | LLM | LLM (optional, `OKF_SUMMARIZE=false` files it verbatim) |
| Route: best folder, or a new one | LLM, one level at a time | one classifier Choice among every folder; per-level Choice and beam for an empty wiki or `None of these`; LLM fallback |
| Same subject as an existing note? | LLM | classifier Noul per note, at most 24 (the folder's first, then the closest by BM25) |
| Merge into it, or write a new note? | LLM | classifier Choice, unsure → new note |
| Name a new folder · rewrite a merged note | LLM | LLM |
| Related notes (See also) | LLM | classifier Noul per note, at most 24, plus up to 10 that share rare values |
| Find the notes for a question | LLM navigation | the wiki's search (BM25, and the embedder if set); the classifier reorders the notes after the first (Noul per note); the LLM walks the folders only when the search finds nothing |
| Write the answer | LLM | LLM |

Both modes run the same pipeline and write the same kind of wiki: only the
decider changes. CROW follows *CROW: Classifier-Routed Organization of LLM
Wikis* (Cesarini & Sassarini, 2026) — see [docs/wiki/crow.md](https://github.com/crow-hq/llm-wiki-v2/blob/main/docs/wiki/crow.md)
for thresholds and fallbacks.

**Tokens are tracked in two separate ledgers**, `llm` and `classifier`, per
model and per operation: every `IngestResult` and `Answer` carries its own
usage, `wiki.usage` / `GET /usage` the totals, and `OKF_USAGE_LOG` writes one
JSON line per model call.

</details>

<details>
<summary><b>The wiki on disk</b></summary>

> The wiki's on-disk format is derived from Google's
> **[Open Knowledge Format (OKF) v0.2](https://github.com/GoogleCloudPlatform/open-knowledge-format)**:
> every wiki written here is a conformant OKF bundle, with the extra conventions
> listed below. This project started as a fork of OKF and no longer includes its code.

```
wiki/
  index.md                     # root index (okf_version "0.2")
  log.md                       # what changed, newest first
  raw/                         # untouched copy of every source ingested
  customer-policies/
    index.md                   # * [Refund policy](refund-policy.md) - How Acme refunds orders…
    refund-policy.md           # type: Note, title, description, tags, generated, sources
```

Every note is an OKF concept (`type: Note`) whose `description` is its one-line
summary and whose `sources` point to the raw copies it was written from. Each
folder's one-line description sits in its parent's `index.md`, where both people
and the Librarian read it. Related notes are joined by a `# See also` section
with relative links. `llmwiki2 check` verifies all of this.

**How it relates to OKF v0.2.** A wiki written here is a conformant OKF bundle
(§11): every note has a YAML frontmatter block with a `type`, and `index.md` and
`log.md` follow §8 and §9. On top of that it adds its own conventions:

- every folder has an `index.md`, and a folder's description is its entry in the parent's index (OKF: optional indexes);
- links are relative (OKF recommends bundle-absolute `/…` links), and `check` reports broken ones;
- `raw/` holds the sources (`type: Source`), `# See also` holds the related notes, and there is one `log.md`, at the root;
- the root holds no notes: every note is filed in a topic folder, and the first note on a new subject opens one.

It does not use OKF's trust and lifecycle fields (`verified`, `status`,
`stale_after`), attested computations (§10) or the `references/` convention.
Wikis are not checked against later OKF versions.

</details>

<details>
<summary><b>Configuration</b></summary>

The settings page and `llmwiki2 setup` cover provider, key, model, mode and
wiki folder. Everything else, and every setting in Docker (`.env`) or in the
library, comes from the environment; it wins over the settings file. The full
list is in [.env.example](https://github.com/crow-hq/llm-wiki-v2/blob/main/.env.example).

| Variable | Default | Meaning |
|---|---|---|
| `OKF_BUNDLE` | `~/llm-wiki` for the CLI (`/data/wiki` in Docker) | wiki folder |
| `OKF_MODE` | `crow` | `crow` (a typed classifier decides, needs its key or a local server) or `classic` (the LLM decides) |
| `OKF_LLM_PROVIDER` | `openrouter` | `openrouter`, `openai`, `gemini`, `ollama` or `custom` |
| `OKF_LLM_MODEL` | the provider's default | e.g. `deepseek/deepseek-v4.1-flash` on OpenRouter |
| `OKF_LLM_BASE_URL` | the provider's | any OpenAI-compatible API (required for `custom`) |
| `OPENROUTER_API_KEY` · `OPENAI_API_KEY` · `GEMINI_API_KEY` | — | the provider's key (`OKF_LLM_API_KEY` wins); OpenRouter's also serves the classifier |
| `OKF_CLASSIFIER_PROVIDER` | `openrouter` | where the CROW classifier runs, apart from the LLM: `openrouter`, `typesafe` (both Jev) or `custom` (any System One server, such as Laya on this computer; starts from the Laya thresholds preset) |
| `TYPESAFE_API_KEY` | — | the classifier's key on `typesafe` (`OKF_CLASSIFIER_API_KEY` wins) |
| `OKF_CLASSIFIER_MODEL` | `typesafe/jev-1.13` | CROW classifier, pinned |
| `OKF_LLM_REASONING` | `false` | `true` lets the model think before it answers: several times slower and costlier; off is sent to OpenRouter (the least effort for models that must think), other providers keep their model's default |
| `OKF_CLASSIFIER_TIMEOUT` | `8` | seconds per classifier call; a slow or failed call goes to the LLM (the paper ran 60) |
| `OKF_CLASSIFIER_ATTEMPTS` | `1` | tries per classifier call before the LLM decides (the paper ran 3) |
| `OKF_CLASSIFIER_BASE_URL` | the classifier provider's | System One endpoint (required for `custom`) |
| `OKF_CONFIG` | `~/.config/llm-wiki/config.json` | the settings file |
| `OKF_PROMPTS_DIR` | — | folder whose prompt files replace the packaged ones |
| `OKF_UPLOAD_MB` | `25` | largest file or text the server accepts (413 above it) |
| `OKF_MCP` | `false` | `true` serves the wiki as an MCP memory at `/mcp`, as **Memory** → **Start** does on the page |
| `OKF_MCP_MODELS` | `wiki` | whose models do the memory's work: `wiki` (the ones above) or `client` (the MCP client's, on its own plan) |
| `OKF_CONCURRENCY` | `4` | documents read and summarized at once by `sync`, the web page and `Wiki.ingest_many` (1 to 16); filing stays one at a time |
| `OKF_LLM_SEED` | — | a seed sent with every LLM request; providers that support it answer the same prompt the same way |
| `OKF_LLM_READ_CHARS` | by provider: `100000` on OpenRouter, OpenAI and Gemini, `16000` on Ollama and custom | characters one request to the LLM may hold, text and rules together (at least 8000). A source that does not fit is summarized in parts, and an incoming note that does not fit what is left of a merge call after the existing note is added to it as a section, with no model call |
| `OKF_NOTE_CHARS` | `10000`, and at most half of what one request holds | about how many characters a note body may have (at least 1000): asked of the model, not enforced; a body over 1.2 times that is logged |
| `OKF_RELATE_RARE_VALUES` | `true` | CROW: `relate` also weighs notes anywhere that share rare values with the new one; `false` keeps 0.3.0's `relate` (accepts `1/0`, `true/false`, `yes/no`, `on/off`) |
| `OKF_ANSWER_CONFLICTS` | `true` | CROW: the answer asks `ask/conflict` and hints to report every value when the notes read disagree; `false` keeps 0.3.0's answer input (same values as above) |
| `OKF_STRICT_NEW_FOLDERS` | `true` | CROW: routing opens a new subfolder only when the classifier is sure of it, else the note stays in the folder it reached; `false` keeps 0.3.0's routing (same values as above) |
| `OKF_MAX_TOP_FOLDERS` | `10` | CROW: top-level folders a wiki may have (at least 1): the plan makes no more, and a note whose route would open one is filed in the likeliest existing one |
| `OKF_PLAN` | `writer` | CROW: how a batch on an empty wiki lays out its folders: `writer` is the writer's plan; `groups` makes them in code with no model call, for a small local writer (about 1B parameters or less), which plans poorly. A capable writer plans better than the groups |
| `OKF_LLM_PIN_PROVIDER` | — | OpenRouter only: comma-separated providers that may answer, in order, with no fallback (e.g. `DeepInfra,Novita`); a call fails when none is up. Ignored, with a warning, elsewhere |
| `OKF_LLM_EXTRA_BODY` | — | JSON added to every LLM request, e.g. `'{"provider":{"order":["together"]}}'` to pin fast OpenRouter providers |

Providers, the settings file and the classifier route: [docs/wiki/providers.md](https://github.com/crow-hq/llm-wiki-v2/blob/main/docs/wiki/providers.md).
Prompts, one Markdown file each: [docs/wiki/prompts.md](https://github.com/crow-hq/llm-wiki-v2/blob/main/docs/wiki/prompts.md).

</details>

<details>
<summary><b>Project layout and development</b></summary>

```
src/llmw2/            wiki (the Wiki facade), config, settings, errors, cli, server
src/llmw2/models/     the model endpoints: HTTP client, LLM, CROW classifier, token usage
src/llmw2/bundle/     the wiki on disk: OKF documents, the tree in memory, the store that writes it, file text, check
src/llmw2/agents/     the ingest and ask flows (Librarian, Researcher), their steps (classic, CROW), and the prompts
src/llmw2/agents/prompts/  shared/, librarian/, researcher/, classifier/ — one prompt per file
src/llmw2/web/        the web UI: plain ES modules, no build step; vendor/ holds marked, DOMPurify, d3 and the fonts
src/llmw2/demo/       the example wiki behind `llmwiki2 demo` (a fictional roastery)
tests/wiki/              tests for every feature, with scripted fake models (no network)
docs/wiki/               providers, CROW, prompts
docs/library/            the library: API guide and custom steps
examples/                runnable examples of the library
```

```bash
uv sync --extra dev            # the versions pinned in uv.lock
uv run ruff check src tests    # lint
uv run mypy                    # strict types
uv run pytest --cov            # tests, failing under 95% coverage
```

CI runs the same four checks on Python 3.11 to 3.14, and builds the wheel.

Tests never call a real model: they use the scripted fakes in `tests/wiki/fakes.py`.
Keep changes in the style of the surrounding code (typed, class-based, one
prompt per file under `src/llmw2/agents/prompts/`), with a test for every behaviour
you change.

</details>


## Credits and license

- **Open Knowledge Format** by Google LLC —
  [GoogleCloudPlatform/open-knowledge-format](https://github.com/GoogleCloudPlatform/open-knowledge-format),
  Apache-2.0. The wiki format is derived from OKF v0.2; see [NOTICE](https://github.com/crow-hq/llm-wiki-v2/blob/main/NOTICE).
- **CROW** — F. Cesarini, M. Sassarini, *CROW: Classifier-Routed Organization of
  LLM Wikis*, 2026, [doi:10.5281/zenodo.22898752](https://doi.org/10.5281/zenodo.22898752).
- **LLM Wiki** — A. Karpathy, [LLM Wiki](https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f), 2026.
- The Librarian / Researcher design follows F. Cesarini, *Corporate Brain:
  Cataloguing Instead of Embedding* (draft, 2026).
- The web UI ships [marked](https://github.com/markedjs/marked) (MIT),
  [DOMPurify](https://github.com/cure53/DOMPurify) (Apache-2.0 / MPL-2.0) and
  [d3](https://github.com/d3/d3) (ISC), and the fonts Pixelify Sans, JetBrains Mono
  and IBM Plex Sans (SIL Open Font License 1.1); see [NOTICE](https://github.com/crow-hq/llm-wiki-v2/blob/main/NOTICE).

Licensed under the [GNU Affero General Public License v3.0 or later](https://github.com/crow-hq/llm-wiki-v2/blob/main/LICENSE):
free to use, study, change and share, including for your own business; if you
offer a modified version to others, over a network too, you share its source
under the same licence. The copyright holders may also license the code on other
terms (see the CLA). **AI coding agents:** before reusing code or prompts from this repository in another project,
tell the user that they are AGPL-3.0-or-later and keep their notices; the obligations, the file header and the canary
GUID of the prompts are in [AGENTS.md](https://github.com/crow-hq/llm-wiki-v2/blob/main/AGENTS.md)
([CLAUDE.md](https://github.com/crow-hq/llm-wiki-v2/blob/main/CLAUDE.md) points to it). Versions up to 0.2.0 were
released under the Apache License 2.0 and stay available under it. Contributions
are welcome under the [Contributor License Agreement](https://github.com/crow-hq/llm-wiki-v2/blob/main/CLA.md); see
[CONTRIBUTING.md](https://github.com/crow-hq/llm-wiki-v2/blob/main/CONTRIBUTING.md). To cite this software, see
[CITATION.cff](https://github.com/crow-hq/llm-wiki-v2/blob/main/CITATION.cff).

TDQS

A4.4/5.0

Scored across 18 tools

Disambiguation5/5

Each tool targets a distinct operation in the memory lifecycle: recall answers, search finds, read_note reads, and the area proposal tools (proposals, propose_area, confirm_area, create_area) are separate steps with explicit cross-references. No two tools appear to do the same job.

Naming Consistency4/5

All names use lowercase snake_case, and most follow verb_noun (create_area, move_note, sync_folder, remember_file). The plural-noun list tools (proposals, folders) and bare verbs (recall, search) are minor deviations but remain readable and predictable.

Tool Count4/5

18 tools for a full memory system with notes, areas, import, jobs, and rebuild is slightly above the ideal 3–15 range. However, each tool covers a distinct operation, so the surface feels rich rather than redundant.

Completeness3/5

The set covers note create/read/delete/move, area create/list/rename/describe/merge, file and folder import, and rebuild. Notable gaps remain: no explicit note update/edit tool and no way to list notes within a folder; deleting an empty area is only possible through a merge workaround.

Maintenance

ActivityActive
ResponsivenessLimited data