okf-ctx

A context handler for LLMs over an Open Knowledge Format bundle.

Your docs stay as markdown in gitthe source of truth. This builds a derived, disposable index over them, serves them to an agent through MCP, and logs every retrieval so you can see which docs get used, which get ignored, and which questions have no answer.

No API key. No embeddings. No network.

Install

pipx install 'okf-ctx[server]'        # recommended; `apt install pipx` first on Debian/Ubuntu

On Debian/Ubuntu, plain pip install into system Python fails with PEP 668 (externally-managed-environment). That's correctuse pipx, uv tool install, or a venv. Don't pass --break-system-packages.

Indexing and search need only PyYAML; the [server] extra pulls the MCP SDK's tree (~28 packages). Skip it if you only want the CLI.

Related MCP server: docs-mcp

Quickstart

Try it in 30 seconds no ingestion, no API key

The repo ships a sample bundle, so you can see it work before pointing it at your own docs:

pipx install 'okf-ctx[server]'
git clone https://github.com/aminHZDEV/OpenContextScitool && cd OpenContextScitool
okf index  --bundle examples/bundle --db /tmp/demo.db
okf search --db /tmp/demo.db how long is a token valid

You get the conflict caveat back the docs disagree on the token TTL, and the tool tells you so. See Example for the output.

Use it on your own project

1. Install and scaffold. From your project root:

   pipx install 'okf-ctx[server]'
   okf init                      # detects ./docs, writes the MCP config + skills/agents

   For a non-Claude client: okf init --client cursor|codex|gemini.

2. Build the bundle from your docs the one step that uses the model:

   # Claude Code:      /okf-ingest
   # any other agent:  okf prompt   > then paste the output into your agent
   okf index                     # turn the concept files into the search index

3. Restart your agent so it loads the MCP server. It now has four tools: search, read, links, report.

4. Ask it questions. It searches the bundle instead of paging whole documents into context.

5. Maintain it once real usage has accumulated this is the half that pays off:

   okf report                    # what's failing: gaps, oversold docs, contradictions
   # Claude Code:      /okf-curator
   # any other agent:  okf prompt --curate
   okf dashboard                 # optional: the same report as a browser page

The whole loop is ingest → index → serve → report → curate. Below okf init and each command has its own section.

Should you use this at all? Below ~30 documents, grep beats it see When not to use this. It earns its place on large, changing prose corpora that get queried repeatedly.

Use

okf check  --bundle ./bundle                      # validate against ingest-context.md §12
okf index  --bundle ./bundle --db ./.okf/index.db # build; unchanged files are skipped
okf search --db ./.okf/index.db rango api key     # BM25 over title/aliases/tags/description/body
okf render --bundle ./bundle --db ./.okf/index.db # fill <!-- concepts:auto --> in index.md

Wire the MCP server into Claude Code:

{ "mcpServers": { "okf": {
    "command": "okf-serve",
    "args": ["--bundle", "/abs/path/bundle", "--db", "/abs/path/.okf/index.db"] } } }

That gives the agent four tools: search, read, links, and report (what's failing in the bundle).

Workflow

The fastest start is okf init, which scaffolds the bundle directory, wires the MCP server into your client's config, andfor Claude Codewrites two skills/agents:

cd your-project
okf init                 # or: okf init --client cursor|codex|gemini|all

Then the lifecycle is two halves, create and maintain:

ingest-context.md is the authoring instruction the create step follows; okf init copies it into the skill so it's self-contained.

The library makes no LLM calls of its own. The model is already on the other end of the MCP connectionokf ingest drives your local claude, and okf prompt / okf prompt --curate just print instructions for any agent to run. No second API client, no API key.

The maintain half is the point: okf report (and the report MCP tool) reads the usage log and names each bad conceptoversold descriptions, missing aliases, gaps, unmarked contradictionsand the curator fixes the markdown, then re-indexes. See Telemetry.

Example

A hand-written sample bundle lives in examples/bundle/ a fictional auth service's docs, 7 concepts, no ingestion required. It's the fastest way to see the format and try the tool:

okf index  --bundle examples/bundle --db /tmp/demo.db
okf search --db /tmp/demo.db how long is a token valid

A concept is just markdown with frontmatter here's caveat-logout-does-not-revoke.md:

---
type: Caveat
title: Logout does not revoke the token
description: 'Logout only clears the client cookie; the bearer token stays valid until it expires, so a copied token keeps working after logout'
tags: [auth, token, logout, revocation, security, gotcha]
aliases: [logout security, token still valid after logout, revoke token]
source: [docs/auth.md#logout]
confidence: high
---
`POST /logout` deletes the client-side cookie only. The session token itself is
**not** added to any revocation list it remains valid until `TOKEN_TTL` elapses…

And the search above returns note that a plain-English question surfaces the conflict caveat, because its aliases carried that phrasing:

1. [ 4.41] Caveat  Docs disagree on the token TTL
   caveat-token-ttl-conflict.md
   The API reference says TOKEN_TTL is 15 minutes; the operations guide says 24 hours…
   …token lifetime  how long is a token valid  TOKEN_TTL value…

The bundle deliberately includes two docs that disagree on the token TTL (token-ttl-api.md says 15 min, token-ttl-ops.md says 24 h) with a Caveat naming the conflict so a reader searching the TTL learns both claims exist and contradict, which a lexical index can't otherwise express. That, and the honesty rules, are what this tool adds over grep.

Relationship to OKF

This tool speaks OKF v0.1 — a directory of markdown files with YAML frontmatter, where the only required field is type. A bundle here is a valid OKF bundle; it renders on GitHub and reads in any editor.

Where it goes further, and why:

So: fully OKF-shaped, with a linter on top and a feedback loop the spec leaves open. A foreign OKF bundle (e.g. Google's data catalogs, with type: BigQuery Table) would need a relaxed check to pass here — the vocabulary is the one deliberate narrowing.

How search works

Keyword (BM25), not semantic. A concept is found only if the searcher's words are literally in its indexed text nothing infers that "churn" and "attrition" are related. That is why description, tags, and aliases are the retrieval layer, and why ingest-context.md spends most of its length on how to write them.

The tradeoff is deliberate: a bad alias is a line of YAML you can read and fix. A bad embedding is a number you can't.

Field weights (descending): title → aliases → tags → description → body.

When not to use this

Below roughly 30 documents, grep is genuinely better and you should not install this. The bundle costs a full ingestion pass to build; that only amortizes if the docs get queried repeatedly.

Telemetry

Every MCP tool call writes to the index. The join between retrieval (what search offered) and read (what the model took) is the pointit distinguishes:

Honest limit: this observes retrieval, not whether the answer was right. A read means a doc was consulted, not that it helped.

Data model

A graph of nodes and typed relationships. OKF has concept linked by edge. The figures below use that project's presentation style, with a real slice of a bundle.

Figure 1 OKF data structure for a real bundle. Blue = concept (the knowledge, with instance values). Green/red = edge (a typed relationship: link or conflicts_with). Arrows read concept → edge → concept.

Figure 2the same edges in the inverse direction. links(path, direction="in") walks edge.dst backward: what points at a concept. This is how a reader arriving at the chat protocol learns which concepts reference and contest.

Database schema

One SQLite file (default .okf/index.db, WAL mode). Canonical DDL is okf_ctx/schema.sql.

Figure 3OKF database schema (core tables). Blue = derived from the bundle (rebuilt by okf index). Green = telemetry (the only non-derived data). Solid arrows are enforced foreign keys; dashed are logical links via path, which are deliberately not FKs because OKF tolerates broken links. meta, source_file, and the concept_fts search index are omitted for clarity.

The two halves behave completely differently, and it matters:

Derived

**concept**one row per non-reserved .md file.

**edge**the link graph. PK (src, dst, kind).

**concept_fts**FTS5 virtual table, porter unicode61. Column order is load-bearing: bm25() weights are positional, and the code passes (10, 8, 5, 3, 1) for title, aliases, tags, description, body. Reorder the columns and you silently reweight search. path is UNINDEXED.

metakey/value. Currently one row: bundle_path, the absolute path the index was built from. The server refuses to start if it doesn't match the bundle it was told to serveotherwise a stale --db answers this project's questions with another project's docs, silently.

History (telemetry)

sessionid (hex), started_at, client. One per server process; a reconnect starts a new one.

**query**one row per search().

**retrieval**what search offered. PK (query_id, concept_path), plus rank, score.

**read**what the model took. id, session_id, ts, concept_path, query_id (NULL = opened without searching).

The join that matters

retrieval and read are separate tables for one reason: offered ≠ taken. That difference is where bad context hides, and a single view-count column cannot express it.

-- concepts search keeps pushing that the model keeps refusing:
-- the description promises what the concept can't deliver
SELECT rt.concept_path, count(*) AS offered,
       sum(rd.id IS NOT NULL) AS taken
FROM retrieval rt
LEFT JOIN read rd ON rd.concept_path = rt.concept_path
                 AND rd.query_id = rt.query_id
GROUP BY rt.concept_path
HAVING taken = 0 AND offered >= 3;

-- read, then searched again => the doc was found and FAILED to answer.
-- Raw view counts score this as a success. Use EXISTS, not JOIN: a JOIN
-- multiplies each read by every later query in the session.
SELECT rd.concept_path, count(*) AS times
FROM read rd
WHERE EXISTS (SELECT 1 FROM query q
              WHERE q.session_id = rd.session_id AND q.id > rd.query_id)
GROUP BY rd.concept_path;

License

MIT