Ken

A frontend-free memory layer that turns any folder of documents into a searchable knowledge graph. Ken converts, ingests, links, and retrieves your files locally, with no cloud and no runtime dependencies on external repositories.

Quick Start

Install Ken globally (or run it on demand with npx):

npm install -g @foresights/ken
# or
npx @foresights/ken <command>

Turn any folder into a vault. Files are never moved, copied, or renamed. Ken only creates a .ken/ directory inside the folder.

ken init ~/Documents/my-notes

Search it:

ken search "quarterly revenue trends"

Results come back in two flavors: DIRECT hits (text match) and GRAPH hits (linked neurons surfaced by Personalized PageRank).

Related MCP server: RAG MCP Server

Claude Desktop Integration

Ken speaks the Model Context Protocol (MCP) over stdio. Add it to your Claude Desktop config file.

macOS: ~/Library/Application Support/Claude/claude_desktop_config.json Windows: %APPDATA%\Claude\claude_desktop_config.json

{
  "mcpServers": {
    "ken": {
      "command": "npx",
      "args": ["-y", "@foresights/ken", "mcp", "--vault", "/path/to/vault"]
    }
  }
}

Replace /path/to/vault with the absolute path to your Ken vault folder. Restart Claude Desktop and Ken's tool catalog becomes available to the agent.

CLI Commands

How It Works

Ken has three layers working together.

1. Rust conversion core. A native Rust binary converts 11 document formats to Markdown. No system calls to Python, LibreOffice, or external tools. The binary ships inside the npm package.

2. TypeScript service layer. The orchestration logic (ingestion, search, graph algorithms, memory lifecycle, MCP server, HTTP server) is written in TypeScript and runs on Node.js.

3. SQLite storage. Every vault stores its data in a single .ken/data.db SQLite file. Full-text search uses SQLite FTS5. Graph structure, embeddings, FSRS state, and task queues all live in the same database.

The retrieval model is graph-first. When you search, Ken does not just match text. It follows synapses (links between neurons) to surface related knowledge that may not share any keywords with your query.

Search Architecture

Ken search runs four layers in sequence. Each layer adds recall without sacrificing precision.

1. FTS5 full-text search. The base layer. SQLite FTS5 matches your query tokens against neuron content with BM25 ranking. This is always available and requires no model.

2. PMI query expansion. Ken builds a Pointwise Mutual Information co-occurrence matrix from all neuron content. When you search, it expands your query terms with their statistical neighbors. Searching "revenue" also hits neurons about "income", "ARR", "MRR". Run ken reindex-pmi to rebuild this matrix.

3. Personalized PageRank. Seed neurons from layers 1 and 2 become the personalization vector for a PageRank walk over the synapse graph. High-authority neurons connected to your seeds get boosted, even if they had no direct text match. This is what produces the GRAPH results.

4. Semantic embeddings (optional). If you run ken semantic build, Ken computes LSI word similarities and node2vec graph embeddings using a bundled Python script. This adds latent semantic matching on top of the graph. Requires Python with numpy. Ken works fully without it.

Memory Lifecycle

Ken treats knowledge like biological memory, not like a static index.

• FSRS spaced repetition. Every neuron has an FSRS (Free Spaced Repetition Scheduler) state tracking its stability and retrievability. When you recall a neuron (via ken recall or through the MCP tools), its stability increases and its next review date moves out. Neurons you never recall gradually become stale.

• Hebbian strengthening. Synapses that get traversed frequently during search and recall get stronger. Infrequently used synapses decay. Run ken consolidate to apply this.

• Nightly consolidation. The consolidation worker (ken consolidate) does three things: (1) replays high-contribution neurons to boost their stability, (2) marks neurons whose retrievability has dropped below the threshold for review, and (3) prunes decayed synapses and edges. This keeps the vault from accumulating dead weight.

• Observation and synthesis. The ken observe command clusters related neurons and presents them for synthesis. The agent (or you) can then create new neurons that capture higher-level patterns.

• Self-improvement proposals. Ken generates typed metric facts with provenance and can detect contradictions. It proposes improvements (new synapses, entity merges, schema updates) as tasks that require human approval before they take effect.

Supported File Formats

Native Rust converters (11 formats)

Plain text formats (read directly)

Markdown (.md, .markdown), plain text (.txt), JSON (.json), YAML (.yaml, .yml), XML (.xml), HTML (.html, .htm).

Code file formats (40+, read as plain text)

TypeScript (.ts), JavaScript (.js, .mjs, .cjs), Python (.py), Rust (.rs), Go (.go), Java (.java), C (.c, .h), C++ (.cpp, .cc, .cxx, .hpp, .hh), C# (.cs), Ruby (.rb), PHP (.php), Swift (.swift), Kotlin (.kt), Scala (.scala), Shell (.sh, .bash, .zsh), SQL (.sql), Lua (.lua), Perl (.pl, .pm), R (.r, .R), Julia (.jl), Dart (.dart), Clojure (.clj), Haskell (.hs), Erlang (.erl), Elixir (.ex, .exs), F# (.fs), Lisp (.lisp, .cl), Nim (.nim), Zig (.zig), Crystal (.cr), OCaml (.ml), Vue (.vue), Svelte (.svelte), CSS (.css), SCSS (.scss), Less (.less), TOML (.toml), INI (.ini, .cfg), Dockerfile, Makefile, and more.

Self-Contained

Ken has zero runtime dependencies on external repositories, cloud services, or API keys.

• Works offline. No network calls are required for ingestion, conversion, search, or graph analysis.

• No cloud. Your documents never leave your machine. Everything runs locally.

• No external repos. The Rust conversion binary ships inside the npm package. The optional semantic layer bundles its own Python script.

• No API keys. Ken does not call OpenAI, Anthropic, or any external model. The optional local LLM integration (ken models) talks to your own Ollama, LM Studio, or llama.cpp instance on localhost.

• Single database file. The entire vault (content, graph, embeddings, FSRS state, task queue) lives in one SQLite file at .ken/data.db.

License

MIT