# agent-wiki

**The knowledge base that makes AI agents smarter over time.**

Instead of retrieving raw fragments every query (RAG), your agent compiles, refines, and interlinks knowledge — like a team wiki that writes itself.

Works with Claude Code, Cursor, Windsurf, and any MCP client. Also installable as a native skill for Claude Code. No LLM built in — your agent IS the intelligence.

agent-wiki is also evolving into the reference implementation of **Open Knowledge Format (OKF)**: a Git-native package format for agent-maintained knowledge with immutable source evidence, mutable compiled pages, schemas, indexes, evidence metadata, and machine-checkable governance.

[![npm](https://img.shields.io/npm/v/@agent-wiki/mcp?label=%40agent-wiki%2Fmcp)](https://www.npmjs.com/package/@agent-wiki/mcp)
[![CI](https://github.com/xinhuagu/agent-wiki/actions/workflows/ci.yml/badge.svg)](https://github.com/xinhuagu/agent-wiki/actions/workflows/ci.yml)
[![Node](https://img.shields.io/badge/node-%3E%3D20-brightgreen)](https://nodejs.org)
[![MCP](https://img.shields.io/badge/protocol-MCP-blue)](https://modelcontextprotocol.io)
[![License: MIT](https://img.shields.io/badge/license-MIT-yellow.svg)](LICENSE)

### agent-wiki's built-in 3D graph view

Pages as nodes, `[[wikilinks]]` as edges, edits push live — included in the main package.

<p align="center">
  <img src="docs/graph-1.gif" alt="agent-wiki realtime 3D knowledge graph viewer — live-updating force-directed graph of Markdown pages and [[wikilinks]]" width="900" />
</p>

## Quick Start

### Option A: MCP Server (Cursor, Windsurf, Claude Desktop, any MCP client)

Add to your MCP client config:

```json
{
  "mcpServers": {
    "agent-wiki": {
      "command": "npx",
      "args": ["-y", "@agent-wiki/mcp", "serve", "--wiki-path", "/path/to/knowledge"]
    }
  }
}
```

### Option B: Native Skill (Claude Code)

```bash
npm install -g @agent-wiki/mcp

# Install as Claude Code plugin
agent-wiki install claude-code
```

### Option C: CLI only

```bash
npx @agent-wiki/mcp call wiki_search '{"query": "deployment"}'
```

### Option D: 3D Graph Viewer

See your wiki as a realtime 3D knowledge graph — edits push live via SSE. Included in the main package, no separate install needed.

```bash
npm install -g @agent-wiki/mcp
agent-wiki web --wiki-path ./wiki --open
```

Heavy browser libs (3d-force-graph, three.js) load from a CDN at runtime. See
[graph-viewer/README.md](graph-viewer/README.md) for the full feature list and
interaction guide.

That's it. Your agent now has a persistent, structured knowledge base.

## Why Not RAG?

| | RAG | agent-wiki |
|---|---|---|
| **Approach** | Retrieve fragments at query time | Build and maintain compiled knowledge |
| **Memory** | Stateless — forgets after each query | Persistent — knowledge accumulates |
| **Quality** | Raw chunks, often noisy | Curated, structured, interlinked |
| **Cost** | Embedding + retrieval every query | One-time compilation, free reads |
| **Contradictions** | Invisible — buried in source docs | Flagged automatically by lint |
| **Source tracking** | Lost after retrieval | Full provenance chain (raw -> wiki) |

## Features

| Feature | Description |
|---------|-------------|
| **Batch Mode** | Generic `batch` tool + semantic pipelines — collapse multi-step workflows into single requests |
| **Knowledge Pipelines** | Unified `knowledge_ingest` modes — end-to-end ingest/digest/write-back loop without expanding the public tool surface |
| **Structured Extraction** | PDF (per-page), DOCX, XLSX (per-sheet), PPTX (per-slide) — segments with source provenance |
| **Immutable Sources** | SHA-256 verified `raw/` layer — write-once, tamper-proof, full provenance |
| **Knowledge Compilation** | Agent builds structured wiki pages from raw sources — not retrieve-and-forget |
| **BM25 Search** | Field-weighted scoring, synonym expansion, fuzzy matching, CJK tokenization — zero LLM |
| **Hybrid Search** | Optional BM25+vector re-ranking via `@xenova/transformers` — enable with one config line, no external API |
| **Auto-Classification** | Zero-LLM heuristic assigns entity types and tags across 10 categories |
| **Multi-Level Indexes** | Auto-generated `index.md` at every directory level — nested topic hierarchies with sub-topic navigation |
| **Self-Checking Lint** | Catches contradictions, broken links, orphan pages, stale content |
| **Coverage Report** | `raw_coverage` tells the agent which raw sources have not yet been compiled into any wiki page — drives active knowledge completion |
| **Atlassian Import** | One-command Confluence pages and Jira issues with full hierarchy. Supports both Atlassian Cloud (`*.atlassian.net`) and self-hosted Server / Data Center, with auto-routed API endpoints and Bearer / Basic auth handling. |
| **File Versioning** | Auto-version same-name files, query latest, list all versions |
| **Language Plugins** | Deterministic parsers + cross-file knowledge graphs for legacy code. **COBOL** shipped with field lineage in three families (shared-copybook reuse, `CALL ... USING` boundary, cross-program DB2 flow), DB2 column-level pairing, dynamic CALL resolution, and a precision / recall eval harness. **JCL** planned. See [Language Plugins](#language-plugins) below. |
| **Skill Install** | One-command install as native skill for Claude Code and compatible clients |
| **Git-Native** | Plain Markdown — diffable, blameable, revertable |
| **Open Knowledge Format (OKF)** | Directional package contract: `agent-wiki.yaml` manifest, immutable `raw/`, mutable `wiki/`, schemas, rebuildable indexes, evidence artifacts, and conformance checks. See [OKF Adoption Plan](docs/okf-adoption-plan.md). |
| **3D Graph Viewer** | Built-in — realtime 3D graph of pages and `[[wikilinks]]`, edits push live over SSE. Run `agent-wiki web`. |

## Architecture

Three immutability layers, inspired by how compilers work:

| Layer | Mutability | Role |
|-------|-----------|------|
| **raw/** | Immutable | Source documents — write-once, SHA-256 verified |
| **wiki/** | Mutable | Compiled knowledge — structured pages that improve over time |
| **schemas/** | Reference | Entity templates — consistent structure across knowledge types |

<p align="center">
  <img src="architecture.svg" alt="agent-wiki architecture" width="700" />
</p>

### Open Knowledge Format (OKF)

The same layers are being formalized as OKF: a portable knowledge package that another agent or tool can inspect without a hosted service.

```text
agent-wiki-package/
  agent-wiki.yaml   # portable package manifest
  raw/              # immutable source evidence
  wiki/             # mutable compiled knowledge
  schemas/          # page/entity contracts
  indexes/          # rebuildable search/graph artifacts
  evidence/         # provenance, confidence, coverage, page classes
  logs/             # optional operational telemetry
```

Important distinction: `.agent-wiki.yaml` remains runtime/operator config; `agent-wiki.yaml` is the portable OKF manifest. OKF v0.1 now has an executable path: `schemas/agent-wiki-okf.schema.json`, `wiki_admin action: "format-check"`, and optional persisted package reports at `evidence/okf-report.json` via `wiki_admin action: "rebuild"` with `okf_report: true`.

See [Open Knowledge Format](docs/open-knowledge-format.md) for the thesis and [OKF Adoption Plan](docs/okf-adoption-plan.md) for the implementation plan and feasibility assessment.

## Design Principles

1. **Raw is immutable** — Source documents are write-once, SHA-256 verified. Ground truth never changes.
2. **Wiki is mutable** — Compiled knowledge improves with every interaction.
3. **No LLM dependency** — Zero API keys, zero cost per operation. Your agent IS the intelligence.
4. **Self-checking** — Lint catches structural issues and flags potential contradictions.
5. **Knowledge compounds** — Every write enriches the whole wiki. Synthesis creates higher-order understanding.
6. **Provenance matters** — Every wiki claim traces back to raw sources.
7. **Git-native** — Plain Markdown. Every change is diffable, blameable, and revertable.

## Integration

| Method | Best For | Setup |
|--------|----------|-------|
| **MCP Server** | Cursor, Windsurf, Claude Desktop, any MCP client | Add to `.mcp.json` |
| **Native Skill** | Claude Code (native plugin) | `agent-wiki install claude-code` |
| **CLI** | Any agent with shell access | `agent-wiki call <tool> '{json}'` |
| **3D Graph Viewer** | Visual exploration of the whole wiki | `agent-wiki web -w ./wiki` |

## Language Plugins

agent-wiki extends to source-code analysis via language plugins —
deterministic parsers + cross-file knowledge graphs, no LLM. Each
plugin emits structured artifacts (`raw/parsed/<lang>/`) and writes
wiki pages with full provenance back to the source files.

| Language | Status | Capabilities |
|----------|--------|--------------|
| **COBOL** | Shipped | AST parser (fixed-format with mainframe alphanumeric sequence areas + free-format). Programs, copybooks, sections, `CALL` (incl. dynamic-call constant propagation), `COPY` / `REPLACING` (incl. via-replacing cohorts and REPLACING-aware inferred matching), `EXEC SQL` (DB2 column-level host-var pairing), `EXEC CICS`, file access modes. Field lineage in three families: shared-copybook reuse (deterministic + inferred), `CALL ... USING` boundary, cross-program DB2 flow. Depth-bounded impact queries via `code_query`. |
| **JCL** | Planned | Job / step / dataset / proc extraction, batch-flow wiki pages, dataset-mediated cross-program lineage. See [PRD Phase 2](docs/legacy-code-knowledge-compiler-prd.md). |

Tier-gate decisions (Phase C precision gates, dynamic-call resolver,
DB2 column pairing) are evaluated against ground-truth fixtures via a
built-in [precision / recall eval harness](docs/field-lineage-eval.md) —
each PR runs against a committed NIST CCVS slice as a corpus-level
regression anchor.

## Hybrid Search Setup

Upgrade from keyword-only to semantic search with two steps:

**1.** Add to `.agent-wiki.yaml`:

```yaml
search:
  hybrid: true
```

**2.** Run `wiki_admin` once to rebuild and embed all pages:

```bash
agent-wiki call wiki_admin '{"action":"rebuild"}'
```

The first run downloads the `Xenova/all-MiniLM-L6-v2` model (~90 MB) from HuggingFace Hub and caches it locally. After that, every `wiki_write` automatically keeps the vector index up to date.

Hybrid mode blends BM25 + cosine similarity scores. If embedding fails for any reason, search falls back to pure BM25 — queries never fail.

See [Search configuration](docs/tools.md#hybrid-bm25vector-search) for weight tuning.

## Documentation

- [MCP Tools (15 public tools) & Entity Types](docs/tools.md)
- [Configuration, CLI & Security](docs/configuration.md)
- [Request Optimization — Batch Digest, Pagination, Context Limits](docs/request-optimization.md)
- [Open Knowledge Format — Git-native knowledge packages for agents](docs/open-knowledge-format.md)
- [OKF Adoption Plan — feasibility, current-state audit, and implementation plan](docs/okf-adoption-plan.md)

## Acknowledgment

Inspired by Andrej Karpathy's [LLM Wiki](https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f) concept — the idea that AI agents should compile and maintain knowledge, not just retrieve raw fragments. This project is an independent, full implementation of that vision.

## License

MIT