# knowledge-forge

Turn a plain list of links (videos, posts, repos, articles) into a **verified,
browsable knowledge base** — and let your AI coding tool enrich it with grounded,
hallucination-checked summaries.

It ships three ways to drive the same pipeline so it works with **any** AI tool:

- a **CLI** (`knowledge-forge ...`)
- an **MCP server** (`knowledge-forge-mcp`) for Cursor, Claude Code, Claude Desktop, etc.
- portable **skills** + editor **hooks**

## What it does

```
links.txt ──▶ extract (yt-dlp / whisper / youtube-transcript)
          ──▶ classify into categories + pull verified learnings
          ──▶ videos.json ──▶ generate a category/detail canvas (.canvas.tsx)
          ──▶ [optional] rule-based enrichment + an agent queue
          ──▶ agent writes 300-1000 word grounded summaries
          ──▶ validate (anti-hallucination gate) ──▶ apply ──▶ refresh
```

Key idea: **summaries are grounded.** A validator enforces word count, per-paragraph
token overlap with the source material, source-URL provenance, and a no-engagement-bait
rule — so the agent can't invent facts, repos, or URLs.

## Install

```bash
pip install "knowledge-forge[all]"     # CLI + MCP + extraction + transcription
# or pick extras: .[extract] .[transcribe] .[mcp]
```

Core has **no third-party dependencies**; the extras add `yt-dlp`,
`youtube-transcript-api`, `faster-whisper`, and `mcp`.

## Quick start

```bash
knowledge-forge init                 # scaffold config + folders in the current dir
#  ... add URLs to links.txt (see samples/links.sample.txt) ...
knowledge-forge update               # extract + classify + build the canvas
knowledge-forge run                  # rule-based enrich + build the agent queue
knowledge-forge queue                # see what needs an agent-written summary
# agent writes payload.json, then:
knowledge-forge validate --file payload.json
knowledge-forge apply    --file payload.json
```

Generate a one-command launcher for your OS:

```bash
knowledge-forge bootstrap --write    # writes kf.ps1+kf.bat (Windows) or kf.sh (macOS/Linux)
```

## Configuration

Everything is path-driven via `knowledge-forge.json` (see `config.example.json`).
All relative paths resolve against the config file's directory — **no machine- or
OS-specific paths are baked in**. The config is auto-discovered by walking up from
the CWD, or set `KNOWLEDGE_FORGE_PROJECT` / `KNOWLEDGE_FORGE_CONFIG`.

| Key | Default | Meaning |
|-----|---------|---------|
| `linksFile` / `linksDir` | `links.txt` / `knowledge-base/links` | inputs |
| `dataFile` | `knowledge-base/videos.json` | the knowledge store |
| `transcriptsDir` | `knowledge-base/transcripts` | cached transcripts |
| `queueFile` | `knowledge-base/.agent-enrich-queue.json` | agent work queue |
| `canvasOutput` | `knowledge-base/canvas/knowledge-base.canvas.tsx` | generated UI |
| `enrichMinSummaryWords` / `MaxSummaryWords` | `300` / `1000` | summary gate |
| `agentEnrichBatchSize` | `5` | entries per agent batch |

## Use it from your AI tool

| Tool | MCP | Hooks | Skills |
|------|-----|-------|--------|
| **Cursor** | `integrations/cursor/mcp.json` | `integrations/cursor/hooks*` | `skills/*` → `.cursor/skills/` |
| **Claude Code** | `integrations/claude-code/.mcp.json` | — | `skills/*` → `.claude/skills/` |
| any MCP client | run `knowledge-forge-mcp` | — | — |

MCP tools: `kf_update`, `kf_refresh`, `kf_run`, `kf_queue`, `kf_list`, `kf_validate`, `kf_apply`.

See `integrations/<tool>/README.md` for copy-paste setup.

## Project layout

```
knowledge_forge/      core package (config, pipeline, classify, verify, enrich, canvas, cli)
mcp_server/           FastMCP stdio server
skills/               tool-agnostic SKILL.md files
integrations/         per-tool hooks + MCP config + setup docs
samples/              anonymized example links + data
config.example.json   starter config
```

## License

MIT — see `LICENSE`.