headcleaner convert ~/Documents/inbox --format both --output ~/Documents/inbox.clean

# 1. The Office engine — single binary, no Office install needed
npm install -g @officecli/officecli

# 2. The CLI itself (Python ≥3.12, uv-managed)
uv tool install headcleaner

# Or for development:
git clone <this repo>
cd headcleaner-cli
uv sync
uv run headcleaner --help

headcleaner ~/Documents/inbox --format both --output ./clean

clean/
├── manifest.json                  # run summary: per-file status, engine, sha256
├── REPORT.md                      # count, average time, and error rate by engine
├── _md/                           # plain Markdown (one file per source)
│   ├── notes.docx.md
│   ├── q3.pdf.md
│   └── ...
└── okf/                           # OKF v0.2 bundle (one concept per source)
    ├── index.md                   # auto-generated directory index
    ├── notes.md                   # OKF concept: type=Document
    ├── q3.pdf.md
    └── ...

headcleaner convert <INPUT_DIR> [OPTIONS]

Options:
  -f, --format {md,okf,both}   Output format(s) [default: both]
  -o, --output DIR             Output directory [default: ./out]
  --ocr                        Enable Tesseract OCR for scanned PDFs
  --officecli-timeout <secs>   Timeout per OfficeCLI subprocess call (default: 60)
  --include, -i GLOB           Include glob (may be repeated)
  --exclude, -e GLOB           Exclude glob (may be repeated)
  --jobs, -j N                Parallel worker processes (default: 1 = sequential)
  --no-cache                  Re-convert every file (skip the SHA-256 cache)
  --no-continue-on-error       Stop on the first failure
  --obsidian-compat            Add Obsidian-friendly flat fields to OKF frontmatter
  --clean                       Run the 12-stage heuristic cleanup pipeline (any2md-inspired) on each body
  --tui / --no-tui             Force / disable the animated TUI (default: auto-detect TTY)
  --no-okf-index               Skip OKF directory index.md generation

## Why OKF?

OKF (Open Knowledge Format, v0.2) is just **markdown + YAML frontmatter in a directory hierarchy**. That means:

- Every concept is a single `.md` file you can `cat`, `grep`, edit in any text editor
- Bundles live in git — pull requests, diffs, blame all work
- Obsidian, Notion, MkDocs, Hugo, Jekyll all consume OKF natively
- Required frontmatter key is just `type` — anything beyond that is producer freedom

See [docs/OKF_NOTES.md](docs/OKF_NOTES.md) for the OKF v0.2 specifics this CLI emits.

## Trust stance (honest defaults)

We never auto-claim review. Every emitted OKF concept gets:

- `status: unverified`
- `verified: human:pending`
- `generated: human:<user>@<host>` (OKF §7 actor convention)
- `stale_after: <today + 180d>`
- `sources: [{uri: file://..., sha256: ...}]`

A human can grep `human:pending` later to find concepts needing review. See [docs/OKF_NOTES.md](docs/OKF_NOTES.md) for the full contract.

## Supported formats

See [docs/FORMAT_MATRIX.md](docs/FORMAT_MATRIX.md) for the full engine × library table. At a glance:

| Format | Engine | Library |
|---|---|---|
| `.docx`, `.xlsx`, `.pptx` | OfficeCLI binary | (native DOM) |
| `.pdf` | pdfplumber (text-layer), pytesseract if `--ocr` | pdfplumber / pytesseract |
| `.html`, `.htm` | BeautifulSoup | beautifulsoup4 |
| `.txt` | chardet + read | chardet |
| `.md`, `.markdown` | pass-through + frontmatter inject | stdlib |
| `.csv`, `.tsv` | Sniffer dialect + GFM table (zsv SIMD when installed) | stdlib `csv` (or `zsv` binary) |
| `.json` | pretty-print + fenced block | stdlib `json` |
| `.eml` | headers + text/html body + attachments | stdlib `email` |
| `.epub` | per-chapter HTML → MD | ebooklib (+ bs4 fallback) |
| `.rtf` | control-word stripping | striprtf (+ regex fallback) |
| `.odt`, `.ods`, `.odp` | paragraph/row extraction + GFM tables | odfpy (+ raw-XML fallback) |
| `.msg` | Outlook headers + body + attachments | extract-msg |
| `.pst` | **per-message** (one OKF concept per email) | readpst (libpst) + libpff-python fallback |
| `.docx`, `.xlsx`, `.pptx` | **office_oxide** (primary, ~100x faster), OfficeCLI binary (fallback) | office_oxide 0.1.8 (PyO3) |
| `.ipynb`, `.latex`, `.rst`, sourcecode, `.enex`, `.chm`, etc. (38 formats) | all2md (when installed) | all2md 1.12 |
| `.doc`, `.xls`, `.ppt` | clear error path | needs `libreoffice --convert-to` first |

## Live mode

```bash
headcleaner watch ~/inbox --output ~/out --webhook-url https://hooks.slack.com/...

headcleaner convert ~/inbox --format okf \
    --output ~/Documents/MyVault/Concepts \
    --obsidian-compat

headcleaner review ./out/okf
# Textual TUI: a=approve, r=reject, s=skip, n=next, p=prev, q=quit

headcleaner convert         IN_DIR [flags]    # walk + convert
headcleaner watch           IN_DIR [flags]    # live mode + webhooks
headcleaner review          BUNDLE            # human sign-off TUI/REPL
headcleaner attest          BUNDLE [--private-key PEM]   # Merkle root + optional ed25519 sig
headcleaner verify          BUNDLE [--public-key PEM]    # verify an attestation
headcleaner serve           BUNDLE [--host] [--port]    # local HTTP browser for the bundle
headcleaner glob            DIR               # interactive include REPL (Textual)
headcleaner notion-import   EXPORT.zip OUT    # reverse a Notion workspace export
headcleaner lint            DIR [--fix]       # OKF + MD rule checks
headcleaner doctor          [--output-dir]    # dependency and permission preflight
headcleaner agents          [stdout]          # emit AGENTS.md
headcleaner templates                        # list supported formats

git clone <this repo>
cd headcleaner-cli
uv sync
uv run pytest                # 314 tests, ~14s
uv run headcleaner convert ./tests/fixtures --format both --output ./out

src/headcleaner/
├── walk.py         # recursive folder walker
├── router.py       # extension → engine dispatch
├── normalize.py    # CanonicalDoc + OKF/MD frontmatter builders
├── lint.py         # post-conversion linter (OKF + Markdown)
├── run.py          # pipeline orchestrator
├── cli.py          # Click CLI (headcleaner command)
├── tui.py          # Textual TUI (omp-style)
├── engines/
│   ├── base.py     # Adapter ABC
│   ├── officecli.py
│   ├── pdf.py
│   ├── html.py
│   └── txt.py
└── emit/
    ├── markdown.py
    ├── okf.py
    ├── okf_index.py
    └── manifest.py