# docrag

Local RAG over folder trees of markdown documents. One persistent index, two
front ends: a CLI research loop backed by a local Ollama model, and an MCP
server that exposes the same index to Claude Code.

Everything runs on-device. No network egress at query time.

See [docs/project-plan.md](docs/project-plan.md) for the full design.

**Status:** Phase 1 complete — chunker, store, embedder, indexer, and the
`index` / `status` commands. `search` and `ask` land in Phase 2; the MCP server
in Phase 3.

## Setup

Requires [Ollama](https://ollama.com) and [uv](https://docs.astral.sh/uv/).

```bash
ollama pull nomic-embed-text
uv sync
```

Point `docrag.toml` at the markdown trees you want indexed, then:

```bash
uv run docrag index
```

Re-run it whenever the corpus changes — only files whose content hash moved are
re-embedded, so a no-op run costs about a tenth of a second per 200 files.

```bash
uv run docrag status --pending
```

## Commands

| Command | What it does |
|---|---|
| `docrag index` | Incremental index of every configured root |
| `docrag index --root NAME` | Limit to one root (repeatable); other roots are left untouched |
| `docrag index --force` | Re-embed everything, ignoring content hashes |
| `docrag index --dry-run` | Report what would change, write nothing |
| `docrag index --reset` | Drop the collection first — use after changing chunk params or the embedding model |
| `docrag index -v` | Print each added/changed file |
| `docrag status` | Roots, file counts, chunk counts, last index time, orphaned roots |
| `docrag status --pending` | Also report how many files an index run would touch |

## Configuration

All tunables live in `docrag.toml`. Paths are resolved relative to that file,
so the commands work from any directory in the tree.

```toml
[[roots]]              # repeatable; at least one required
name = "docs"          # optional, defaults to the directory name
path = "./docs"        # relative to docrag.toml, or absolute; ~ is expanded

[chroma]
path = ".chroma"       # where the persistent index lives
collection = "docrag"  # 3-512 chars, [a-zA-Z0-9._-]

[models]
embed = "nomic-embed-text"
generate = "qwen3:8b"  # unused until Phase 2

[embedding]
batch_size = 32
document_prefix = "search_document: "   # nomic task prefixes; "" for other models
query_prefix = "search_query: "

[chunking]
max_chunk_chars = 1500  # hard cap on chunk length
overlap_chars = 200     # carried from the tail of the previous part when a section splits
min_chunk_chars = 40    # a trailing sliver smaller than this folds into the previous part

[files]
extensions = [".md", ".markdown"]
exclude = ["**/.git/**", "**/node_modules/**"]

[ollama]
host = "http://localhost:11434"  # optional; omit to use the ollama client default
```

Model tags live only here (DD6) — nothing is hardcoded. Verify current tags on
ollama.com/library before changing them.

## How indexing works

- **Chunks are heading sections.** A chunk is a heading plus everything under it
  up to the next heading of any level. Sections over `max_chunk_chars` split
  again on blank-line boundaries, with `overlap_chars` carried across the seam.
  Fenced code blocks are never split; one too large to fit is cut on line
  boundaries with each piece re-fenced.
- **Every chunk is self-locating.** The stored text is prefixed with a
  breadcrumb — `project-plan.md › 4. Key Design Decisions › Open Questions` — so
  a retrieved chunk names its own source without a second lookup.
- **The diff is content-based.** Each file's SHA-256 (over BOM- and
  EOL-normalized text) is stored on its chunks. Unchanged hash, no work;
  changed hash, that file's chunks are dropped and rebuilt; gone from disk, its
  chunks are purged. `docrag index` is idempotent and cheap to run often.
- **Chunk ids are deterministic:** `{root}/{relpath}#{ordinal}`.

Measured throughput and chunk shape: [docs/phase1-baseline.md](docs/phase1-baseline.md).

## Development

```bash
uv run pytest
```

Chunking edge cases are pinned in `tests/test_chunker.py` (code fences, indented
code, setext headings, front matter, oversized sections); the incremental diff
in `tests/test_indexer.py`.