# DocGraph

[![M8ven Score](https://m8ven.ai/badge/mcp/nettenz-docgraph-1bna4l?v=db56bd3c20a931839bc0ff8bbbf4f420)](https://m8ven.ai/mcp/nettenz-docgraph-1bna4l)
[![Python](https://img.shields.io/badge/python-3.10%2B-blue)](pyproject.toml)
[![MCP](https://img.shields.io/badge/protocol-MCP-5A67D8)](https://modelcontextprotocol.io)
[![SQLite FTS5](https://img.shields.io/badge/index-SQLite%20FTS5-003B57)](docs)
[![License](https://img.shields.io/badge/license-none%20specified-lightgrey)](#license)

Repo-native markdown context broker — an MCP tool that gives coding agents
task-relevant docs instead of dumping `docs/**`.

Point it at a repo, and Claude Code (or any MCP client) gets a single tool,
`docgraph_context(task, max_tokens)`, that turns a task description into a
ranked, token-budgeted markdown pack pulled from that repo's own
documentation — instead of reading whole files wholesale and hoping the
relevant part is in there somewhere.

## Why

Agent context windows are finite and doc trees aren't curated for retrieval.
"Read `docs/**`" either blows the budget on a big repo or silently misses
files outside `docs/`. DocGraph indexes what's actually documentation
(skills, monorepo subproject READMEs, loose root files — not just `docs/`),
splits long catalog-style files into their real sections, and serves back
only what a specific task needs.

No embeddings, no LLM calls in the retrieval path. Deterministic and
inspectable — you can always see *why* a doc made it into a pack.

## How it works

```plaintext
repo markdown
     │
     ▼
discover.py    4-bucket rule: root files, docs/, skills/, monorepo
     │         subproject READMEs (all-caps filename, one level deep)
     ▼
index.py       SQLite + FTS5 (porter stemming), recursive H2→H4 chunking,
     │         content-hash dedup; link, code-reference, and symbol edges
     ▼
db/docgraph.db
     │
     ▼
context.py     task → AND-first/OR-fallback FTS query → co-location, link,
     │         then code-reference expansion → token-budget trim
     ├─────────────────────────────┐
     ▼                             ▼
mcp_server.py                  serve.py       live web graph: task box ->
wraps it as one                real retrieval -> highlighted nodes +
MCP tool, stdio                rendered markdown pack panel
transport
```

## Install

```bash
pip install -e .
```

## Usage

```bash
# Build the index for a repo
python -m docgraph.index /path/to/repo db/my-repo.db

# Generate a context pack directly (useful for testing before wiring into an agent)
python -m docgraph.context /path/to/repo db/my-repo.db "task description" --max-tokens 8000

# Run as an MCP server (stdio) — point your MCP client's config at this
python -m docgraph.mcp_server /path/to/repo db/my-repo.db

# Simple graph visualization (file-level nodes, co-location edges) — static, no server
python -m docgraph.visualize db/my-repo.db graphs/my-repo_graph.html --title "my-repo"

# Live web graph: same corpus graph, plus a task box that runs real retrieval
# and highlights exactly which files were selected, with the rendered pack
# in a resizable side panel
python -m docgraph.serve /path/to/repo db/my-repo.db --port 8765
```

Task strings are used as keyword search, not semantic search — be specific,
and avoid naming a file you're about to create (it can't match anything
that doesn't exist yet).

### Index freshness

Indexes store metadata and hashes, not source bodies. Before retrieval,
DocGraph compares every indexed source file with its stored hash and fails
closed if any source changed, disappeared, or cannot be read. Rebuild after
source changes rather than accepting a silently wrong section or whole-file
fallback:

```bash
python -m docgraph.index /path/to/repo db/my-repo.db
```

### Registering with Claude Code

```bash
claude mcp add my-repo-docs -s user -e PYTHONIOENCODING=utf-8 -- \
  python -m docgraph.mcp_server /path/to/repo /full/path/to/db/my-repo.db
```

One server instance = one repo + one index. For multiple repos, register
multiple servers with distinct names and separate `.db` files.

**Query logging.** `context.retrieve()` supports an opt-in, append-only
JSONL query log via the `DOCGRAPH_QUERY_LOG` env var (unset by default —
see `docs/V4_NEXT_STEPS.md`'s validation-gate entry for why it exists). The
MCP server is a process Claude Code spawns, not a child of your interactive
shell, so exporting `DOCGRAPH_QUERY_LOG` in a terminal has no effect on it —
it must be passed via `-e` at registration time:

```bash
claude mcp add my-repo-docs -s user \
  -e PYTHONIOENCODING=utf-8 \
  -e DOCGRAPH_QUERY_LOG=/absolute/path/outside/any/indexed/repo/query_log.jsonl \
  -- python -m docgraph.mcp_server /path/to/repo /full/path/to/db/my-repo.db
```

Re-registering without `-e DOCGRAPH_QUERY_LOG=...` silently produces an
empty log, not an error — after registering, run a couple of real tasks
through the tool and confirm the file actually has entries before trusting
a longer stretch of silence. Keep the path outside any repo `docgraph`
indexes (a log inside an indexed repo will contain the next query's task
string verbatim and self-match on the next reindex).

## Live web graph (`serve.py`)

```bash
python -m docgraph.serve /path/to/repo db/my-repo.db --port 8765 [--host 127.0.0.1]
```

The same force-directed corpus graph as `visualize.py`, served locally
(stdlib `http.server`, no new dependency) with:

- A task box that calls real retrieval (`context.retrieve`) and highlights
  which file nodes were actually selected into the pack — solid glow for
  seed matches, dashed for co-location neighbors pulled in via expansion —
  with link and code-reference chunks represented in the pack. Referenced
  code files appear as extension-colored nodes; purple `code_ref` edges are
  directional from documentation to code.
- A resizable side panel rendering the selected pack as formatted markdown
  (headings, code blocks, tables — via `marked`, sanitized by DOMPurify).
  Drag its left edge or click the `⤢` button to expand it.
- A status line under the box showing chunk count, and graceful empty-state
  when a task matches nothing.

`GET /context?task=...&max_tokens=...` is the underlying JSON endpoint if you
want to hit it directly. Local dev tool only — no auth, binds `127.0.0.1` by
default.

## Discovery rule

- **root** — loose `.md` files directly at repo root
- **docs** — anything under a directory named `docs`, any depth
- **skills** — same, for a directory named `skills` (catches
  `.claude/skills/` and `.agents/skills/`)
- **subdir-allcaps** — files exactly one level under root, in another
  subdirectory, whose filename stem is ALL-CAPS (`README`, `TODO`,
  `ARCHITECTURE`...) — covers monorepo subproject meta-docs

Any bucket can be excluded per-run with `--exclude-bucket`.

## Design notes

- **FTS5 with porter stemming, no embeddings.** Deterministic, cheap, and
  good enough for retrieval seeding.
- **Co-location edges.** Files in the same directory get a weak "related"
  edge. Capped at 10 files per directory — past that, "same folder" stops
  being a meaningful relationship and starts being noise.
- **Link edges (V2).** Raw markdown-link coverage tested near-zero across
  the first three audited repos, so V1 shipped without them. A follow-up
  audit asked a narrower question — among the links that *do* exist, how
  many connect content sharing no vocabulary with each other — and found
  real (if thin, hub-concentrated) signal, so `kind='link'` edges were
  added: directional (unlike co-location), doc-level, and pulled into
  `retrieve()` unconditionally (no FTS gate) as the lowest-ranked tier,
  below every seed and co-location neighbor. A per-doc fan-out cap drops
  *all* link edges from a hub doc (an INDEX.md linking to everything)
  rather than truncating an arbitrary subset.
- **Code references and symbols (V3/V4).** Backticked code filenames and
  fenced code snippets create directional `code_ref` edges to real source
  files. An inline-backticked symbol can refine a Python target to a def/class
  chunk and add bounded intra-file symbol neighbors. Every fan-out cap is
  skip-not-truncate. Python is AST-sliced; JS/TS/Go/Rust remain deliberate
  whole-file references until language-specific parsers are added.
- **Recursive chunking, not fixed-depth.** Long docs split at H2; any
  section still oversized with real substructure splits again at H3, then
  H4. Some repos have flat catalogs of H2 sections, others have one
  catch-all H2 hiding the real structure at H3 — fixed depth is wrong for
  one of them either way.
- **AND-first, OR-fallback queries.** Try requiring every query word to
  co-occur first; only widen to OR if that finds nothing. A single precise
  match is better evidence than several noisy ones.
- **Content-hash dedup at index time.** Mirrored files (e.g. a skill
  duplicated under `.claude/` and `.agents/`) get indexed once, not twice.

## Status

MVP, validated against three real repos of different shapes (10, 8, and
72-file corpora) and in live use via Claude Code. The live web graph
(`serve.py`) closes the "real graph UI" gap — file-level highlighting only
for now, chunk/heading-level nodes deferred. Not built: embeddings, watch
mode, cross-repo search.

## License

Personal project, no license specified.