intent-index

Local-first retrieval over a long design conversation, for auditing an implementation against what was actually decided.

The problem

A long design conversation is the most accurate record of a system that exists, and it is the least usable. Decisions are stated once, in the middle of a turn, in prose. Months later an engineer asks three questions and gets three different answers:

• Where in the conversation was this decided?

• Why was it decided that way, and what was rejected instead?

• Has it changed since?

A file-based search answers none of them. It finds the word, not the ruling. It cannot tell you that the ruling you are reading was replaced in turn 26.

Related MCP server: MemPalace

What it does

One pipeline, three mechanisms, each kept in its lane.

Everything a reviewer receives carries a turn number and a line range. A citation without coordinates cannot be checked, so an item without coordinates is not returned as evidence.

Quick start

Requirements: mise and pnpm. The toolchain is pinned in mise.toml; dependencies install from the committed pnpm-lock.yaml.

mise install
pnpm install
pnpm build

Build the index over the sample conversation in examples/. This needs no credentials:

node dist/cli.js build --no-generative
pnpm doctor
pnpm eval

That gives you sections, a hybrid index, and a manifest. Add a generative pass to extract decision records and write the intent map:

export OPENROUTER_API_KEY=...
node dist/cli.js build

Point intent-index.yml at your own transcript to index a real conversation. Nothing in the configuration references a path outside the repository.

Ask it things

node dist/cli.js retrieve "why was the reorder buffer rejected"
node dist/cli.js grep "quarantine lane"
node dist/cli.js read --turn 6
node dist/cli.js read --turn 6 --context 4
node dist/cli.js trace D0010
node dist/cli.js overview
node dist/cli.js doctor
node dist/cli.js eval

trace is the one worth understanding. It walks a record's history: what it supersedes, what supersedes it, and the transcript lines behind each step. When a ruling changes, the correction is visible rather than implied.

grep reads the transcript and nothing else. Generated records and the intent map are navigation aids, so they are excluded from exact search unless you ask for them by name. If a generated artifact and the transcript disagree, the transcript is right.

Add --json to any command for machine-readable output, and --dir <path> to operate on another project directory.

Tools over MCP

intent-index mcp serves five tools over stdio:

Registration for a harness that speaks MCP:

{
  "mcpServers": {
    "intent-index": {
      "command": "node",
      "args": ["/absolute/path/to/intent-index/dist/cli.js", "mcp"],
      "env": { "TYPESAFE_API_KEY": "..." }
    }
  }
}

What gets indexed

Every document is a small markdown file with typed metadata the engine can filter on.

Sections come from the transcript's own structure: turn headings, pseudo headings, and paragraph blocks grouped to a target size. No model decides where a boundary falls, so a rebuild reproduces the same identifiers and line ranges byte for byte.

Configuration

corpora:
  - name: atlas
    path: examples/atlas-transcript.md
    role: conversation      # conversation | documents
    ledger: true            # extract records and an intent map for this corpus

generative:
  baseUrl: https://openrouter.ai/api/v1
  model: deepseek/deepseek-v4.1-flash
  apiKeyEnv: OPENROUTER_API_KEY
  reasoning: false

judgment:
  model: jev-latest
  apiKeyEnv: TYPESAFE_API_KEY
  rerank: true

retrieval:
  limit: 10
  candidateLimit: 40

Generative calls are cached by a hash of their inputs under .intent-index/cache/generative, so a rebuild replays identical artifacts instead of paying for them twice.

Three settings matter for reliability on a long transcript:

• concurrency caps calls in flight. 8 is what the measured throughput supports.

• reasoning is off. These passes are mechanical extraction. When the model spends output tokens reasoning first, it exhausts its budget before writing the structured answer and the provider returns an empty completion.

• Turn text is windowed to 6000 characters per call, so one oversized turn cannot fail as a single request.

Structured output is schema-validated, so a reply that does not match the requested shape fails the call instead of entering the ledger. A window whose generation fails is reported by name and left out of the ledger; it does not fail the build silently.

The intent map is written one section at a time and each section is checked for its heading, its content, and a citation on every bullet. A section that cannot be written fails the build, because a shorter map that looks complete is worse than an error.

Verification

pnpm eval runs probes whose evidence location is already known: each probe names the turns that carry the answer and, where applicable, a literal phrase that must exist there.

The second row is a real corpus that is not shipped with this repository. It is listed because a sample corpus is easy to be good at, and a 7,971-line conversation is not.

doctor checks the index against its own manifest: source files still hash to what was recorded, every document on disk matches its digest, no stray documents exist, every record carries a turn, all ten intent map sections are present, and the engine sees the same document count the build reported.

ok   manifest: 72 artifacts
ok   sources: 1 sources unchanged
ok   documents present: 72 files
ok   no stray documents: none
ok   document digests: all match
ok   decision turn coordinates: 49 records carry a turn
ok   intent sections: 10 sections
ok   index document count: 72 indexed, 72 expected

pnpm check typechecks and pnpm lint lints. A change is done when those pass, doctor is green, and eval has not regressed.

Requirements

• Node 24, pinned through mise.toml.

• OPENROUTER_API_KEY for the passes that extract records and write the intent map.

• TYPESAFE_API_KEY for the judgment rerank.

Both are read from the environment. Put them in .env beside intent-index.yml; that file is git-ignored and .env.example is the committed template. Retrieval, exact search, reading, tracing, and a structure-only build work without either key.

The index is a local SQLite file under .intent-index/, alongside the rendered documents and the manifest. It is disposable build output: delete it and rebuild. Nothing is sent anywhere except the generative and judgment calls you configure, and those carry the transcript windows they are given.

Layout

src/corpus/       normalization: transcript to turns and sections
src/index/        document rendering and the build pipeline
src/ledger/       record extraction, turn digests, intent map
src/judgment/     typed judgment calls over a retrieval shortlist
src/retrieval/    query planning, retrieval, exact search, reading, tracing
src/mcp/          the five tools over stdio
src/eval/         probe runner
docs/diagrams/    the diagram above, as SVG
docs/design-decisions.md   every decision this system was built from
examples/         the sample conversation and its probe set

docs/design-decisions.md records each choice behind the architecture with the question, the criteria, and the answer it received. It is the design authority for this repository: a change that contradicts a recorded decision is a decision to revisit, not an implementation detail.

License

MIT. See LICENSE.