ddflow

A portable, agent-agnostic work-queue kernel for AI coding agents.

You keep a queue of phases and tasks with declared dependencies. You say "implement phase P2". Independent tasks fan out to parallel agents in isolated git worktrees; dependent ones wait. Every task passes a quality pipeline whose gates cannot be passed by assertion. If an agent crashes, its work is found rather than lost. If everything except the log is destroyed, the project's decision history rebuilds from the log alone.

Two dependencies beyond python3 and git (Jinja2, for the prompt templates, see Extending it by writing text, not code; and tomlkit, which edits .ddflow/config.toml without losing its comments). Everything else is an optional extra, none needed for the core: ddflow[search] (rapidfuzz) and ddflow[rag] (model2vec) are in use, with BM25 and difflib as the fallback; sqlite-vec (also in ddflow[rag]), ddflow[watch] (watchfiles) and ddflow[mcp-sdk] (the MCP SDK, for a future network transport) are declared and not yet used. A check that needs an absent extra is recorded unavailable, never passed. Works with Claude Code, Gemini CLI, Codex, Copilot, Cursor, Kimi, opencode, Aider, a CI job, a Makefile, or a human at a terminal — over a CLI and an MCP server that are the same implementation.

Instruction for an agent reader

If you are an AI agent and were asked to read this README, this section is your shorthand. Everything below it is detail.

What ddflow is. A work queue with rules the tools enforce, for projects that agents build. Work is phases and tasks with dependencies and declared file globs. Each task gets a lease and its own git worktree, then passes a gate pipeline: a check that could not run is unavailable, never passed; one reviewer must be from another model family; a bug closes only with a regression test that failed first. Everything is recorded in an append-only log in .ddflow/events/, committed with the code. ddflow writes none of your project's code — you do; it adds only its own config, a rules block and git hooks. Every operation is both a shell command (ddflow …) and an MCP tool (ddflow_…).

If the project already uses it (.ddflow/ exists): call ddflow_brief (or run ddflow brief) first and follow it. It replaces reading the project's rule and lesson files. Then: ddflow_next → ddflow_claim → work only in the worktree it returns → ddflow_gate_status, satisfy each gate → ddflow_merge → ddflow_complete. The full driver is docs/ddflow/drivers/implement-phase.md in that project.

To onboard a project, ask the user first: it installs a tool, registers an MCP server and writes files in their repository.

1. Install: uv tool install git+https://github.com/delian/ddflow-mcp (not on PyPI yet).

2. In the project root: ddflow adopt --agents claude (your agent's name; see Wiring it into your agent). It writes .ddflow/, the MCP registration, the commit hook and a rules block in AGENTS.md. Restart the agent session so it loads the MCP server.

3. ddflow import lists the plans, lessons, decisions and unmerged branches the repository already has, and writes nothing. Run ddflow import --apply only once the user agrees with that list.

4. ddflow config --set gate.unit_tests.command "<their test command>". Then ddflow companions lists the tools the gates expect that are missing: propose them, never install one unasked.

5. Put the user's plan in the queue (ddflow phase add …, ddflow task add … --globs …) and finish with ddflow doctor.

Exit codes, everywhere: 0 ok · 1 failure · 2 could not run / nothing to do · 3 refused. Never treat 2 as 0.

Related MCP server: forge-orchestrator

Everything is yours to change, per project

ddflow ships defaults, never a fixed process. Every part of how it works is a file or a setting in your repository, and each project can differ from the next: the gate pipeline and each gate's command or prompt, which reviewers run, the rules your agents follow, every prompt an agent is handed (including the instructions it receives the moment it connects), the slash-command workflows, your own [[macro]] modes, the export document templates, the branching and release model, parallelism and lease limits, cadences, enforcement strictness, and the per-agent driver docs. Nothing is hard-wired except the four enforced rules at the top of this page, and even those are tuned through documented knobs, never silently bypassed.

Three ways to change anything, all validated before anything is written:

• Edit the file. It is plain TOML or Markdown under .ddflow/ (committed, shared by every clone) or .ddflow/local/ (git-ignored, yours alone).

• Use the CLI (ddflow config --set, ddflow workflow …, ddflow prompts eject, ddflow rule …, ddflow export eject).

• Ask your agent, which has the same operations as MCP tools (ddflow_configure, ddflow_workflow_*, ddflow_rule_*). It changes a project's workflow only with your agreement.

The complete list of what can be changed, and where, is the Customisation reference at the end of this page.

Introduction

The problem it solves

An AI coding agent is good at a task and weak at a project. One agent in one session mostly works. Run it for weeks, or run three at once, and the same failures come back:

• Work disappears. A session crashes or is closed mid-task, and the half-finished change sits in a directory nobody remembers.

• Agents collide. Two of them edit the same file, and the second merge quietly undoes the first.

• Checks that never ran look like checks that passed. The linter was missing, the reviewer endpoint was down, the tests were "run" in a summary. The agent reports done, and nothing on record says otherwise.

• The project forgets. Last week's hard-won lesson, the reason behind a design choice, the bug that was already fixed once — gone at the next session, or at the next context compaction in this one.

• Nobody can say what happened. Which instruction led to which change, and which review looked at it, lives in a chat transcript that no longer exists.

ddflow is the layer between you and your agents that makes those failures structurally hard rather than a matter of discipline. It does not write code and it is not an agent. It is a queue, a set of rules the tools enforce, and a log of everything that happened.

What changes for you

Who it is for

• One developer with one agent. A plan that survives the session, a memory that survives compaction, and a record of which checks really ran. The queue is useful even with no parallelism at all.

• Several agents in parallel — subagents, several terminals, several vendors. The dependency graph says which tasks are independent, worktrees keep them apart, and the merge step lands them without anyone switching the main checkout's branch.

• A team or a CI pipeline. The log is committed with the code, so a fresh clone knows the queue and its history. Read commands have a machine-readable --json form and every command returns the same four exit codes, so a Makefile or a CI job can drive it exactly as an agent does.

It works with the agent you already use, because everything it does is reachable both ways: as a shell command and as an MCP tool. Use whichever your agent, script or CI job has. Your workflow is text, not code — the gate pipeline, the reviewer instructions and the agent-facing prompts are files in your repository that you can edit.

What it is not

• Not an agent or a model. Your agent does the work; ddflow decides what may start, checks what was claimed, and remembers.

• Not a hosted service. Everything is files in your repository and a disposable local cache. No account, no server to run beyond the local MCP process.

• Not a replacement for your tests or CI. It runs the commands you configure and records their real exit codes and output.

A first run

# ddflow-mcp is not on PyPI yet; until the first release, install from the repository:
uv tool install git+https://github.com/delian/ddflow-mcp   # or: pipx install git+https://github.com/delian/ddflow-mcp
cd /path/to/your/project
ddflow adopt                        # registers the MCP server with your agents, writes .ddflow/ and a block in AGENTS.md
ddflow phase add P1 --title "Password reset"
ddflow task add P1.T1 --phase P1 --title "Reset-token endpoint" --globs 'src/auth/**'
ddflow next                          # what can start now, and why the rest is blocked

adopt registers the server you just installed, by its full path: an install that did not come from a package index (from git, a local directory or an archive) carries a direct_url.json in its metadata (PEP 610), and for one of those adopt writes the ddflow-mcp installed beside its interpreter rather than uvx ddflow-mcp, which would fetch from PyPI. Only an install from an index gets uvx. --launch python still forces the interpreter-plus-PYTHONPATH form.

Then tell your agent "implement phase P1". The driver adopt installed tells it to start with ddflow_brief, claim the task, work in its own worktree, satisfy each gate and land the change. ddflow cannot make an agent follow instructions, but it makes skipping them visible: the commit hook adopt installs flags a commit made without a lease (or refuses it, if you set [enforce].commit_without_lease = "block"), and ddflow complete refuses an item whose gates carry no outcome. Watch it with ddflow board, and ask ddflow doctor at any point whether the project is healthy.

How do I…?

Every row is a command you can run in a terminal and a tool an agent can call over MCP — the same implementation, so neither drifts from the other.

Every read command takes --json. Every exit code means the same thing everywhere: 0 healthy · 1 real failure · 2 could not run / nothing to do · 3 coordination refused. 2 is never collapsed into 0 — "nothing is ready" and "everything is fine" are different facts, and an agent that cannot tell them apart invents work.

What --json and the MCP tools return has a name. A result that is a JSON object carries a top-level "schema": "<command>@<n>" -- for example "claim@1"; the command is the MCP tool's name without ddflow_, and --json of the CLI command that tool serves says the same. Within a version fields are only ever added; removing or retyping one bumps n. On a refusal the refusal object still leads and the tag follows it. A result that is a bare array (progress, loops, lesson search, ...) keeps its exact shape and has no tag, and text bodies (board, doctor) have none. python -m ddflow.surfaces.tool_table --schemas prints every tool's declared result schema; tests/golden/json/result_schemas.json pins them.

Table of contents

• Instruction for an agent reader

• Everything is yours to change, per project

• Introduction

  • The problem it solves

  • What changes for you

  • Who it is for

  • What it is not

  • A first run

• How do I…?

• Help: what it can do, and the workflow

• The default workflow at a glance

• Two ways to drive it

  • Standalone: a terminal, a Makefile, CI

  • Every command

  • As an MCP server

  • What goes in AGENTS.md / CLAUDE.md

• The workflow, and changing it

  • Changing it

• Why it is built this way

• Install into any project

  • Docker — for operators with no Python toolchain

  • Extending it by writing text, not code

  • Publishing and registry

  • Cutting a release

  • What CI checks

  • Any LLM as a reviewer — local, remote, SaaS, or a CLI

  • Companion tools

• Pin files regenerate; they are never hand-merged

• Finding a test polluter

• Wiring it into your agent

  • One set of rules, every agent

• From plan mode to the queue

• When a companion is missing

• Adopting a project that already has history

  • What it will and will not decide for you

  • What an unticked box means, and where things live

  • Verifying an import, at any time

  • What it reports rather than fixes

• The model: phases, tasks, dependencies, globs

  • Model-tier hint (advisory)

• Work that changes shape while you do it

• Architectural decisions

• Recall — "have we been here before?"

  • The Embedder: semantic ranking, optional

  • One path for guidance: rules and decisions reach you ranked, budgeted and fenced

  • Who wrote it: provenance and the data fence

  • Similar — "is this already filed?"

  • Dupes and link — settling a pair already in the log

  • The check every add runs

• Operational memory

• Resources and long-running jobs

• Dependencies on another repository

• Status, progress, and loops

• The task pipeline

  • Proving a gate can fail at all

  • A gate that runs only on the work it concerns

  • A pass that must name its evidence

  • When the exit code is not the verdict

• The phase pipeline

• Human approval: a gate the agent cannot clear

• Parallelism and coordination

  • Lanes: who is working and what to do next

  • Adaptive parallelism: configuration

  • Adaptive parallelism: the sample ring and the derived limit

  • Adaptive parallelism: host signals

  • Log-derived flow signals

• Gitflow, pull requests and version tags

  • Several release lines: fixes to older majors

  • Environment branches: promoting downstream

  • Workflow choices: asked, recorded, defaulted on the record

• Many agents, one server: identity, state and sharing

  • Is it stateless?

  • Who is calling?

  • Can one server serve several projects?

  • Locking, contention and measured cost

• Crash recovery

• Reconstruction from logs alone

• Lessons that check themselves

• Checking that the checks are working

• Reading the log, and why it is never compacted

  • Events that vanished from a shard

  • Managed definitions: the def.* event kinds

  • A log from a newer ddflow, and the link event kinds

  • The compatibility contract

  • Upgrading ddflow

  • Upgrading a project: the whole picture

  • The version stamp and the skew guard

  • Versioned data repairs

  • The upgrade manifest: what changed since your version

  • Migrations: breaking changes P-unify makes to what a project holds

  • Applying an upgrade

  • The compaction that was declined

• Lessons, research and bugs

  • Bugs are queue items

  • Listing tasks, phases, bugs and research

  • Searching everything

• Cadences

  • Scheduled jobs

  • Event triggers

• Exporting documents

• Keeping AGENTS.md true

• Surviving a compaction

• Keeping session-start cost flat

• Agent portability

• Keeping the two surfaces honest

  • All MCP tools

• Command reference

• Configuration

  • All knobs

• Id schemes

• What is automated, and what is not

• Customisation reference

  • Files you can edit

  • Configuration sections (.ddflow/config.toml)

  • Prompts and workflow commands

  • CLI commands that change the project's setup

  • MCP tools that change the project's setup

  • Environment variables

• Testing

• Documentation index

Help: what it can do, and the workflow

$ ddflow help                 # what this is, the loop, every capability grouped
$ ddflow help workflow        # workflow · import · gates · parallel · memory · recovery · config

Reachable as ddflow_help over MCP, and that is the point: an agent connecting had 59 tool descriptions and a state-aware handshake, neither of which answers "what is this, and how am I meant to work here". A tool description explains one tool to someone who already picked it; the handshake describes this repository right now.

Two halves, deliberately:

• The narrative is a template under ddflow/templates/prompts/help/, so ddflow prompts eject-style overriding applies — put your own .ddflow/prompts/help/workflow.md in place and the tool teaches your workflow.

• The capability inventory is generated from the live tool table. A hand-kept command list in a second place is the documentation-drift class, and this project has paid for it twice.

Three ratchets keep the prose honest, because a page recommending a flag that was renamed is worse than no page — whoever finds nothing reads the code, and whoever finds a wrong answer trusts it. Every command a page names must exist as a CLI leaf or an MCP tool; every topic the index offers must resolve; and every tool must fall into a group, so a new capability has to be classified rather than quietly dropped from an inventory that claims to be complete.

The default workflow at a glance

Three pictures of what ships by default. Every step is configurable (see The workflow, and changing it); ddflow workflow prints what this project actually runs.

The agent's loop. One item at a time: pick it, lease it, clear its gates, land it. Each arrow labelled with an exit code is what the tool says, not a convention — 2 means nothing to do, 3 means refused, and neither is ever read as success.

flowchart TD
    S["Session start<br/>ddflow brief · recover"] --> R{"Crashed agent's<br/>work left over?"}
    R -- yes --> SV["Inspect the worktree, salvage,<br/>ddflow release"] --> N
    R -- no --> N["ddflow next"]
    N -- "exit 2: nothing ready" --> W["ddflow wait<br/>sleeps until a holder lets go"] --> N
    N -- "exit 0: items ready" --> C["ddflow claim<br/>lease + isolated git worktree"]
    C -- "exit 3: refused<br/>(lease or file-glob conflict)" --> N
    C --> P["Task pipeline<br/>satisfy every gate in order"]
    P --> M["ddflow merge<br/>lands the branch from the primary checkout"]
    M --> D["ddflow complete<br/>checks every gate has an outcome<br/>+ a different-family review"]
    D -- "exit 3: refused" --> P
    D -- done --> N

With [flow].integration = "pr", merge opens a pull request and parks the item in review instead; next completes it when the request merges.

A lease is evidence of intent, not a lock, but a LIVE lease held from another clone is not overwritten by accident: release and gate run|record|skip on it exit 3 naming the holder, unless you pass --force --reason "<why>" (CLI only; recorded as forced on the event). Agents of the same clone, an expired lease and a lease with no recorded clone are unaffected.

The task pipeline. Ten gates, in order. Heavy borders are the gates gates.required names by default (implement, unit_tests, merge); the rest still need some outcome — passed, failed, unavailable, partial, or an explicit skip with a reason — because silence is not a pass.

flowchart LR
    subgraph A["You, the agent"]
        direction TB
        g1["1 · research<br/>falsifiable claim + probe"] --> g2["2 · rules<br/>ddflow brief --item"] --> g3["3 · implement<br/>in the item's worktree"]
    end
    subgraph X["A different model family"]
        direction TB
        g4["4 · rubber_duck<br/>try to refute it"] --> g5["5 · critic<br/>diff vs. stated intent"]
    end
    subgraph T["Tooling"]
        direction TB
        g6["6 · standards<br/>linters, architecture"] --> g7["7 · unit_tests<br/>the suite, executed"]
    end
    subgraph B["You, again"]
        direction TB
        g8["8 · bug_hunt<br/>recurring bug classes"] --> g9["9 · dedupe<br/>already exists?"]
    end
    g10(["10 · merge<br/>ddflow lands it"])
    A --> X --> T --> B --> g10

    classDef req stroke-width:3px
    class g3,g7,g10 req

The phase pipeline. A phase wraps its tasks and checks the whole before it lands:

flowchart LR
    p1["research<br/>the phase's open questions"] --> p2["tasks<br/>each runs its own<br/>ten-gate pipeline,<br/>in parallel where<br/>globs allow"]
    p2 --> p3["unit_tests"] --> p4["bug_hunt"] --> p5["dedupe"]
    p5 --> p6["live_test<br/>run the real thing"] --> p7["corrections"] --> p8["docs<br/>README and docs<br/>match the change"] --> p9(["merge"])

Details: The task pipeline · The phase pipeline · Parallelism and coordination · Crash recovery.

Two ways to drive it

The CLI is the whole product. The MCP server is a second surface over the same commands, and tests/test_mcp_parity.py fails if the two diverge — every subcommand has a tool, every flag is reachable, and each exemption carries a written reason.

Standalone: a terminal, a Makefile, CI

$ ddflow init
$ ddflow config --set gate.unit_tests.command "python -m pytest -q"
$ ddflow phase add P1 --title "Billing" --globs "src/billing/**"
$ ddflow task add P1.T1 --phase P1 --title "Tax rules" --globs "src/billing/tax.py"

$ ddflow next                          # exit 2 = nothing actionable; exit 1 = unknown --phase
$ ddflow claim P1.T1                   # exit 3 = refused, with the reason
leased P1.T1 · worktree .ddflow/worktrees/P1.T1 · branch ddflow/P1.T1

$ cd .ddflow/worktrees/P1.T1 && ...   # do the work
$ ddflow gate status P1.T1             # what the pipeline wants next
$ ddflow gate run P1.T1 unit_tests     # runs it; the exit code IS the evidence
$ ddflow gate record P1.T1 implement --outcome passed --evidence "added tax.py"
                                       # failed/unavailable/partial/skipped also need --reason
$ ddflow complete P1.T1                # exit 3 lists whatever is unsatisfied
$ ddflow merge P1.T1

You get everything except the judgement. Command gates run themselves; agent gates wait for a human to record an outcome, and ddflow gate skip <id> <gate> --reason "..." is the escape hatch — recorded as a skip, never as a pass.

Merge queue. Where several agents land work at once, ddflow queue add <id> enqueues an item, ddflow queue run (any agent may) integrates the head of the queue — one entry's ci at a time, --max-batch N entries per run, --unit-tests / --no-unit-tests to run or skip each entry's unit_tests gate on the merged result — ddflow queue status (the default) prints the line, and ddflow queue remove <id> drops an entry. Defaults live in [merge_queue].

Changelog line (optional). ddflow complete P1.T1 --changelog "Added: tax rounding" (and ddflow bug fixed B1 --regression-test ... --changelog "Security: ...", MCP changelog on ddflow_complete / ddflow_bug_fixed) records one line for the project's changelog on the completion event. The category is one of Added, Changed, Deprecated, Removed, Fixed, Security (case-insensitive; anything else is refused with that list); --changelog skip (or internal) marks work that must not appear in the changelog. complete never asks for it and never requires it, and logs without it fold exactly as before.

In CI, the exit codes are the interface:

check:
	ddflow doctor        # 1 = integrity problems, each named
	ddflow workflow      # 1 = the pipeline does not hang together
	ddflow cadence       # 2 = no periodic pass is due

2 is never "no problem". A job that treats it as success reports a green build for a suite that never ran.

Every command

Every top-level command, with the first line of its --help. The list is generated from the command parser (python -m ddflow.surfaces.command_table README.md rewrites it, and ddflow pins regenerate does so after a merge), so a command added on another branch never needs a line written by hand here.

As an MCP server

ddflow mcp speaks newline-delimited JSON-RPC over stdio, and treats the directory it was started in as the caller's location (as ddflow-mcp does), so a merge from inside a worktree keeps that worktree. You rarely run it by hand — ddflow adopt writes the launch entry into each agent's own config and leaves existing servers alone:

22 agents are supported. The full table, with what each one gets, is in Wiring it into your agent.

It also copies the driver to docs/ddflow/drivers/, and installs the pre-commit hook that enforces claim-before-you-edit.

What an agent sees the moment it connects, with no call to make:

• Instructions, returned inside the initialize result itself — and state-aware: what is ready, what is in flight, which setup is missing, whether this project has history worth importing, whether an import was left unfinished.

• Tools — one per CLI command.

• Resources — ddflow://board, ddflow://brief, ddflow://lessons, ddflow://research.

• Prompts — which a client turns into slash commands. Tools are things an agent calls; prompts are things you invoke.

A refused call says so first. Over MCP a tool that did not do what was asked — a claim refused for an overlap (exit 3), or any call whose result would otherwise be its success shape in nulls — returns a JSON body whose first key is "refusal": {"reason": ..., "outcome": ..., "exit": ...}, followed by whatever the operation actually said (a refused claim's alternatives). The lead is named refusal for any call that did not do what was asked: outcome (failed, nothing or refused) and exit say which, so an exit-1 failure is not mistaken for a refusal. A tool whose body is one field (ddflow_decision_show on an unknown id) leads with the same object instead of a bare null. Exit 2 ("nothing") keeps its declared keys after that lead, and any other data it carried; a data field that is itself named refusal is kept as refusal_data. A result that fills its declared shape is left as the CLI's --json prints it, with the reason in the second content block.

Two tools exist so an agent can orient itself without being told: ddflow_help (what is this, what is the loop) and ddflow_workflow (what are the rules here).

Protocol versions: both eras. A client that opens with initialize negotiates 2025-11-25, 2025-06-18, 2025-03-26 or 2024-11-05 (an unknown version falls back to the newest) and is served exactly as before; 2025-11-25 adds nothing a tools, resources and prompts server must do differently -- a bad argument was already a tool result with isError, never a protocol error. A client on the stateless 2026-07-28 revision sends no initialize: each request carries io.modelcontextprotocol/protocolVersion and io.modelcontextprotocol/clientCapabilities in params._meta, and the server answers it on its own. server/discover returns the supported versions, capabilities and the same instructions initialize carries; a version not served per request is refused with -32022 (data.supported lists every version), a missing required _meta field with -32602; every modern result carries resultType: "complete" and the server's identity in _meta, and tools/list, resources/list, resources/read, prompts/list and server/discover carry ttlMs and cacheScope: "private" (an hour for the two lists fixed for the life of the server, 0 for everything read from the repository's state). Identity is per request in that era (D-mcp-identity-per-call): a modern request names its caller with the as_agent argument or ddflow/agent in params._meta (the argument wins when both are given, and _meta is then not read; a malformed name that is read is a tool error), and one that names nobody is attributed to the tree-derived default. Like as_agent, a _meta name other than the connection's own does not adopt the tree the server stands in on ddflow_claim (it could be a subagent riding the connection): the item gets a tree of its own, as from the CLI in the primary. ddflow_identify on a modern request persists nothing for the connection and answers how to name yourself instead; a connection that opened with initialize keeps connection-wide ddflow_identify exactly as before, with as_agent as the per-call form.

One module, ddflow/surfaces/mcp_protocol.py, holds everything that differs between revisions; no tool sees which one a request came in under. It also carries the 2026-07-28 multi round-trip: a tools/call, resources/read or prompts/get may be answered resultType: "input_required" with inputRequests (only the kinds the client declared in its capabilities) and a sealed requestState that the client echoes on its retry. The state is HMAC-bound to the method, the request's name/arguments/uri, the ddflow/agent caller and a 10-minute expiry, for the life of the server process. A retry whose requestState is forged, tampered with, presented on another request or expired is refused with -32602. No tool asks for input yet. CI replays every reply against the official schema.json of each revision served (tests/fixtures/mcp-schema/, the upstream files pinned by commit and sha256).

What goes in AGENTS.md / CLAUDE.md

ddflow adopt writes it as a managed block between <!-- ddflow:begin rules/work-queue ddflow=V fmt=N sha=H --> and <!-- ddflow:end rules/work-queue --> (the version that wrote it, its format level and a digest of the body, so an older, newer or hand-edited block is told apart). Your own prose around it is preserved; re-running updates only what is inside. A block under the DDFLOW:BEGIN/DDFLOW:END markers older ddflow wrote is read as before and moved to the new ones the next time it is written; a block a newer ddflow wrote is refused (exit 3, "upgrade ddflow to >= X") instead of rewritten, and ddflow adopt --refresh-docs copies the files it is about to rewrite to .ddflow/backups/ first. The /implement command file is a region of the same kind (commands/implement); delete its begin line to keep your own. If you write it by hand, four things have to be in it:

1. Start every session with ddflow_brief (or ddflow brief in a shell).

2. Claim before you edit — ddflow_next → ddflow_claim → work in the worktree it creates.

3. The loop — ddflow_gate_status → satisfy each gate → ddflow_complete → ddflow_merge.

4. The exit codes, and that 2 is not success.

Without that block an agent sees the tools and has no reason to reach for them before editing. The block is what makes the queue authoritative rather than optional — and it is 232 words, because an instruction file nobody finishes reading is one nobody follows.

The workflow, and changing it

$ ddflow workflow
# The workflow this project runs

   1. research      agent
   2. rules         agent
   3. implement     agent       (required)
   4. lint          command     (required, NOT proven able to fail)
      $ ruff check .
   ...

## The rules, and where each came from

  gates.require_outcome                  True              [default]
  gates.enforce_order                    block             [file]
  schedule.parallel                      auto              [default]
  schedule.max_parallel_tasks            4                 [default]

One answer to "what are the rules here": every gate in order, which are commands and which you perform, which are required, which need evidence, which need a different-family reviewer, which have been proven able to fail — plus the completion rules, the caps, the reviewers, and where each value came from, so a deliberate choice is distinguishable from a default nobody touched.

Changing it

$ ddflow workflow gate lint --command "ruff check ." --into task --after implement --required
$ ddflow workflow pipeline task research,implement,lint,unit_tests,merge
$ ddflow workflow drop dedupe

All four reach MCP — ddflow_workflow, ddflow_workflow_pipeline, ddflow_workflow_gate, ddflow_workflow_drop — so an agent can change the workflow with the operator's agreement. Their descriptions say to ask first and offer dry_run, because a pipeline governs every future item, not the one in hand.

A pipeline per item kind. One registry (task, phase, bug, doc, research, job) maps an item's kind to the pipeline it runs; each starts on gates.task_pipeline (a phase on gates.phase_pipeline), and gates.kind_pipelines replaces it for one kind, e.g. { doc = ["implement", "merge"] }. A promotion task keeps gates.promotion_pipeline. ddflow workflow lists every kind that runs something other than the task pipeline (a phase: other than the phase pipeline), and workflow --json (and the ddflow_workflow MCP tool) carries kind_pipelines, every kind's effective pipeline; workflow drop and the undefined-gate and applies_to checks cover these pipelines too.

Nothing is written until it is checked, and the order is the point: compose the change, validate the result, then replace the file atomically.

• Keys are plain keyboard keys (decision D-plain-keys). Every key config --set, ddflow_configure and the workflow commands accept is dot-separated ASCII letters, digits, _ and -; a quoted segment, a \uXXXX escape, whitespace or non-ASCII is refused with exit 3, naming the plain spelling when there is one (use gate.unit_tests.command). A gate id follows the same rule -- it names the [gate.<id>] section, and a dot in it would nest a table (a.b defined gate a). config --append-toml and ddflow_configure toml hold every key of the block to the same rule -- a table header, a key, an inline-table key: [flow] then "tag_prefix" = "v" is refused with use flow.tag_prefix -- and companions add refuses a companion id that is not one bare key (it names the agent's [mcp_servers.<id>] entry). Hand-written TOML files are still read as TOML.

• A pipeline naming an undefined gate is refused, naming the near miss. That one is otherwise silent and permanent: the outcome folds to empty, completion refuses it forever, and gate record rejects the id as unknown — so the item can never be completed at all, and nothing says why.

• An unknown section or knob is refused, with a suggestion. [gatez] is valid TOML and used to be written happily, breaking every later command — the write path validated the merged text for syntax and then validated the config already on disk, which is a writer checking the state it is replacing.

• Dropping a gate takes it out of required too, or it becomes a requirement that quietly requires nothing.

• What is required is [gates].required, only. required = true inside a [gate.<id>] table used to be accepted and ignored, so the gate was never enforced (and required = false left a listed gate required). A table that disagrees with the knob is now warned about on every command (and refused in ddflow's own tree), naming the edit to [gates].required that does what it says; one that agrees is redundant and passes.

ddflow workflow and ddflow doctor both re-run those checks against what is on disk. Everything is a file you can also edit by hand: gates in [gate.<id>], reviewers in [[reviewer]], companions in .ddflow/companions.toml, and every prompt — including the instructions your agent receives at connect — under .ddflow/prompts/.

One caveat with MCP: the connection instructions are computed once, when the server starts. A workflow changed mid-session is live for every tool call immediately, but the text the agent was handed is stale. Tell it to call ddflow_workflow, or restart.

Why it is built this way

The append-only event log is the source of truth; everything else is a projection that can be deleted and re-derived. The SQLite index, the markdown boards, the search index, the recovery bundle — all disposable, all rebuilt by ddflow rebuild.

That single inversion is what makes the four hard properties fall out for free rather than needing to be engineered:

The design decisions, with the probes that settled each, are in docs/RESEARCH.md. The two that most shaped it:

• SQLite-on-NFS is correct here but 36× slower than local (measured, 12 processes × 40 increments). So the log is authoritative and the database is a disposable cache — which also happens to be the choice that stays correct on filesystems where locking is broken.

• An expired lease must never be reclaimed automatically. A crashed agent's worktree is sometimes irreplaceable work and sometimes a superseded draft, and nothing in the metadata distinguishes them. Recovery measures and advises; it never deletes.

Install into any project

One line in your agent's MCP config. Nothing else.

Until the first release is on PyPI, uvx ddflow-mcp has nothing to fetch: install from the repository and run ddflow adopt, which registers that installation instead, as in A first run.

{ "mcpServers": { "ddflow": { "command": "uvx", "args": ["ddflow-mcp"] } } }

uvx fetches and runs the published package in an ephemeral environment on first use — no clone, no virtualenv, no PYTHONPATH, no install step for an operator to forget, and no vendored copy to drift from upstream. ddflow needs two runtime dependencies beyond python3 and git (Jinja2 and tomlkit, both pure Python), which is what lets it install inside sandboxes, CI images and other tools' ephemeral containers.

Then, from the agent, with no shell at all:

That is the whole adoption. The per-project instruction text is 232 words — a managed block in AGENTS.md, because the MCP tool descriptions already carry the how, and a second copy of that would drift from the one the model actually reads.

uv tool install ddflow-mcp        # or: pipx install ddflow-mcp
cd /path/to/your/project
ddflow adopt            # every supported agent
ddflow adopt --agents claude,cursor,vscode,kimi   # or name the ones you use

After upgrading ddflow, ddflow doctor notes any driver doc (implement-phase.md, an adopted agent's delta) that differs from the template the running ddflow ships. ddflow adopt --refresh-docs (MCP: ddflow_setup with refresh_docs) rewrites only those docs, the AGENTS.md/CLAUDE.md blocks and the agents' native rules -- never the MCP launch, hooks or command files -- and refuses a project that was never adopted.

Stale references. A file the project holds can name a command or tool that a release renamed: a settings hook, a git hook, .pre-commit-config.yaml, a driver doc, an AGENTS.md or CLAUDE.md block, a slash command, an ejected prompt, a macro. ddflow doctor (and ddflow_doctor; an MCP server has no command parser loaded, so it checks tool names and says in one note how many command references it could not check; the CLI's doctor checks both) resolves every ddflow <command> (in backticks, a fenced block or a command line; prose is not read) and ddflow_<tool> they name against this ddflow: ok, deprecated (an alias, with the replacement), unknown, or unchecked where the process has no table to judge it by, each with file:line. A deprecated name, or an unknown one in the project's own text, is a note; an unknown name in a region ddflow wrote and nobody edited is a problem, because the command it runs will fail. Text inside a ddflow-managed region is rewritable (services.compat_refs.rewrite: backed up first, the region re-stamped); anything else is the project's own and only gets a proposal, never a silent edit. A hook entry in a settings file is ddflow's by its stamped region (hooks/<name>), not by the command it runs, so renaming a ddflow hooks ... subcommand refreshes the installed hook in place instead of adding a second one; an entry written before the stamp is still found by its command text.

A driver doc is a stamped region (<!-- ddflow:begin drivers/implement-phase ddflow=V fmt=N sha=H --> ... <!-- ddflow:end drivers/implement-phase -->): the version that wrote it, its format level and a digest of its body. That is how a refresh and ddflow upgrade tell an unedited copy from an older release (replaced) from a hand edit (the digest no longer matches). A hand edit is never silently overwritten: before the refresh rewrites the file, your version is copied whole to <file>.local-edits (.local-edits.2, .3 ... when that name is taken) and the action says so; text you wrote outside the region stays where it is, and the originals also go to .ddflow/backups/. The same holds for an edit inside the AGENTS.md/CLAUDE.md block, and for the /implement command file when a plain ddflow adopt rewrites it (a refresh never touches command files). A copy written before the stamp (a driver an older ddflow copied) has no digest to tell by: it is replaced after a backup, and ddflow upgrade --plan calls it stale when a release has shipped since the project last worked and hand-edited (needs the operator) when none has. A doc written at a higher format level than this ddflow understands is left alone: upgrade ddflow.

adopt is idempotent and writes managed blocks, so re-running after an upgrade updates them and leaves your own prose alone. It writes the MCP registration into each agent's own config location, merged with whatever servers are already there, through the same writer companions add uses: an entry that already holds this launch is left untouched, a stale ddflow entry is refreshed (in .codex/config.toml too, where a header that is only text -- a comment, say -- no longer counts as a registration), and a config it cannot parse is refused with "add the server by hand". From a source checkout it points the config at that checkout instead of the published package, so developing ddflow does not silently configure your project against the released version.

Docker — for operators with no Python toolchain

{ "mcpServers": { "ddflow": { "command": "docker", "args": [
    "run", "-i", "--rm",
    "-v", "${workspaceFolder}:/repo",
    "--add-host=host.docker.internal:host-gateway",
    "ghcr.io/delian/ddflow-mcp:latest" ] } } }

ddflow adopt --launch docker writes exactly that. The image is 107 MB (Alpine; ddflow is pure standard library, so there is no compiled dependency to worry musl about) and behaves identically on Linux, macOS and Windows.

Four things go wrong when a containerised tool touches a bind-mounted git repo. All four are silent, one of them loses work, and all four are handled:

And one that cannot be fully handled, so it is reported: 127.0.0.1 inside a container is the container. A model server on your own machine is not reachable from there. ddflow rewrites loopback reviewer URLs to host.docker.internal, and ddflow doctor tells you that on Linux you must also pass --add-host=host.docker.internal:host-gateway, because unlike Docker Desktop the Linux engine does not provide that name.

The related portability fix: worktree paths are stored in the event log relative to the repo root. The log is committed and shared, so an absolute path is true only on the machine that wrote it — false for a teammate who cloned elsewhere, for CI, and for a container where the repo is /repo. Pinned by test_the_event_log_carries_no_absolute_paths.

Extending it by writing text, not code

Every prompt is an external template, resolved config → project → shipped:

ddflow prompts list              # where each template currently comes from
ddflow prompts show <name>       # its source, as eject would write it
ddflow prompts get <name> --arg KEY=VALUE   # a workflow command or macro, rendered
ddflow prompts eject             # copy the shipped ones into .ddflow/prompts/
$EDITOR .ddflow/prompts/review_system.md

prompts get renders through the same code as MCP prompts/get (and the ddflow_prompts tool's get): every declared argument bound, a macro's parameters required and its tools preamble on top. show prints the text as written, {% if %} tags and all; a machinery template (mcp_instructions, the review prompts) takes no caller arguments, so get refuses it and names show.

Adding a mode of your own: [[macro]]. Overriding a shipped workflow needs no code, and neither does adding one. A macro is a named, parameterised prompt — "enter debugger mode" — that appears everywhere the shipped workflows do: prompts/list and prompts/get over MCP, which is what a client turns into a slash command, and ddflow prompts list|show in a terminal.

# .ddflow/config.toml   (or .ddflow/macros.toml, if you prefer to split it out)
[[macro]]
name = "debugger"
title = "Enter debugger mode"
description = "Reproduce first, then bisect. No fix without a failing probe."
params = ["symptom"]                                  # required, not optional
tools  = ["ddflow_bug_found", "ddflow_gate_run", "ddflow_bug_fixed"]
prompt = """
You are debugging: {{ symptom }}

Reproduce it before you theorise. Paste the command and its output.
"""

Use prompt_file = "docs/modes/debugger.md" instead for anything long enough that TOML quoting gets in the way.

When to reach for a macro rather than a gate. A gate is a step every item passes through, recorded against that item and blocking its completion. A macro is a MODE an operator enters, belonging to no item and recorded nowhere — "audit this release", "handle this incident". If the thing should hold up a task until it is done, it is a gate; if it is a way of working you want to name and re-enter, it is a macro. Putting a mode in the pipeline makes every task wait for something that was never about that task.

tools is declarative, not a sandbox. It is rendered into the prompt as the ordered set the mode expects, so the agent is told what the mode is for and the next reader can tell what it was supposed to do. It does not restrict what the agent may call — MCP has no mechanism for that, and claiming a security property this cannot honour would be worse than not having it. This is the deliberate departure from dx-zero/mcpn, whose toolMode: situational lets the model pick freely from a bound set with no recorded ordering: a session you cannot replay is a session you cannot review, which is the property the event log exists to give you.

Three things a macro refuses, because each alternative fails quietly: a missing parameter (a prompt with a hole in it reads as a complete instruction), a name that belongs to a shipped command (silent shadowing leaves you editing a block that does nothing), and both prompt and prompt_file (two sources for one body means one is dead and looks live). A refused macro is refused alone and by name — the others still load — and ddflow prompts list, prompts show, doctor and the MCP prompts/list say which one and why; an undecodable prompt_file is a named problem too. When the whole [[macro]] config cannot be read (a parse error, an unknown field), MCP prompts/get for an unknown name says why (not loaded: ...) instead of only "unknown prompt".

Including the one the agent actually reads first. mcp_instructions.md is the block an MCP client injects into the model's context on connect — the workflow, the reporting duties, and which companion tools to reach for. It is the file to edit when you want this project to work differently:

ddflow prompts eject mcp_instructions
$EDITOR .ddflow/prompts/mcp_instructions.md      # or [prompts] mcp_instructions = "..."

It renders against the live state — adopted, task_pipeline, setup_todo, companions, missing_companions, gate_gaps, recoverable, loops — so the instruction is the next concrete action rather than a fixed blurb the model learns to skip. A broken override says so in the instruction block itself instead of falling back to the default: this is the one surface where nobody would ever notice their edit was not live.

Templates render with Jinja2, one of ddflow's two runtime dependencies (the other, tomlkit, writes ddflow config --set and the other config edits in place, keeping comments, blank lines and key order; every TOML value ddflow writes goes through the one writer in infra/tomlcfg.py), and with a strict standard-library renderer when it is absent — a stripped deployment with no reachable package index still starts. The shipped templates use the subset both engines agree on, and tests/test_template_engines.py walks the template REGISTRY, rendering every entry through both engines and asserting the outputs are byte-identical.

That test is iterated rather than hand-listed for a reason. Its predecessor named three templates in a dict, mcp_instructions.md was never added, and in 0.1.1 the largest and most important template rendered correctly under Jinja2 and failed under the fallback — so the entire MCP handshake for an unadopted repository, the first thing a new user ever sees, degraded to ddflow's instruction template could not be loaded. Jinja2 was not a declared dependency at the time, so developers had it and the project venv did not: python -m pytest was green and uv run pytest was red on the same commit.

The fallback now raises on any construct it does not implement rather than copying it through. The old regex engine emitted what it could not parse, so a condition as ordinary as {% if a or b %} — which its single-name pattern never matched — reached the client as literal template source.

Both renderers are strict about undefined variables: a prompt silently missing the diff it was supposed to carry is the vacuous review in template form — the model dutifully reviews nothing and reports no findings.

The rest is TOML: gates and their pipelines ([gate.*], gates.task_pipeline), reviewers ([[reviewer]]), companions ([[companion]]), enforcement ([enforce]), cadences, and the rest of the 469 knobs. ddflow config --set <key> <value> edits one key in place, preserving comments.

Every config change goes through the one write pipeline, services/configwrite.apply_edit: config --set, --append-toml, ddflow_configure, workflow pipeline|gate|drop, reviewers add|detect --write and export enable all compose the edit, judge the RESULT once (format, schema, plain keys, the adaptive range, [gate.*] tables, the human-approval gates, and the workflow's coherence) and replace the file atomically under a lock. The same guards hold for the committed and the local layer and for every kind of edit, so an appended block is held to what --set is held to. Only a person typing on the command line writes a number or boolean as text; a workflow gate --command 5 is the string "5". A committed write brings .gitattributes up to date with [lease] append_only_globs, whichever command made it.

What is committed, and what stays on your machine

ddflow recommends services; it never ships one person's configuration. Two layers:

The local files are read last, so they win; ddflow config --explain reports such a value's source as local. A command gate that ran with a command set in the local layer records command_layer: "local" (else "committed"), a command_digest, and, when the committed layers define a different command, committed_command_digest; gate status, show and complete then label that pass "passed with a local-only command (not the committed one)", since it is not the project's pass. The evidence also names output_log_host (the agent whose machine holds the git-ignored .ddflow/runs log, which a teammate's checkout does not have). Evidence recorded before these keys is not labelled. Every writer of an operator-specific value targets the local layer by default:

ddflow reviewers add --preset ollama --model qwen3:8b   # -> .ddflow/local/reviewers.toml
ddflow reviewers detect --write                         # -> .ddflow/local/reviewers.toml
ddflow config --local --set gate.unit_tests.command "pytest -q -n 48"
ddflow config --local --append-toml "$(cat my-reviewer.toml)"

--shared on reviewers add / reviewers detect --write commits the block to .ddflow/config.toml instead — only for a service every clone reaches at the same address. config --set and --append-toml stay committed unless you pass --local (over MCP: ddflow_configure with local=true, ddflow_reviewers_detect with shared=true), because a test command or a pipeline is project policy. The human-gate guards hold on both layers: no writer sets gate.<id>.human, and none can drop a human gate from a pipeline. .ddflow/local/ carries its own * .gitignore, so it stays uncommitted even in a project whose .ddflow/.gitignore predates it. API keys are never written anywhere — only the name of the variable that holds one.

A fresh ddflow init says the same in the config.toml it writes: the commented [[reviewer]] example points at .ddflow/local/reviewers.toml (not at config.toml), and a short comment block separates what is committed (generic project policy) from what is local (your endpoints, hosts, key variable names, machine sizing: ddflow config --local).

Publishing and registry

Nobody should have to paste JSON into an IDE to use this. server.json is the MCP registry manifest (io.github.delian/ddflow-mcp), and publishing it is what makes ddflow findable in the VS Code and Cursor marketplaces rather than something you configure by hand. It offers three ways to run the same server, so a client picks whichever it supports:

The image is built for amd64 and arm64, because an Apple-silicon operator running it under emulation pays that cost on every tool call, and tool calls are all this server does.

How CI authenticates — four mechanisms, one stored secret:

Docker Hub is the only one that needs a long-lived credential, because it has no OIDC equivalent. Use an access token scoped to this repository, never an account password. If that is one secret too many, delete the Docker Hub login and its two tags — ghcr.io alone satisfies the OCI entries a marketplace needs, and server.json lists both so a client picks whichever resolves.

environment: release on the publishing jobs is a control worth knowing about: point it at a GitHub environment with required reviewers and every release waits for a human, with no change to the workflow.

Order matters and the workflow encodes it. mcp-publisher validates that every package named in the manifest exists, so the registry step runs after both PyPI and Docker — publishing the manifest first would advertise a version nobody can fetch.

The registry verifies ownership, and the workflow checks it first. The registry accepts a package only when the artefact itself names the server: the PyPI package's README carries <!-- mcp-name: io.github.delian/ddflow-mcp --> (the first lines of this file), and each image carries the label io.modelcontextprotocol.server.name with the same name. The verify job runs tests/test_registry_ownership.py first, so a release the registry would reject — a missing marker, or a label that does not match — is refused before anything is uploaded; a server.json description over the registry's 100-character limit fails the suite the same way. So do the registry's per-package rules, which the JSON schema does not express and the registry enforces only at publish: an oci package must not carry a version field (or registryBaseUrl/fileSha256) — the tag in its identifier is the version — while the pypi package must carry one. publish #40 failed on exactly that, at the last job, after PyPI and both images had shipped; tests/test_registry_manifest_rules.py now encodes the rules offline and verify runs it first. The publish step also retries a transient registry failure (a 504, a 408/429, a network error) with backoff, checking the registry for the exact version after every attempt, because the publish behind a 504 may have committed; a 4xx that retrying cannot fix fails at once, and only after the budget does it fail with a "Re-run failed jobs" hint.

Four things gate a release, and each exists because the failure it catches is public and irreversible:

• the tag, ddflow/__init__.py (the one declared version), server.json's version and every OCI identifier's tag must agree — a :0.1.0 left behind while version moved on publishes a manifest pointing at the previous image, installable and wrong;

• the full suite, plus the slow end-to-end scenarios, which -m 'not slow' otherwise excludes from every ordinary run;

• the wheel must install into a clean venv and run, and carry its templates — uv build succeeding proves the metadata parses, not that ddflow help works;

• the image must answer initialize over stdio. A built image that cannot is a broken release every marketplace will happily offer.

Cutting a release

$ git push origin main           # that is the whole release

Every push to main that changes shipped code releases, numbered by the impact its changes declare. A patch unless the upgrade manifest declares a breaking change (then a minor while ddflow is 0.x); major moves only when you move it. "Shipped" means ddflow/, pyproject.toml, uv.lock, Dockerfile, docker-entrypoint.sh, .dockerignore, server.json, server.template.json or README.md (the PyPI page, and the line the registry verifies): a push of other docs, tests or the ddflow event log releases nothing, because PyPI keeps every version forever and one identical to the last is noise nobody can withdraw. CI runs scripts/bump.sh at the level scripts/release_impact.py reads from the manifest, commits release 0.1.2 to main, publishes PyPI, Docker Hub, ghcr.io and the MCP registry, then creates v0.1.2 and a GitHub release — last, and only once every publish succeeded, because a tag pointing at a half-release is worse than no tag: it looks authoritative.

Pull after a release. The release commit is CI's, so your main is one commit behind it and the next push is refused until you git pull.

How the gate picks the version: it publishes the declared version if PyPI does not have it yet, and bumps (by the declared impact: patch, or minor for a breaking entry while 0.x) only if it does. Moving to an exact version, or major, is yours to do —

$ scripts/bump.sh minor          # 0.1.4 -> 0.2.0 (or: major, or an exact 1.0.0)
$ git commit -am 'release 0.2.0' && git push origin main

— and CI publishes exactly 0.2.0; the next push that bumps nothing releases 0.2.1.

The same rule makes a release that failed before its PyPI upload retry its number with the next push. One that failed after it — Docker Hub, ghcr.io, the MCP registry — does not: PyPI has the number, so the next push moves past it; re-run that run's failed jobs from the Actions page instead. A bump never lands on a number PyPI already holds (a v* tag can publish one out of band): CI takes the next free number at the declared level (moving on by patch when that one is taken) before it commits anything. scripts/release.sh refuses a declared version that is a smaller step than the manifest's impact = "breaking" entries ask for, and a surface change (a command, flag, tool, argument, knob or event kind gone) that the previous release's snapshot shows and nothing declares, with the options. The bump is pushed to main before anything publishes: a push that loses a race with another commit fails the run and publishes nothing, where pushed last it would leave PyPI holding a version main does not declare. Runs are serialized, and only main or a v* tag releases.

The version is declared in one place: __version__ in ddflow/__init__.py, the only line a bump edits. pyproject.toml reads it (a hatch dynamic version, so uv.lock records no version for the project and a bump never makes the lock stale), SERVER_INFO (what the server tells every client it is, and what ddflow --version prints) is built from it, and server.json — the manifest's version, the pypi package's version and the tag inside every OCI identifier (an oci package has no version field) — is a generated, committed file: edit server.template.json, then run scripts/render_server_json.py (--check fails when server.json is not the render; a test and the publish workflow both run it). scripts/bump.sh edits the literal, renders server.json and re-reads the result; tests/test_packaging.py fails if anything drifts. Do not edit server.json by hand.

scripts/release.sh runs all of that locally and publishes nothing. It is dry by default, needs no credentials, and exists because a tag is not reversible: PyPI refuses a re-upload, :latest is on someone's disk before you notice, and a registry manifest is what an IDE offers people. If it fails on your laptop, the tag was going to fail an hour later in public. --publish is the escape hatch for when CI is unavailable, and it makes you type the version to confirm.

What CI checks

.github/workflows/ci.yml runs on every push and pull request, in four jobs that fail for different reasons so you can tell at a glance which:

Two of those exist because of specific failures. The wheel check is there because uv build succeeding proves the metadata parses, not that ddflow help works — a wheel missing its templates fails on the user's machine. And gitleaks is there because this project has already committed a live API key: a secret in git is a leaked secret, rotation is the only remedy, so the check that matters is the one that runs before every push.

bandit deliberately skips tests/, which use subprocess and temporary paths constantly and by design. A scanner that cries wolf on every fixture is a scanner nobody reads.

The same checks run before you push. Install the pre-push hook once per clone:

$ ln -sf ../../scripts/ci/pre-push "$(git rev-parse --git-common-dir)/hooks/pre-push"

It runs what CI runs, from .pre-commit-config.yaml: ruff, the wheel probe, bandit, the dependency allowlist, gitleaks over the commits being pushed, the unit suite and the scenarios, and a schema check of the workflow files. The build probe and the allowlist are scripts/ci/ files that CI calls too, so the two cannot drift.

It runs them in a scratch worktree of the commit being pushed, not in your checkout — which is why it is not pre-commit install. pre-commit fails a hook when anything in the tree changes while it runs, and ddflow agents append to the committed event logs constantly: in your own tree the test hook failed with every test green. A scratch tree has no other writers, and it checks what you push rather than what is uncommitted.

The unit suite and the scenarios run as one pytest session, one worker per CPU (-n auto) with work-stealing, so the scenarios overlap the unit tests instead of following them: about a minute on a 192-thread machine. The scenarios take CI's no-model-server path (DDFLOW_DEMO_NO_REVIEWER=1), since a real review there costs minutes and is not what CI checks.

CI's tests job runs each matrix Python (3.11, 3.12, 3.13) as a uv-managed standalone build (uv python install plus UV_PYTHON_PREFERENCE=only-managed), never the runner's system Python. To reproduce locally: uv python install 3.12 and run UV_PYTHON_PREFERENCE=only-managed uv run --python 3.12 pytest tests/.

It is a push hook, not a commit hook, because ddflow's own hooks own the commit (ddflow hooks install), and because minutes per commit teaches --no-verify. SKIP=tests git push skips one check by id. It does not run the 3.11/3.13 matrix, CodeQL, pip-audit or the load suite.

Any LLM as a reviewer — local, remote, SaaS, or a CLI

The critic and rubber_duck gates are run by ddflow, not claimed by the agent. Point them at whatever you have:

ddflow reviewers presets            # 19 ready-made provider settings
ddflow reviewers add --preset ollama --model qwen3:8b
ddflow reviewers detect --write     # probe local ports, register what is serving (machine-local)
ddflow reviewers test               # send a known-buggy diff, check the reply

Four backends, because "any LLM" means four wire formats in practice:

command is the escape hatch that makes the answer to "can it use X?" always yes: a model with no HTTP API, behind a corporate gateway, or wrapped in an in-house tool is still usable, with no SDK and no dependency.

Who may add a reviewer. A reviewer decides whether a review counts as independent, so an agent must not be able to mint one (decision D-reviewer-trust):

• A kind = "command" reviewer runs any program and can print any verdict, so only a person adds one: by editing .ddflow/local/reviewers.toml, or with ddflow reviewers add from their own terminal. Every agent surface refuses it — ddflow_configure, config --append-toml, and any command run under --agent or DDFLOW_AGENT, or in a Claude Code shell (CLAUDECODE, the one harness marker known for certain; another harness is recognised by the --agent/DDFLOW_AGENT its ddflow setup passes).

• A reviewer a tool writes (reviewers add, reviewers detect --write, ddflow_configure) is recorded with who wrote it, and its reviews do not count toward the cross-family rule until a person checks the entry and runs ddflow reviewers approve <name> (ddflow reviewers approve alone lists what is waiting). Approval is for the entry as it was: a tool changing its endpoint, model, family or command makes a new, unapproved reviewer; tuning max_tokens or hedge does not. reviewers approve has no MCP tool and refuses under an agent identity.

• A reviewer no tool wrote — every entry configured by hand, including all of them from before this rule — counts exactly as before.

Like human gates, this makes a forged reviewer visible in the log; it cannot stop a shell edit of the local files.

The approval is the shared approve-by-digest primitive (services/approval.py, B-uni-approval). It records one event, approval.granted, with the subject (reviewer:<name>), the digest a person approved, the OS user, the host and human. It has one check: the digest of what is about to run must equal a digest a person approved, so a later edit is not approved. A single-use approval also carries the hash of a token minted for the person; whoever presents the token may act once (approval.used), and the token itself is never written. The planned approvals (skill installs, upstream reports, drafts, schedule escalations, upgrades) use the same primitive. A log from before it holds reviewer.approved, which still counts.

[[reviewer]]
name   = "local-qwen"
kind   = "openai"
base_url = "http://127.0.0.1:11434/v1"
model  = "qwen3:8b"
family = "alibaba"              # must differ from the author's family
gates  = ["critic"]
# Optional: start it if it is not already running.
launch = { command = "ollama serve", ready_url = "http://127.0.0.1:11434/v1/models" }

[[reviewer]]
name    = "claude-via-cli"
kind    = "command"
command = "claude -p --model {model}"
model   = "claude-sonnet-5"
family  = "anthropic"
gates   = ["rubber_duck"]

Auto-launch is opt-in per reviewer — starting a multi-gigabyte model server as a side effect of asking for a code review is a surprise nobody wants by default. When it fails it never leaves a half-started process behind, because a reviewer stuck "starting" forever is indistinguishable from one that is down except that it also holds a process.

Keys are never written to the config. Only api_key_env, the name of an environment variable — the config file is committed, and a key in git is a leaked key.

To not export the variable in every shell, a reviewer in the git-ignored .ddflow/local/reviewers.toml may also name api_key_file = "~/.config/me/gemini.key". The variable wins; the file is the fallback. The file must be a regular file you own with no group or other access (chmod 600, as ssh insists), is read at each call and never cached, and a missing, empty or too-open file makes the reviewer UNAVAILABLE with the reason. The path is data and the key never is: it appears in no log, event, error, review record or listing. api_key_file is refused in the committed .ddflow/config.toml.

Every way of not reviewing is reported distinctly, with its remedy: no key names the variable, a missing CLI names the binary, a dead server names the launch block you could add, a non-zero exit shows stderr, and empty output on exit 0 is UNAVAILABLE rather than "no findings" — the vacuous pass arriving by the most innocent-looking path there is.

Reasoning models need a large max_tokens. Default 32000, measured not guessed: on Qwen3.8-Flash-Next over a 30 KB diff, a 6000-token budget produced zero characters of content — the whole budget went to reasoning and the reply was truncated. That case is reported as TRUNCATED with the remedy named, never as an empty completion and never as a clean review. A chunk lost that way on every copy is tried once more, split in halves when it splits, before it is reported unreviewed.

Leave temperature unset unless you mean to override the model. Unset, the request carries none and the server applies the model's own recommended sampling. A reasoning model sampled cold (ddflow used to send 0.3) can loop — "let me reconsider" — until the whole budget is gone; at its vendor's recommended 1.0 the same diff reviewed fully.

Reviews run in parallel, and each chunk is raced. A large diff is reviewed in chunks (max_chunk_chars). They go out together — by default every chunk and every copy in one wave (up to 32 requests); max_concurrency caps it — so a review takes as long as its slowest chunk rather than the sum. Each chunk is also sent hedge times (default 2): the first copy that answers on contract is the chunk's review, and the rest are cancelled — the connection is closed so the server aborts the request, a command reviewer's process group is killed. A chunk whose every copy failed is reported exactly as before; nothing is counted as reviewed that was not.

Why: a reasoning model's time is its reasoning length, and that is random. Measured on a LAN vLLM: the same 3 KB diff took 64–391 s across eight identical calls, about one call in four on a hard diff never answers before max_tokens, and the server was idle — eight concurrent requests each ran 28% slower for 5.7× the throughput. A 24 KB review went from 1 274 s (chunks in sequence) to the time of its slowest chunk.

[[reviewer]]
# ...
hedge = 3             # copies per chunk; 1 turns racing off
max_concurrency = 10  # cap on requests in flight; 0 (default) = all in one wave

On a metered API a cancelled copy is still billed for what it generated before it was stopped: hedge = 1 if cost matters more than time there. On a single-slot local server (an ollama with OLLAMA_NUM_PARALLEL=1) the extra copies queue behind the first ones, which are always sent first; there, set max_concurrency to the server's slots so a queued request's timeout_s does not run out while it waits.

What a review reports, and how to finish one that was partial. Every chunk that was not reviewed is named, with its files and its cause: a chunk whose generation ran out the clock is labelled did not converge (the model was reachable), distinct from an endpoint that was down. No chunk is header-only and none is dropped; hunk-header context is stripped so a finding cannot cite a function the diff does not touch. Progress is printed per chunk, the caller's lease is renewed while the review runs, and extra_rules on a [[reviewer]] reaches its prompt. Findings are numbered #1..#N.

A review that outlives its caller keeps its findings. Each chunk's whole reply is appended to .ddflow/local/reviews/<item>.<gate>.<reviewer>.<run>.jsonl (git-ignored; one file per reviewer and run, so a re-review never overwrites an earlier one) the moment it arrives, and the gate evidence records output_file (repo-relative) and output_digest beside the per-finding text, so a call that is cut off (a long critic over MCP) loses nothing. Over MCP, ddflow_review sends notifications/progress for each progress line when the client passed a progressToken, which resets a client's idle timer.

ddflow review T1 --gate critic --chunk 2,5      # re-review only chunks 2 and 5 of that review
ddflow review triage T1 --gate critic --finding 3 --refuted --probe "tests/test_x.py::t shows..."
ddflow review triage T1 --gate critic --finding 1 --confirmed --probe "fixed in 4f2a, test_y"

--chunk re-runs the named chunks of the same cut — the same diff, max_chunk_chars and reviewer, checked before anything is sent — and merges their coverage into the gate's recorded outcome. review triage records what became of each finding as its own event (review.triaged): refuted, with the run that shows it false, or confirmed, with the fix or test that answers it. The gate's outcome is not changed — a review that reported findings stays failed, and that does not block completion; the log now shows what became of each finding (once the round budget is spent, the triage that settles the LAST finding records the gate passed on refutation, flagged: see below). Likewise, when every finding of a failed round was refuted, the triage that settles the last one records the gate passed on refutation at once, flagged and with each probe as evidence ([review].settle_all_refuted, default true) -- no hand-recorded pass is needed. Triage appears in gate status and show, and a re-review carries it over only for a finding whose text is identical (decision D-review-triage). Over MCP it is ddflow_review_triage; --chunk is an argument of ddflow_review. The triage verb is required: --finding/--refuted/--confirmed/--probe on a plain review are refused (exit 1) before any reviewer is contacted, as is --chunk on review triage. Finding numbers are per gate, so --gate is never defaulted on review triage: omitted, it is refused (exit 1) naming the gates that have findings when several do, and resolved to that gate (the output names it) when exactly one does. ddflow_review_triage's gate works the same.

ddflow review show <item> [--gate G] lists the recorded round's numbered findings -- the #N that review triage takes -- each with severity, location, digest, triage state and probe; omit --gate for every gate that has a round (a gate whose findings blob is gone is named, not hidden). Over MCP, ddflow_review_triage without finding, verdict and probe lists them.

The review-round budget ([review]). A shipped default in every project, with no configuration: a gate (rubber_duck, critic) gets 2 review rounds per item, full and delta alike (decision D-gate-economy 2: a delta after the cap was the way around it), then a third is refused (exit 3) with the way forward — later rounds each find fewer defects than the one before (measured: rounds 3 and later yielded about a quarter of the confirmed findings). What stays allowed after the cap:

ddflow review triage T1 --gate critic --finding 2 --refuted --probe "..."   # settle what is left (always allowed)
ddflow review T1 --gate critic --force --reason "..."   # one more round; the reason is recorded

Passed on refutation (decisions D-gate-economy 2, D-unify 5). Once the budget is spent, the review triage that gives the gate's LAST finding a verdict records the gate passed itself: the review's own evidence and reviewer, plus passed_on_refutation (how many findings were refuted and confirmed, and the rounds used). The pass is flagged, never silent: gate status shows -- PASSED ON REFUTATION beside the triage counts, status counts such gates (refuted_passes in --json), the brief names those of unfinished items, complete prints a PASSED ON REFUTATION: line for each (refuted_passes in its result), and ddflow gate list --refuted (MCP ddflow_gate_list with refuted=true) lists every one for the operator to spot-check; --since <ISO date or timestamp> (MCP since) keeps the ones recorded at or after it (refused without --refuted). Without --refuted, gate list lists the gates the project defines. A gate whose findings were ALL confirmed (and fixed), none refuted, is recorded passed too but is not flagged: gate status shows -- findings fixed and gate list --refuted leaves it out; a mix of refuted and confirmed is flagged with both counts. A finding still without a verdict holds the gate where it is, and the triage output says so: settle it, or ask the operator for one more round (--force --reason, theirs to grant). With review.max_rounds = 0 (no budget) a gate is never passed this way; a re-review settles it.

A full round is any review that can cover the item's whole diff. What is not one is judged by what it covers, not by the flag: --delta, a --commit <sha> or --base <ref> at or after the head the gate's last review covered (so it can only be narrower), and a --chunk re-run of a recorded review. A delta still counts against the cap (a --chunk re-run does not: it repeats part of a recorded round); a --base or --commit that reaches back past that head counts as a full round. Rounds are counted from the log's recorded reviews (review_kind, round, rounds and reviewed_head in the gate evidence), so a re-claim, a delta or a manual gate skip does not reset the count; a round that reached no reviewer is not counted. --delta reads the reviewed head from the log too, so it works after a re-claim or a gate skip; it is refused only when no review was ever recorded or nothing changed. A delta is the item's OWN change: after git merge main into the branch, the commits that came in from the base branch are not sent (they were reviewed as their own items). The delta diffs the tip against the reviewed head merged with what came in (keeping the incoming side where the two conflict), so the branch's own commits and where its merge result departs from main are what the reviewer sees: a conflict resolved the item's own way (overriding another item's change) is reviewed, one where it took main's side is not. Criss-cross histories (more than one merge base) merge each base the reviewed head lacks. When that merge cannot be written (a modify/delete conflict, say), the delta is the item's whole own change on top of the merged-in main commit, or, with several merge bases, everything since the reviewed head. A delta of the item's tree or branch whose diff cannot be produced is refused with the reason, never reported as "nothing changed".

A re-review sees the whole diff and what was said last time (decision D-gate-economy 3). Once a gate has a recorded review, a plain ddflow review T1 --gate critic is a full round over the item's whole diff against its base, and the prompt lists each earlier finding with the author's triage -- confirmed (check the fix), refuted (check the probe) or untriaged -- asking the reviewer to report one again only if it still holds, then to look for new issues; the output says re-review: the whole diff of T1, with N previous critic finding(s) and their triage. A delta that showed the reviewer only the follow-up commit kept reporting the fix as absent, because the code it fixed was not in what it saw. A re-review counts against review.max_rounds like any full round.

One review for both gates of a small change (decision D-gate-economy 2). ddflow review T1 --gate rubber_duck,critic on a change with fewer than review.combined_under_lines changed lines (default 150; 0 = never) sends ONE reviewer request whose intent names both lenses -- refute the change, and criticise its design and tests -- and records the outcome for each gate with the same evidence and a shared review_id (each gate's findings are triaged on that gate); the output starts combined review <id>: N changed line(s) ...; it is a full round of each gate. A larger change runs one review per gate, one after the other. --chunk, --delta, --commit and --base work on one gate's record, so with several gates they run per gate; so does a pair where either gate is out of rounds, is served by other reviewers than the first, or is not one of the two review lenses.

Delta re-reviews, on request. --delta (MCP delta=true), or review.delta_default = true for every plain re-review, is for a diff too large to send twice. Once a gate has a recorded review that reviewed the whole diff, a delta reviews only the commits since the head that review covered (reviewed_head), says so (delta review of 1 commit since a1b2c3d4e5), is not a full round but counts against the cap like one. Its findings and coverage are merged into the gate's record: earlier findings stay where they were (their #N and their triage, which is keyed by the finding's exact text, are unchanged), a byte-identical finding is not duplicated, the delta's new ones are appended and the output maps each to its place in the record; the gate's outcome is the delta's own, except that a clean delta does not pass while an earlier finding on the record has no triage verdict (it records failed and says so; review triage settles each, then the next clean delta passes). gate status shows the rounds separately (1 full round, 2 delta rounds). --full forces a full round (counted against review.max_rounds; combined with --delta it is an error). A delta falls back to a full round, and says why, when the reviewed head is no ancestor of the current head (the branch was rebased or amended) or, for the automatic delta, the earlier review was partial; a review that never reached a reviewer records no reviewed head at all, so the next delta starts from the last real one; with nothing changed since the reviewed head the review is refused and names --full. A --force --reason review is a full round, like --full (and, like --full, an error with --delta); a --chunk, --commit or --base review is not second-guessed. review.delta_default = false (the default) makes every re-review a full round with the previous findings. Knobs, changeable at every layer:

ddflow config review.max_rounds 3                 # this project (committed .ddflow/config.toml)
ddflow config review.max_rounds 0 --local         # this machine only (.ddflow/local/config.toml)
ddflow config review.delta_default true           # automatic deltas again (add --local for this machine)
ddflow review T1 --gate critic --full             # one full round, whatever the default
                                                  # (`--set KEY VALUE` is the same)
DDFLOW_REVIEW_MAX_ROUNDS=0 ddflow review ...      # one run

Over MCP, ddflow_configure accepts review.max_rounds, review.on_exceed and review.delta_default on either layer (local=true), and tells the operator in its reply and in a session note — an agent does not lift the cap for itself; --force --reason is CLI-only and ddflow_review takes delta=true and full=true. ddflow config --explain --filter review. documents the knobs. A tool built before this knob skips an unknown [review] key with a note rather than failing.

Companion tools

ddflow imposes the order and demands the evidence. It does not perform the judgement inside most of its gates: standards wants an automated standards review, research wants documentation to check a claim against. (rules wants memory of the last time somebody hit this, and ddflow serves that one itself: ddflow brief shows the lessons and the operational memory.) A project that installs ddflow and stops has the others wired to nothing — and because an agent gate passes on an assertion, that gap is invisible in exactly the way the rest of this design exists to prevent.

So the gap is named:

$ ddflow companions
Companion tools

  [x] context7   Current library documentation
       gates: research, standards
       registered for: claude, cursor
  [x] roborev    Automated second-opinion code review
       gates: standards, bug_hunt, dedupe
       installed (roborev 0.9.1). A cli tool — the agent shells out to it, so
       there is nothing to register.
  [+] codeguide  Language and framework coding standards
       gates: standards
       installed (…) but no agent is configured to launch it.
       -> ddflow companions add --id codeguide
  [ ] sequential Structured step-by-step reasoning
       gates: research, rubber_duck, bug_hunt
       not here: `npx --no-install @modelcontextprotocol/server-sequential-thinking` exited 1
       -> ask the operator, then: npx -y @modelcontextprotocol/server-sequential-thinking

Gates in this project's task pipeline with no companion behind them:
  implement, rubber_duck, critic, unit_tests, bug_hunt, dedupe, merge

The gap is also told to the agent, not only shown: the MCP handshake and the SessionStart brief (hook path, nothing probed, "not checked" stated as such) both list the default companions that are not wired up. To install them with the operator's consent run the install-companions prompt (MCP: /mcp__ddflow__install-companions; ddflow prompts show install-companions elsewhere).

A companion counts as registered only when an entry under its id can actually launch something: a bare or junk table with its name does not hide a real launch registered elsewhere, and does not pass for one. A non-table mcp_servers is refused with the parser's reason, and companions add appends TOML only when the result still parses. A stale entry under the id is refreshed in .codex/config.toml too, as in .mcp.json: the old [mcp_servers.<id>] table (and its sub-tables) is cut out and the registry's launch appended. A table that cannot be cut out safely (an inline or dotted form) is refused with "replace it by hand", and a launch that already runs under another name is not doubled.

Three states, reported separately because the remedies differ: registered, installed but not wired up (one command away), not installed (with the command and the URL). ddflow adopt prints the same summary, so the gap is visible at adoption rather than discovered six tasks later. Exit 2 when a default companion is missing — "no data", never collapsed into "no problem".

A server is found by its launch as well as its name. A project that registered codeguide-mcp as coding-guides before ddflow knew it reads as registered for: claude (as `coding-guides`), and companions add leaves it alone instead of writing a second copy under the id. "The same launch" is the same command (by basename) with the companion's arguments in order — only flags, and the value of a flag known to take one (-e TOKEN) may sit between them, the server's own arguments may follow, and a version tag on an npm package (@latest, @1.2.3) is ignored. Another image, package or launcher is another server, and so is reordered arguments.

rules never shows as uncovered: ddflow brief serves it from ddflow's own lessons and operational memory (ddflow memory add|list|forget), which is the job the memory companions below were once recommended for.

Servers and command-line tools are different things, and the registry says which: kind = "mcp" is registrable into an agent's config, kind = "cli" is a tool the agent shells out to. OptMem and pre-commit are examples — real tools with no MCP mode, so companions add refuses them and says why instead of writing a launch entry that would fail its first handshake. A cli companion counts toward its gate's coverage once it is installed; registered is a state it cannot reach.

Your stack needs servers this registry cannot know about. ddflow prompts show research-companions walks an agent from the pipeline's uncovered gates, through the repository's actual manifests, to candidates checked against their primary sources — provenance, maintenance, what they execute, what credential they want — and produces [[companion]] blocks you can read and delete. It proposes; you install. A rejection is part of its report, so the next session does not re-research it.

Registering is previewable. ddflow companions add --dry-run (and ddflow_companions_add with dry_run=true) reports the exact config entry it would write and writes nothing — not the file, not even its parent directory. The handshake tells an agent to dry-run first and show the operator the actual entry rather than a description of it, because registering changes which processes their agent launches. The preview is asserted to match what the real write produces; a preview that drifts from the write is worse than none, since the operator has now signed off on it.

ddflow never installs anything itself — running an install command on someone's machine is the operator's decision. What it does instead is instruct the agent to ask: the MCP instruction block lists each missing companion with the gates it serves and the exact command that would install it, and tells the agent to put that to the operator early, install it if they agree, and record the affected gates unavailable if they decline. Never on its own word.

companions add also refuses to register a server that is not present: that writes a launch command which fails mid-task, at the moment a gate told the agent to reach for it. Detection is read-only and bounded — and when it has not run, the state is reported as unknown, not as absent. ddflow companions probes; the MCP handshake does not, because making an agent wait on npx before it can do anything is the wrong trade.

Verifying that a companion is a server. Detection says a binary of that name exists; it cannot say the binary speaks MCP (B113 was a registry entry whose launch command was not a server at all). ddflow companions --verify (MCP: ddflow_companions_verify) spawns each MCP companion's command args, sends a JSON-RPC initialize on stdio and requires a response, then stops it. A response, even a JSON-RPC error, is speaks_mcp; a missing command, an early exit, or a binary that only echoes the request back (cat) is not_mcp, exit 1; no answer within 30 s is unknown, exit 2, because a cold npx cache is slow and silence is not proof. Without --id it launches only companions that are registered with an agent or detected as installed (launching a registry entry that is merely known would download it); --id a,b launches exactly those. It is opt-in and never on the scan path: ddflow companions and the MCP handshake still launch nothing.

Adding another is a TOML block in .ddflow/companions.toml, not a patch:

[[companion]]
id      = "my-linter"
title   = "House linter"
gates   = ["standards"]
detect  = ["my-linter", "--version"]
command = "my-linter"
args    = ["mcp"]
install = "cargo install my-linter"

Pin files regenerate; they are never hand-merged

A golden snapshot, a pinned CLI transcript or a generated table changes whenever a knob, command or event kind does, so every parallel branch touches the same few files. ddflow re-records them instead of leasing or merging them by hand. A project lists its pin kinds as data in .ddflow/pins.toml ([pins].recipes_file), one [[pin]] per kind with files, check (exit 0 = current) and regen (re-records only what differs):

ddflow pins regenerate --check      # list the stale pins, write nothing (exit 1 if any)
ddflow pins regenerate              # re-record exactly the stale ones
ddflow pins regenerate --only NAME  # one pin
ddflow pins regenerate --install    # optional: the git merge driver + post-merge hook

--install (also run by ddflow adopt when the recipes file exists) writes <file> merge=pinsregen to .gitattributes, sets merge.pinsregen.driver and installs a post-merge hook: on a conflict in a pin file the driver keeps our side, the hook then regenerates the stale pins and folds them into the merge commit ([pins].after_merge = amend, leave or off). List the pin files in [lease].shared_globs so a lease on them is not exclusive. A recipe never runs under xdist.

The hook never makes a merge wait for a test suite. It runs only the recipes that allow it (a [[pin]] with after_merge = false, the golden-snapshot and transcript suites in this repository, is left to an explicit ddflow pins regenerate), bounds each command by [pins].after_merge_timeout_s, and reports a recipe it could not finish as unavailable, never as failed. Two re-recordings never overlap in one tree: the second is unavailable.

Finding a test polluter

A test that passes alone and fails only after others have run is being polluted: a leaked environment variable, a module-level cache, a file left behind. ddflow bisect finds which earlier file does it (illustrative output):

$ ddflow bisect tests/test_report.py::test_totals --cmd 'pytest -q -p no:randomly {tests}'
victim: tests/test_report.py::test_totals
candidates before it: 212

Run before the victim, these make it fail (remove any one and it passes):
  tests/test_cache.py

9 run(s)

It confirms the victim passes alone and fails after every candidate, then delta-debugs (Zeller's ddmin) the candidates to a 1-minimal set -- usually one file, two when the pollution needs an interaction -- in about 2 * log2(n) runs. Candidates are the files that sort before the victim's (--glob, default tests/**/test_*.py) or --candidates a,b in the order given.

It is driven by your command, not by ddflow running your tests. The original idea assumed ddflow owned test execution; it does not, and building that means understanding every test framework. The only requirement is a command that runs a list of tests and exits non-zero on failure, with {tests} where the list goes (pytest -q {tests}, go test {tests}, npm test -- {tests}). Turn off test-order randomisation in it, or a shuffled victim will not reproduce. The search never looks inside a test.

Probes are three-valued, like every check here: a run that could not be made (timeout, a command that will not spawn) is unavailable and ends the search with exit 2; it is never counted as a pass or a fail. Exit 2 also covers "the victim fails alone" (not an ordering problem), "it passes after every candidate" (flaky, or not about earlier files) and an exhausted --max-runs (the smallest set so far is printed). --repeat N runs each probe N times and counts any failure, for a pollution that shows one run in three.

Wiring it into your agent

ddflow adopt --agents claude,cursor,codex writes everything below. This table is what it writes, so you can check it or do it by hand.

7 more are supported with no MCP file to write — a verified absence, not an unresearched gap. adopt writes the delta doc and the AGENTS.md block and names the one manual step. Inventing a path would be worse: ddflow would write a file the agent never reads, and you would believe it was wired up.

One set of rules, every agent

AGENTS.md is the cross-agent convention and most of the 22 read it. Seven do not read it first, or at all, so adopt writes the same managed block into their own surface too:

The rules are inlined, not pointed at. A one-line "see AGENTS.md" stub was the obvious design, and ddflow's own notes had already refuted it: a link is only followed if the agent chooses to follow it. A rule that binds only when the model feels like opening a file is not an enforced rule.

That means several copies of one text, and the answer is that a check owns them: one generator, a managed ddflow:begin rules/work-queue block in each, and ddflow doctor comparing every copy against the generator. Five kinds of break are reported and each fails doctor:

$ ddflow doctor
note:    QWEN.md's ddflow section is from an older version and has drifted
note:    .clinerules/ddflow.md exists but its ddflow section was removed
PROBLEM: .goosehints does not exist — the agent has no project rules at all
PROBLEM: .cursor/rules/ddflow.mdc exists but does not bind: `alwaysApply` is not
         true, so the agent may never load it
PROBLEM: .aider.conf.yml exists but does not bind: it does not list `AGENTS.md`
         under `read:`, and Aider loads no instruction file it was not told to load

Files the project already owns — QWEN.md, replit.md, .goosehints — get a block merged into them; your own content stays. Aider's read: list is extended, not replaced. Adopting is idempotent: re-running never appends a second block.

Every other agent reads AGENTS.md directly, which is the point of it being canonical.

Cursor gets its own rules file because its precedence puts project rules above AGENTS.md — writing only AGENTS.md there would be writing to a file the agent outranks. adopt merges into these files rather than overwriting: they hold your other servers and your other rules, and a tool that stomps them is a tool you run once.

The MCP entry is one line in any of them:

{ "mcpServers": { "ddflow": { "command": "uvx", "args": ["ddflow-mcp"] } } }

uvx fetches and runs it in an ephemeral environment on first use — no clone, no PYTHONPATH, no install step to forget. Prefer Docker? docker run -i --rm -v "$PWD:/repo" ghcr.io/delian/ddflow-mcp, which needs the repo bind-mounted because ddflow operates on your actual git checkout.

Standalone, with no MCP at all, is a first-class mode rather than a fallback. Add to AGENTS.md / CLAUDE.md:

This project's work is a queue managed by ddflow. Before doing anything, run
`ddflow brief`. Claim before you edit (`ddflow claim <id>`), satisfy every gate
(`ddflow gate status <id>`), then `ddflow merge` and `ddflow complete`.
Never pass a gate you did not perform — record `unavailable` with the reason instead.

That is the whole integration. An agent with nothing but a shell can drive the entire workflow, which is why MCP is a convenience layer here and never a requirement.

From plan mode to the queue

Agents plan well and forget reliably. A plan that lives in a chat transcript is gone at the next session; a plan in the queue survives, fans out to parallel agents, and carries its own gates.

Tell the agent, at the end of planning:

Put that plan in ddflow before you build any of it. One phase for the whole plan, one
task per independently-shippable step. Declare each task's globs — the files it will
write — and its needs, the tasks that must finish first. Then show me `ddflow next`.

A glob names files as they are when the task is filed. When a module later becomes a package (api/reporting.py split into api/reporting/), an open task's lease on the old name covers none of the new files, and the conflict detector cannot see the overlap: ddflow doctor notes each such glob with the ddflow update <id> --globs ... that corrects it.

What the agent does with that:

$ ddflow phase add P3 --title "Rate limiting"
$ ddflow task add P3.T1 --phase P3 --globs 'limiter/**'      --title "token bucket"
$ ddflow task add P3.T2 --phase P3 --globs 'api/middleware/**' \
      --needs P3.T1 --title "wire it into the request path"
$ ddflow task add P3.T3 --phase P3 --globs 'docs/**' --needs P3.T2 --title "document it"
$ ddflow next
Ready (1 ready, 0 running, 2 blocked):
  P3.T1  token bucket
      writes: limiter/**
  (blocked) P3.T2: deps — P3.T1 is open
  (blocked) P3.T3: deps — P3.T2 is open

The two fields that do the work are --globs and --needs. Globs are how two agents are stopped from editing the same file: claim refuses an item whose writes overlap one already held, and names what to take instead. Needs are how ordering is enforced without anyone remembering it. A plan whose tasks declare neither is a list, not a queue — it will look parallel and then two agents will fight over one file.

ddflow split <id> --into a,b,c exists for when a task turns out to be three, which is the normal case rather than a failure of planning.

When a companion is missing

ddflow imposes the order and demands the evidence. It does not perform the judgement inside most gates — that is what the companion tools are for. So the honest question is what happens when one is absent, and the answer is deliberately never "the gate passes".

The rule, and it is enforced: a gate whose tool could not run is recorded unavailable with the reason, never passed. ddflow complete reports those as a coverage gap on the completion event, so a finished item never silently implies that a check happened. Set [gates].unavailable_is_failure = true and a gap blocks completion outright.

ddflow companions reports four states, and the difference between the last two is the whole point: registered, installed but not wired up (one command away), missing (with the install command and the URL), and not checked — because the MCP handshake does not probe, and "nobody looked" must never render as "not there".

ddflow never installs anything. Detection is read-only and the report is advice. ddflow companions add --dry-run shows the exact config entry it would write, so an agent can show you the change before making it.

Adopting a project that already has history

A queue that starts empty tells the next agent "nothing is in flight" about a repository with three branches in flight and forty open items in a todo file — and the agent believes it, because the tool said so. That is worse than having no tool at all.

$ ddflow import                      # looks; writes nothing
What this project already has (nothing written yet):

  314 phase(s):
    [ ] 142.A     the scaling-law advisor is wrong (P0; CONFIRMED)   docs/todo.md:26517
  1170 task(s):
  442 lesson(s):
  ...
  47 memory(s):
    [ ] M-0002    Hardware: 8x H200 GPUs on this box, usually idle.  .agent_memory/LOG.txt:3

  NOTE: 3631 already-ticked task(s) were NOT imported. They are history, not a queue.
  NOTE: 32 phase heading(s) say the work is finished while their checkboxes are still
        unticked: 99 (4 open), 103 (3 open), ... Ask the operator which is stale.

$ ddflow import --apply              # writes them, each recording its source line

Seven sources, all optional, all in the places projects actually keep them:

What it will and will not decide for you

Mechanical, and verifiable: a ticked checkbox is a fact, a ## heading is a section, a branch with unmerged commits is work. The id in ### 142.A — … or - [ ] **WFOPT.4.6** — … is read, not invented, so the imported queue uses the ids the project has been writing in commit trailers for months.

Judgement, and yours: which open items are actually live, what each task writes, what depends on what. The /import-existing-project prompt walks an agent through that with the operator. It is not automatable, and a confident guess produces a wrong queue the scheduler then hands out.

Four guard rails, each of which exists because the alternative is silent:

• Dry run by default. --apply writes. Looking is free and never a side effect.

• Finished work stays out — it is history, not a queue — except a completed item that open work depends on, which comes along as done so the open item is not stranded on an id the queue has never heard of. A needed phase is judged over every task under it, so a ticked task pulled in as a dependency does not keep the phase open, and a plain import completes a finished phase that open work needs (re-running repairs one left empty).

• [importer] max_tasks (default 200) refuses a whole history. An import writes events into a log that is committed to git; one real repository yielded 4,799 checkboxes. Over the cap it proposes none and says so — the phases are withheld with them, because a queue of empty phases is not a smaller import, it is a misleading one.

• Readable ids. A phase or task id made from a heading is cut at a WHOLE WORD, never inside one, at [importer] phase_slug_chars (default 32) and task_slug_chars (default 40; at least 8 each; only a first word longer than the limit is kept whole; a later word that does not fit is dropped): Phase 106C tool-rl multi-agent run gives PHASE-106C-TOOL-RL-MULTI-AGENT, where it used to give PHASE-106C-TOOLRL-MULTI. A heading that declares its own id keeps it, and a re-import recognises the older truncated ids (legacy_ident), so nothing is minted twice. Set the two knobs BEFORE the first import: a re-import under different values derives different ids and proposes those items again.

• Idempotent. Ids derive from the source, so re-running after you edit the todo adds what is new and leaves the rest alone. A second run over an unchanged project exits 2.

What an unticked box means, and where things live

An open box is not always work. The import reads the project's own dispositions — the vocabulary and positions are those of the picker ddflow was extracted from, and agree with it on 1,166 of that repository's 1,170 open boxes. Of the four, two are the picker's own false positives ("cells run / skipped" in plain prose) and two are recorded non-findings with an unclosed "(… refuted it" aside, which ddflow closes:

[importer] archive_globs names plan files that are history until a section is named (a 20,000-line legacy docs/todo.md): their open boxes import blocked, and ddflow unblock <phase> releases a whole section at once.

Every path pattern ddflow reads (claims, shared globs, [enforce] doc and code globs, archive_globs, lesson globs) means what git means: * and ? stop at /, **/ is any number of directories (including none), [...] is a class; one implementation, ddflow/core/globs.py.

Each source family's location is a knob — todo_globs, lesson_globs, lesson_summary_globs, decision_globs, research_globs, journal_globs, memory_globs — and a set knob replaces the defaults, because the same filename means opposite things in different projects (docs/LOG.md is one repository's whole journal and another's generated index of it; an Index section is never imported).

Lessons are split at the level they actually live at: ### L100. … entries grouped under ## <date> headings import one per lesson with their own ids, so [L147] cross-references still resolve — and so do hyphenated ids (L-12, R-7), kept as ids in headings and citations alike. A lesson's **Compressed:** paragraph becomes its summary; a hand-written lessons-summary.md bullet that cites exactly one lesson becomes that lesson's summary, and every other bullet becomes a consolidated lesson tagged summary. A GENERATED summary file is skipped. ddflow render writes them all back out as docs/ddflow/LESSONS-SUMMARY.md (also ddflow://lessons-summary).

An import adds nothing twice, and says what repeats. Lessons, decisions and research are weighed against every record already in the log (and a summary bullet against the lessons of the same import) with the same similarity engine and [dedupe] thresholds an add uses. Text identical to a held record is left out and counted; a record scoring at least [dedupe].ask_threshold (0.55, with min_words content words) is left out and LISTED in the plan's notes as lesson LS-git-rebase ~ L12 (0.74) docs/lessons-summary.md:9 -- the candidate id and score -- so the operator or onboarding agent decides: nothing is written for it, and filing one anyway is ddflow lesson add (or decision add / research add) once the existing record is judged not to say it. No score separates a duplicate from a related record, which is why it is reported rather than decided. On a real project's lessons-summary, 68 of 86 bullets that restate a corpus lesson were reported this way. Tasks and phases are not checked (they carry dependencies); [dedupe].on_match = "warn" reports the same list but imports them anyway, and "off" turns the check off. Under the shipped default, ask, near-duplicates are withheld. Re-running over the same files adds nothing.

Verifying an import, at any time

The import's weak spot was never the parsing. It is everything after --apply: 1,170 tasks arrived in the real-corpus run, and the workflow prompt tells an agent to give each one globs and declare its dependencies. Nothing checked whether that ever happened — and an imported queue nobody finished misrepresents the project exactly as an empty one does, believed harder because a tool produced it.

$ ddflow import --verify
Imported between 2026-09-25 and 2026-09-25:

       4 decision(s)
    1727 journal(s)
     442 lesson(s)
      47 memory(s)
     314 phase(s)
      72 research(s)
    1170 task(s)

Left to decide or fix:
  - 1078 imported task(s) declare no globs, so the conflict detector cannot protect
    them and two agents can be handed the same file: OPIK.1b, OPIK.2, ...
  - 27 phase(s) say the work is finished while a task under them is still open: 99,
    103, 115.D.2, ... Ask the operator which is stale before anyone claims from them.

Three answers, three exit codes, because collapsing them loses the one that matters:

It reports status (what is imported, per kind, and when), whether it is still true (what a re-run would add, which sources yielded nothing, which source files have since vanished), and whether anyone finished it (tasks with no globs; phases whose heading claims SHIPPED over an open task).

It deliberately does not repeat ddflow doctor, which already reports unresolved dependencies, duplicate globs and cycles. Two commands reporting one defect in different words is how an operator learns to read neither.

Provenance is a field, not prose. Item.source is docs/todo.md:41; the body still says "Imported from docs/todo.md:41." for a human reading ddflow show. Answering "which items came from the import" by regexing that sentence would mean the day someone rewords it, the count silently becomes zero and the verification passes.

The connection handshake follows through. The offer to import stops once the queue has anything in it — but if imported work is still missing globs, or a phase still claims SHIPPED over open tasks, the MCP instructions say so and tell the agent to run ddflow_import_verify before handing any of it out. That check is computed from the already-folded queue, so it costs nothing; the source re-scan (~0.65 s) stays out of every session start and happens only when someone asks for it.

Re-running is a first-class path. /import-existing-project opens by checking what is already imported and switches to finishing and refreshing rather than repeating — fix the globs it names, ask the operator about the SHIPPED drift, re-run ddflow import for sections added since.

What it reports rather than fixes

Three kinds of drift it can see and must not resolve on its own, because either answer could be the wrong one:

• A phase heading that says SHIPPED over unticked checkboxes (32 of them in the repository this was measured against). One-sided risk: if the heading is right, the queue is about to hand out work that is already done.

• A dependency on an id nothing produced. Kept and treated as unmet — deliberately, so a typo surfaces as blocked work rather than as work that starts early — but named, because "never offered" otherwise looks exactly like "nobody has got to it yet".

• A file that matched a source pattern and yielded nothing, which usually means an unusual format rather than an empty file.

The model: phases, tasks, dependencies, globs

Plan ──► Phase ──► Task

A phase is a unit of review: its own research, its own whole-suite test pass, its own live smoke run, merged as one coherent feature. A task is a unit of execution: one agent, one worktree, one pipeline, one merge. Both carry needs (dependencies, which may cross phases) and globs (the files they will write).

ddflow phase add P2 --title "Billing" --needs P1
ddflow task add P2.T1 --phase P2 --title "invoice model"  --globs "src/billing/invoice.py"
ddflow task add P2.T2 --phase P2 --title "tax rules"      --globs "src/billing/tax.py"
ddflow task add P2.T3 --phase P2 --title "checkout wiring" --needs "P2.T1,P2.T2" \
                                                            --globs "src/checkout/*"

Declare globs. They are what lets two agents work at once safely. A task with no declared globs is a task the conflict detector cannot protect.

Dependencies are inherited. A phase is never claimed — only its tasks are — so P2 needs P1 has to govern everything inside P2, or it governs nothing that anyone picks up. The readiness rule therefore consults an item's ancestors as well as itself:

$ ddflow next
Ready (1 ready, 0 running, 1 blocked):
  P1.T1  money
  (blocked) P2.T1: deps — phase P1 has 3 open task(s) (inherited from P2)

The refusal names where the dependency came from, because an operator told only "P2.T1 needs P1" goes looking for a declaration that is not written there. The one dependency not inherited is one pointing into your own subtree or at one of your own ancestors: an umbrella that declares a dependency on its own child would otherwise make the child -- and that child's sub-tasks -- wait for something that closes only after they do, turning a plan typo into a permanent hang.

ddflow claim asks the same predicate ddflow next does. They used to disagree — next withheld a task on its dependencies and claim handed out a worktree for it a second later — so an agent picking work by id rather than by asking bypassed the dependency graph entirely.

There is deliberately no third level of kind: a sub-task is a task whose parent is a task, so depth is unlimited while the rules stay one set.

Model-tier hint (advisory)

Tag a task tier:fast, tier:balanced or tier:deep (ddflow task add ... --tags tier:fast) to say what kind of model suits it: a cheap one for mechanical bulk work, a balanced one for implementation, a top-tier one for architecture trade-offs. ddflow next (CLI text, --json as a tier field, and the bounded ddflow_next body) and ddflow brief (the item's header and the ready list) show it; an untagged item shows nothing extra. A harness that dispatches a subagent may map it to its model choice (see the driver, docs/ddflow/drivers/implement-phase.md). It is advice only: it never affects scheduling, gates or reviewer independence, and an unknown value such as tier:foo is ignored and reported by ddflow doctor as a note, not an error.

Work that changes shape while you do it

Tasks can be added at any time, including while their parent is being worked — mid-task discovery is the normal case, not an exception, and a queue that cannot absorb it pushes the work into someone's head.

Sub-tasks are just tasks whose parent is a task. Not a separate concept with its own rules: a sub-task declares its own globs, carries its own dependencies, is claimed by its own agent, and runs in parallel with its siblings when nothing links them — exactly like any other task.

ddflow task add P1.T1a --parent P1.T1 --globs "src/parse.py"
ddflow split P1.T1 --into "P1.T1a=parse input" --into "P1.T1b=write records"

split works in place: the original keeps its id, its lease history and everything recorded against it, and becomes an umbrella that completes when its children do. Closing it and opening two new ones instead would lose the thread between what was planned and what happened — which is exactly what ddflow replay needs.

An umbrella is never offered as ready (its children are), and cannot complete while any descendant at any depth is unfinished. An abandoned child counts as settled, so a sub-task you decide against does not hold its parent open forever.

Becoming an umbrella releases the lease, however you get there — by split, or by adding the first sub-task to a task you are already working. An umbrella holding a live claim on globs that overlap every child's means a second agent cannot take one of those children, and crash recovery points at a worktree where nothing further will happen. split already did this; task add --parent did not, which is the shape of bug worth naming: one transition, two ways in, guarded on one.

Architectural decisions

The code shows what was built and never why, nor what was rejected on the way. So decisions are recorded as events, and reach the person writing the code:

ddflow decision add --title "Storage is SQLite with WAL" \
  --decision "One file, WAL mode, BEGIN IMMEDIATE for writes." \
  --context "Three call sites were each opening their own connection." \
  --alternatives "Postgres — rejected: no server allowed in this deployment." \
  --globs "src/storage/*" --by operator

--globs is what makes a decision consulted rather than merely filed. ddflow brief and ddflow decision applicable <item> surface the decisions governing an item's declared files automatically — the agent does not have to suspect they exist.

Decisions are never edited or deleted. A reversal is a new decision naming the old one (--supersedes), so the history of how the architecture got here survives, and a superseded decision is shown with a pointer to its replacement rather than silently withheld.

Recall — "have we been here before?"

ddflow recall "how should durations be represented"

One search across everything the project remembers: architectural decisions, lessons, operational memories, research verdicts, past bugs, similar tasks, and the operator's own earlier prompts. Results are labelled by kind, because a binding decision, a transferable lesson and a prompt from three weeks ago should change what you do in different ways.

It exists so the operator does not have to say the same thing twice and the agent does not have to learn the same thing twice. Both failures are invisible in the moment and obvious in the log.

Which existing record is this new one like? The similarity engine answers that without an LLM: a code-aware tokenizer (identifiers split on case and underscores, a light stemmer), TF-IDF weights projected into index.db by ddflow rebuild, and an exact cosine over an inverted index — a record missing from the hits shares no term with the query, so nothing depends on how SQLite was built. Candidates cross kinds, so a new bug is shown the open task that fixes it. Its policy is the [dedupe] section (decision D-no-duplicates): on_match (ask default, warn, off), show_floor (0.35) and ask_threshold (0.55) on the cosine, max_candidates (3), min_words (8) and kinds. The thresholds come from a labelled set of 84 duplicate / related / hard-negative pairs built from real logs (tests/fixtures/dedupe/), which the engine must keep meeting; no score separates a duplicate from a different bug in the same function, which is why the default asks rather than decides. (It was warn for a short while, because no surface could answer an ask; the CLI flags, terminal prompt and MCP relation now can.)

The Embedder: semantic ranking, optional

Every semantic feature asks ONE interface for vectors (ddflow/services/embed.py); it has no second backend, and nothing is semantic until you configure it. [rag].command is a companion: a command (a shell line, run like every operator command, in the repository root) that reads {"texts": ["...", ...]} as JSON on stdin and prints {"model": "<model id>", "vectors": [[...], ...]} on stdout, one vector per text, in order, all the same length, every number finite. The model id is how stored vectors are told apart from another model's. [rag].model_dir instead names a local model2vec directory used in-process when the ddflow[rag] extra is installed; ddflow never downloads a model, and the command wins when both are set. [rag].timeout_s (default 60, 1 to 600) bounds one call. It runs a program, so set the command with ddflow config --set rag.command '...' --local.

An embedder that is not configured, a missing extra, a path that is not a directory, a timeout, a non-zero exit or output that breaks the contract is unavailable with the reason, never a partial answer. Where a feature falls back, it keeps the BM25 order and its answer says "semantic ranking unavailable (...); BM25 order kept". ddflow doctor always carries one embedder: note saying what is configured, or that nothing is (it does not run the companion).

One path for guidance: rules and decisions reach you ranked, budgeted and fenced

The project's rules (the definitions in the log) and decisions reach an agent through four doors -- ddflow brief, ddflow claim, ddflow gate status and the review gates -- and all four ask services/guidance/inject.py. It selects the live guidance that governs the item (its files, its tags, the gate), ranks it, packs it, and fences every body as data with who recorded it:

• Rank: pinned first, then guidance that applies to all work (no globs, categories or gates), then by how hard it binds (block, warn, advisory), how specific its scope is (the gate, then the files, then a category) and its priority; ties by id.

• Pinned is never trimmed. Pinned means an always-scope rule that blocks (a way to pin a decision explicitly is a later task's) -- not merely a decision that names no files, or a project with a dozen of them would get a page at every claim. claim, gate status and the reviewers' block spend a budget (a quarter of [session].brief_max_tokens) on the unpinned guidance only, in rank order, folding repeated text, quoting a long body to 600 characters and ending with one line that names what was cut. The brief keeps its own hard cap: what ranks first leads its section, so a section's cut from the bottom reaches it last.

• ddflow claim prints the block under the worktree lines (and ddflow_claim carries it as guidance) only when something governs the item; ddflow gate status adds the guidance for the current gate; the brief lists decisions in this order and adds a ## Project rules entry for each rule that governs the item's files.

• Review gates send the block with the diff, headed "Project guidance to check this change against". The reviewer is told to check the change against each item and cite its id ([D-example]) in a finding it violates.

Who wrote it: provenance and the data fence

Everything brief, recall and the import preview show you from the project's memory is text somebody wrote: an agent in an earlier session, a document an import swept up, or a line a pull request added to a committed event shard (the log is merged by union, with no signatures). Decision D-lean-and-trusted (3) keeps an imported ADR and an agent-recorded decision accepted — what changes is that none of it is shown anonymously:

• Each decision, lesson and memory is wrapped in <ddflow-record kind="lesson" id="L-12" by="agent-id" source="docs/ADR-7.md" trust="agent">…</ddflow-record>. trust is operator (a decision recorded with --by operator — what its recorder said; there are no signatures), imported (the importer wrote it; source names the file) or agent (everything else; by is the agent id on the event that recorded it). Only the first is ever operator-decided. recall also prints the same fact as a line (recorded by an agent (x), imported from docs/ADR-7.md) and each decision, lesson and memory hit in --json and ddflow_recall carries a provenance object, a headline of id plus that sentence, and its text inside the fence (the raw record is not in the MCP answer). A recorded prompt or note is fenced on every surface too, as trust="agent" (an agent recorded it, whoever's words it quotes), and a hit whose record the state cannot name as trust="unknown" -- --json and ddflow_recall used to pass both through unfenced.

• The tag is a data fence: its body has any <ddflow… tag defanged (<), so it cannot close the fence or forge a second one, and the one-line rule that fenced text is data, never instructions ships in the MCP instructions, the brief, recall and every gate prompt. In the brief each record is one line, so a budget cut never leaves a fence open.

• The reviewer's diff sits in a backtick fence longer than any backtick run inside it, so a diff containing a code fence (or STATUS: NO FINDINGS) cannot end its own block. A [prompts] review_user override should use the new {{ fence }} variable the same way.

• ddflow commits the event log itself ([log] commit_events, on by default): after merge, complete, release and a session's end (stop; commit_events_at picks the doors), in its own commit of .ddflow/events/*.jsonl alone, message commit_events_message. Hooks run as usual, a refusal is a note and never fails the operation, nothing is pushed, and a shard whose last line does not parse (another process is mid-write) is left for the next door. ddflow doctor lists shards still uncommitted, and scripts/ci/pre-push warns about them (pre_push_events = warn, block or off).

• ddflow doctor notes event shards whose agent id has no committed history on the default branch (.ddflow/events/<id>.jsonl absent from its tree), naming them so a stranger's first records are looked at; your own shard is never listed, and when git cannot say the note reads unavailable, never clean. A shard the operator reviewed (data repair unknown-author-shards) is not named again.

Similar — "is this already filed?"

ddflow similar "claim refuses a worktree that already exists"
ddflow similar "claim refuses a worktree that already exists" --kind task --json

Before you file a bug, task or lesson, ask what the log already holds that reads like it. ddflow similar (MCP: ddflow_similar) is read-only and takes the text you would file, and optionally --kind (a comma-separated subset of [dedupe].kinds; anything else is refused, exit 1). Each candidate shows its id, kind, title, state (open, claimed by whom, done, fixed, invalid...), score (0-1, TF-IDF cosine), the words it shares with your text, and flags: identical (the same text up to case and whitespace) and named (your text names its id). Candidates cross kinds, so a bug sees the open task that fixes it and a task the bug it would fix, and closed records are included: a new bug that repeats a fixed one is exactly the case worth catching.

Exit codes: 0 with candidates, 2 with none. --json (and the MCP tool) return the same list of candidates, and an empty list when there are none. It uses the [dedupe] knobs show_floor (lowest score listed, default 0.35), max_candidates (default 3; records whose id the text names are listed as well) and kinds; it runs even when [dedupe].on_match is off, since that setting governs what an add does. A score is a prompt to look, not a verdict: two bugs in one function score high and are different.

Dupes and link — settling a pair already in the log

similar weighs one text you are about to file. dupes sweeps the records the log already holds against each other:

ddflow dupes                     # every unsettled pair at the show floor
ddflow dupes --open-only         # only pairs where both records are still live
ddflow dupes --kind lesson --json
ddflow link B198 --duplicate-of B-dupes-sweep   # a duplicate
ddflow link B2 --distinct B1      # "I looked: different" -- never asked again

ddflow dupes (MCP: ddflow_dupes) lists each pair with both ids and kinds, their titles and the score, and skips pairs already answered — a recorded link (extends / duplicate_of / related) or a distinct dismissal, in either direction, or one lesson superseding the other. So a pair marked distinct never returns. It weighs records the ordinary index drops, too: a removed item or a closed bug (B203 was removed as a duplicate of B-semantic-recall, and the sweep still finds that pair). Exit codes: 0 with pairs, 2 with none; --floor overrides the floor (default [dedupe].show_floor).

ddflow link (MCP: ddflow_link) settles one: --duplicate-of / --extends link the subject to the target, --related links them, --distinct dismisses the pair for good. Two lessons linked --duplicate-of / --extends are merged: the target keeps both texts' tags and seen_in, and the duplicate is superseded by it — the same mechanism as lesson add --supersedes, so a lesson is retired one way, not two. Nothing else is closed here: a duplicate bug is closed only once its original is fixed, with that original's regression test (ddflow bug fixed), not by linking.

ddflow doctor notes how many pairs are unsettled, and the dedupe_sweep workflow command (MCP prompt dedupe_sweep) runs ddflow dupes --open-only as the periodic pass the [cadence] dedupe_sweep_every_tasks knob schedules.

The check every add runs

Every add — task, phase, bug, lesson, decision, research and memory; never a session prompt or note — runs the same check against the log before it writes, with the [dedupe] knobs above. What it does depends on how close the match is:

• A match — a candidate scoring at least ask_threshold (0.55) when the new text has at least min_words (8) content words, or text identical to an existing record — is refused (exit 3, refused: possible duplicate). Nothing is written. The refusal carries candidates (id, kind, title, state, score, the words shared) and options (new, then extends / duplicate_of / related for each candidate), and its reason lists them in prose; records the text merely names by id are listed apart, as information, and never refuse by themselves. Over MCP the refusal leads the reply as {"refusal": {...}}. A bug filed against the task that fixes it (--item T) already answers that candidate and is not asked about it.

• An answer lifts the refusal. new: it is a different record, filed as asked. extends X / duplicate_of X: it is the same thing. While X is open and unclaimed (an unfixed bug, a lesson or decision not superseded, a live memory -- and, for duplicate_of or the explicit merge_into X / --merge-into X, an open task, phase or research note) the text is appended to X as a record.extended — verbatim, with who, when and the score — and no new id is made; a bug's own summary is never replaced. extends X on a task, phase or research note files a new record linked to X instead: appending to something another agent owns, and writing the new task's globs onto it, is never implied (merge_into on a claimed or closed X is refused). If X is claimed, running, done, fixed or superseded, a new record is filed linked to X (extends / duplicate_of on its add event), so nothing is lost, and the result names X and its state so the holder can be told. related X files a new record linked both ways (a link.recorded on X). The API takes the answer as one answer argument (api.DedupeAnswer("extends", "B5d98a4da0a"); DedupeAnswer.parse("related B1")) carried on task_add, phase_add, bug_found, memory_add and the lesson, decision and research drafts. The CLI and MCP surfaces are below.

• Answering a refusal, on every surface.

  • CLI flags on every add command (task add, phase add, bug found, lesson add, decision add, research add, memory add): --new, --extends ID, --duplicate-of ID, --related ID, --merge-into ID and --check; at most one of the six (argparse refuses two, and over MCP relation with check_only is a failure). --check is a dry run: it writes nothing, prints the candidates and whether the add would be refused, and exits 0 with candidates or 2 with none (--json prints candidates, options, would_ask, and for an exact copy would_extend -- the open record it would be added to -- or would_link -- the claimed or closed record it would be filed beside). An id that already exists keeps its own rule and is not checked (--check says so, exit 2 -- except a task or phase, whose existing id is the queue's ordinary already exists refusal, exit 3, exactly as the add itself would answer); an index that cannot be read is exit 1, never "none". ddflow similar "<text>" asks the same question before you have an id or a command to run.

  • On a terminal (stdin and stdout are both terminals, no --json, no answer flag) a refused add asks instead of failing: it lists the candidates numbered and prompts [n]ew / [e]xtends # / [d]uplicate of # / [r]elated # / [a]bort. e 1 and r T-old both work (the number or the id; the number alone is enough when there is one candidate). Anything else asks again; abort, or end of input, files nothing and exits 3.

  • Without a terminal (a script, an agent's shell, a pipe) it exits 3 with refused: possible duplicate, the candidates, and the same command again with each answer appended -- --new, --extends TOP, --duplicate-of TOP, --related TOP -- ready to paste. --json prints the same refusal as candidates plus options (the reason is still on stderr). Nothing is written. An add that goes ahead anyway -- warn, or a match below the asking threshold -- prints It reads like: and the candidates on stderr before its usual line.

  • MCP: every add tool (ddflow_task_add, ddflow_phase_add, ddflow_bug_found, ddflow_lesson_add, ddflow_decision_add, ddflow_research_add, ddflow_memory_add) takes relation -- new, extends:ID, duplicate_of:ID, related:ID or merge_into:ID -- and check_only (the dry run). The refusal leads the reply as {"refusal": {...}} with candidates and options beside it; answer it by calling the tool again with relation. An answer that lands on an existing record returns extended (the record that received the text), extended_kind and relation in place of a new id.

  • Agents: run ddflow similar before filing; prefer extending an open, unclaimed record; a claimed or closed one gets a new record linked to it (extends does that on its own).

• Identical text (up to case and whitespace) as an existing record of the same kind is recorded as a duplicate of it without asking, under the same open-or-linked rule.

• Below the threshold, at or above show_floor (0.35): the add goes through and its result lists the candidates.

• Every answer is recorded on the add event (dedupe: the answer, the score, the candidates shown), new included, and an automatic merge is marked auto.

• A lesson captured by bug fixed --lesson-title runs the same check. Nobody can be asked there (the bug is already closed), so the answer is automatic and marked auto: identical text goes onto the open lesson it copies (no new id), and a lesson that merely reads like one is filed linked to it (related), so the pair meets in the next sweep rather than drifting apart. The result says which (lesson_captured names the lesson that holds the text; lesson_capture has captured, extended, related, candidates, and dedupe_unavailable when the check could not run -- the lesson is then filed unchecked and the CLI says so), on the CLI line, in --json and over MCP.

• Rules (rule add, ddflow_rule_add) are files, not log records, and run their own check against the other rules: content against content, and title against title for rules with no content (two title-only rules no longer read as copies of each other). --related ID files the rule and reports related (a rule carries no link); --check compares the title too. A rule is ALSO checked against every other kind -- decisions, lessons, research, tasks, bugs, memories -- by the same check every add runs (rule is in [dedupe].kinds; remove it to stop): a rule restating a decision is refused with the decision as the candidate. --new files it; --related D files it and reports related; --extends D / --duplicate-of D put the rule's text on that record while it is open (record.extended, no rule filed) and otherwise file the rule naming it; --check lists both. rule edit (ddflow_rule_edit) runs it on a new title or content and takes --new / --related ID (new / related over MCP; ID may be another rule); an edit cannot be folded into another record. When the check cannot run (an index that will not open), the rule is still filed and the result says so: dedupe_unavailable in --json and over MCP, filed UNCHECKED on the CLI line; records it merely resembles are listed after the line (It reads like: ...).

• Where the check does not run, a test says why. tests/test_coherence_coverage.py classifies every event kind, every function that appends an add kind, and every CLI verb and MCP tool that records text as checked or exempt with a reason (a split's parts, a bug's own fix task, the importers' own check, the operator's verbatim words); a new add path fails it until somebody decides.

Seeing what was added. ddflow show X (an item or a bug id; --json carries the same data as additions, links and linked_from) lists the additions on X verbatim with who, when and the score; the links X makes (links); the records that link to X (linked_from) — found by scanning every record's links for X, because a new record filed extends / duplicate_of X holds the link itself and only related also writes a back-link, which is listed once; and, for a bug, the task that fixes it. The holder of a claimed item is told what arrived: ddflow brief for it leads with N new reports on your item since you claimed (additions made, and records linked to it, since its lease was acquired), quoting each (the first five, clipped; show has the rest) inside the brief's token budget, and ddflow heartbeat and ddflow gate status carry the count in one line. MCP ddflow_show and ddflow_brief return the same data.

[dedupe].on_match sets the policy: ask (default) as above, warn never refuses or merges — it lists the candidates and records the add as new — and off skips the check entirely. Adding an id that already exists keeps the refusal or merge it always had, except for research: research add --id R1 when R1 is already filed is refused (exit 3, naming the fields that differ) unless it is the same record again, which writes nothing -- add to R1 with --extends R1 (no --id), or file the finding under another id. An id ddflow generates (no --id) is time-salted, so the same text filed twice gets two different ids; only the duplicate check, never the id, catches a re-filing: an exact copy of an open record is merged into it as an extension, of a closed one filed as a new record linked to it, both without asking under the default ask (warn and off leave two unlinked records). The check reads the log before the add writes, so it is advisory across agents: two adds of the same text racing in different clones can both pass, and a later similar will show the pair. The check is also a function, api.dedupe_check_add, taking a config that may have on_match replaced (dataclasses.replace on cfg.dedupe), so a bulk caller need not go through the add paths. The importer applies the same engine and [dedupe] thresholds to a batch with its own reconciliation (see the import section), not through this function.

Operational memory

ddflow memory add "8x H200 on this box; check nvidia-smi before a GPU test" --tags gpu
ddflow memory list                       # newest first; --query to rank, --all for forgotten
ddflow memory forget M-0003 --reason "the box was upgraded"

One fact about this machine, repository or working state — not a rule (lesson), not what happened (session note), not how the software is built (decision). Capped at [memory] max_chars (280) and refused, not truncated, when longer. The newest [memory] brief_items appear in every ddflow brief, right after the binding decisions, and recall searches them — the job an OptMem store beside the repository used to do, now in the log, so every worktree sees a memory the moment it is written. A memory that stopped being true is forgotten with a reason, never deleted: "we thought X until Y" is what stops the next agent re-learning X. An OptMem LOG.txt imports as memories dated when they became true; one over max_chars is left out and counted, never cut. The log is committed: never put a secret in one.

Resources and long-running jobs

Globs keep two agents out of one file. Work that runs on something — GPUs, a model server, a shared fleet — declares that too, and the same refusal applies:

ddflow update TRAIN.3 --resources gpu:6            # or **Resources:** gpu:6 in the plan
ddflow claim TRAIN.3                                 # refused (exit 3) if it does not fit

[schedule] resources = ["gpu=8", "vllm-fleet=1"] sets capacities; a resource named nowhere is exclusive. Every live claim counts, the claimant's own included — one agent starting two 8-GPU runs overcommits the box just the same — and next withholds what does not fit, saying who holds what.

The run itself is a job:

ddflow job run TRAIN.3 "uv run main.py train -c cfg.toml"   # detached; survives you
ddflow job list        # running | exited N (from its log) | gone (killed) | elsewhere
ddflow job end J3f2 --note "loss 0.12, ckpt in out/"         # refused while it runs

A launched job runs in the item's worktree, in a session of its own (it outlives the agent, the MCP server and a restarted remote-control service), and appends its exit code to its log so a run nobody watched still says how it ended. Liveness is computed, not stored: a zombie is not alive, and a reused pid is caught by the process start time. ddflow job add --pid registers a process started some other way. Every brief lists jobs not yet recorded as ended — "WAIT, do not start it again" for a running one. Running jobs and the jobs of the item a brief is about are listed in full; other items' jobs on another host (they may be running) up to the newest five; their exited or killed jobs collapse to one count line (a brief about an item -- named, held or suggested -- lists none of them; one about no item at all, the newest five). So a backlog of uncollected jobs never pushes the item's own section out of the token budget. ddflow job list shows them all.

Dependencies on another repository

[schedule]
repos = ["run_nemo_run=../run_nemo_run"]

ddflow update GEN.4 --needs run_nemo_run:132.D then waits for item 132.D there to be done. ddflow external sync (and every session-start hook) reads the sibling's log — never writing it — and records what it observed in this one, only when it changed. Readiness is decided from that dated fact, so the fold stays pure and "why was this started?" is answerable later. An unobserved external dependency is unmet; one naming a repository that is not configured is a doctor problem, since it can never be met.

Status, progress, and loops

ddflow status      # what is done, in flight, ready, blocked — one answer
ddflow progress    # attempts, hours held, gate runs, commits, per item
ddflow loops       # circular references and runtime loops (exit 2 = none)

progress counts a landing once (commits holds distinct shas). ddflow show <id> takes a bug id too: its state, where it was found, the fix task(s) — a task that says "fixes bug X", or names X in a fix list ("fixes bugs A, B and X"), is a fix, a mention is not — the regression tests it was closed with, and an invalid closure a later fix superseded. ddflow --json board prints the board as JSON (a task under no phase is listed in an Unphased section of the board, and under unphased in the JSON); --agent, --repo and --json are accepted after the subcommand as well as before it.

A phase's done/total is the same number everywhere — the board, phase list (and ddflow_read kind=phase), the progress line after complete and the ROADMAP export. A phase counts every live task nested any depth below it, sub-tasks and bug-fix tasks included; an abandoned task is settled, so it is out of the total and shown as "N abandoned" where there is room. The progress line after completing a sub-task names the sub-task's phase, not its parent task.

next --phase, brief --phase and board --phase refuse an id that is not an item and next names the phases that start with it; board --phase given a task id refuses, naming that task's phase. ddflow status over MCP is bounded: the counts are exact, but each long list is cut to the 25 most recent (the first 25 in scheduler order for the others), with a truncated note naming the real lengths; the CLI, and ddflow --json status, list everything. The other large MCP reads are bounded the same way, and a cut is never silent: ddflow_next lists the ready items whole and the first 10 blocked ones, with a truncated field giving the exact blocked total and a count per reason; ddflow_show leaves out each gate record's tree ids and merge bookkeeping and cuts any gate or triage string past 160 characters, with truncated naming what went; ddflow_progress (most effort first) and ddflow_decision_list (newest; no context or alternatives, decision text clipped to 400 characters, ddflow_decision_show has one whole) return 25 rows, limit raises it and limit=0 is all, and a second content block states the cut; ddflow_recall drops each hit's raw record; MCP bodies are compact JSON. --json on the CLI is the whole, indented body in every case. recall enforces max_chars (default 4000) once, in the API, for the CLI, --json and MCP alike: hits are taken one per source in turn so no source is crowded out, counted by the headline and fenced body a reader is shown, the first hit is always kept whole, a hit that does not fit is skipped, and a hit whose text repeats one already kept (same words up to case and whitespace) is folded into it; what the budget or the fold left out is stated (the MCP result's second block, the CLI's … truncated at N chars line). On this repository that took next from 38 KB to 3.9 KB, show of a finished task from 8.9 KB to 4.1 KB, recall from 51 KB to 4.1 KB, decision_list from 54 KB to 18 KB and progress from 158 KB to 8.4 KB. doctor and status also say when the log holds events from a newer ddflow than this checkout runs — they were skipped, so the numbers are computed without them — and the remedy is to merge main or run the newer ddflow.

brief says whose work it is. Run under an identity, its ## Current is the agent's own most recent lease (several are listed, with --item <id> for another); an agent holding nothing sees ## Suggested next — the queue's top ready item, to claim first — because calling it Current told an agent it was working on something it never claimed. complete likewise defaults the author model from the agent's open session start.

Dependency cycles are the easy case. The expensive ones are runtime loops, where the graph is perfectly acyclic and the work still never finishes:

Every threshold is a [loops] knob, and on_detect = "block" makes ddflow claim refuse an item that is already looping — a warning is read by a human later, a refused claim is read by the agent now. repeated_failure also shows as a line in the item's ddflow brief, and under block refuses a gate run of that gate until the work has changed (the refusal names the evidence; a different tree lifts it). max_repeated_failures = 0 turns it off.

The task pipeline

Ten gates, in order, configurable per project:

Four things are enforced rather than requested:

Silence is not a pass. Every gate in the pipeline must carry some outcome before an item completes — passed, failed, unavailable, partial, or an explicit ddflow gate skip <id> <gate> --reason "...". Without this, gates.required held only implement, unit_tests and merge, so six of the ten steps could be omitted with no trace at all. gates.require_outcome = false makes the pipeline advisory again; gates.enforce_order ("warn" by default, or "block") reports a gate recorded before an earlier one has run, because a rubber-duck review recorded before implement reviewed an empty diff.

UNAVAILABLE is never a pass. A reviewer whose endpoint was down approved nothing; a linter that is not installed found nothing. Each gets its own outcome and shows as a coverage gap. (The inverse matters too: this codebase's first version classified a missing binary — shell exit 127 — as failed, so an uninstalled linter looked like a linter reporting problems. Fixed, with a mutation-verified regression test.)

Evidence or it did not happen. Gates in gates.evidence_required reject a bare pass; they want the command, its exit code and its output digest.

And evidence says WHICH tree and HOW MUCH. Every gate that produces an OUTCOME — command gates, and agent gates recorded with gate record — carries a tree_sha, ONE identity of the tree it ran against: <head>+clean for a tree that holds exactly HEAD's files, else <head>+st:<content id> -- the commit plus the exact content of every file, tracked changes and untracked ones (a new module is untracked until its first commit, which is the ordinary state of agent work). Older evidence recorded a digest of the status and diff text beside a separate source_tree field; both spellings are still read, so a pass recorded by an earlier ddflow keeps its meaning. ddflow's own .ddflow/ is excluded, or recording a gate's outcome would invalidate the gate that just recorded it. If the tree moves afterwards, complete warns that the pass describes source nobody is shipping — a warning, not a block, because refusing on a comment-sized change is how a check gets switched off. The comparison is of content trees, not commit ids, so an amended message is not a change; once the item has landed it is made against the commit that landed (a merge commit's second parent, which is what the gates ran on — never the primary checkout, whose files are everyone's), and the note names what differs. Evidence that cannot be compared says so rather than passing silently.

Beside it, diff_stat records files, insertions, deletions and untracked count — including the lines in untracked files, because a new module is untracked until its first commit and a task that is entirely new files would otherwise report zero insertions. tree_sha answers which tree and is opaque; this answers how big, and that is what makes a pass auditable later — a review gate that passed over 4,000 changed lines in two minutes is a different claim from one that passed over 12.

Neither is recorded for a skip (nothing was reviewed, so a magnitude would imply an inspection that did not happen) nor for an unavailable gate that never ran.

Hashing untracked content is capped by MAX_UNTRACKED_HASHED (512). Above it the identity falls back to the older digest, which covers file names and says so inside it, because a check that quietly stopped covering content would go silent for exactly the repositories that need it most.

Reviewer independence is checked, and an unidentified reviewer establishes nothing. Same-family reviewers share the author's blind spots, so their agreement measures shared priors rather than correctness. complete refuses unless one reviewer came from a different pretraining family — and a reviewer whose model is not in [agent].families counts as unknown, never as different. Each families key is a case-blind substring of the model name, as each routers key is; an empty key matches nothing. (It used to count as different: gate record defaults the reviewer to the agent id, so a standards gate recorded with no --model arrived as family "host-12345", compared unequal to "anthropic", and satisfied the independence requirement on its own.)

--model on gate record names the reviewer; the author's model is complete --model (or the one declared at session start). Recording the author's own model on a reviewer gate used to overwrite the reviewer's, and complete then judged independence against the author's family. Now a --model in the author's family on rubber_duck/critic is refused (exit 3, naming --reviewer-model); --reviewer-model <m> (MCP reviewer_model) is the explicit spelling for a reviewer that really is of that family. The author is the model the agent declared at session start --model; with none declared there is nothing to compare and the record stands (complete says which it used: the --model flag or the agent's session — author_model/model_source in the result and on the item.completed event). standards is covered too, since it counts toward independence.

Which gates are reviewer gates comes from their definitions: rubber_duck, critic and standards, plus every gate declaring reviewer = "different_family" — the built-in verify, and a project's own ([gate.security_review] with that line in gates.toml). A review recorded through any of them counts toward independence and has its --model checked as above; a same_family_ok gate does neither, built in or not (critic set to same_family_ok stops counting). (A custom reviewer gate used to be ignored, so an item reviewed only through one was refused at complete.)

--reviewed-sha ties a review to the commit that merges. roborev review HEAD run from an item's worktree has enqueued the primary checkout's HEAD, so a standards gate was recorded against the wrong commit. Run roborev review <sha> with your branch head and record gate record <id> standards --reviewed-sha <sha>: the full sha is stored in the evidence, an older commit of the item's branch is accepted with a warning, and any other commit (main, another branch, an unknown sha, or a symbolic ref such as HEAD, which means a different commit in each checkout) is refused with exit 3. An item with no branch to compare against records the sha with a note.

A reviewer tool must leave git as it found it. ddflow gate run on a reviewer gate (standards, or any different_family gate with a command) and a kind = "command" reviewer compare HEAD, the index, the working tree and the stash list before and after the tool runs. The stash list is shared by every worktree, so a tool that ran git stash apply or pop (kilo under roborev did) rewrote someone's tree. If anything moved, the gate is unavailable (never passed), the evidence carries git_state_changed with both digests, and nothing is restored: check the tree and git stash list yourself. A roborev review <sha> you run by hand is outside this check; run it against a clean throwaway worktree.

The reviewer recorded is the one roborev ran. roborev runs its backup_agent when the requested agent fails, so a job enqueued as agent: kilo can be reviewed by claude-code (roborev show N: "by claude-code"). With --reviewed-sha on a reviewer gate, ddflow asks roborev list --json for the newest finished review of that commit (or of a range ending at it) and records the model roborev ran -- else its agent -- as the reviewer, with the job under evidence.roborev, whatever --model said; a same-family reviewer is then judged as one at complete, as --reviewer-model would be. Without roborev on PATH, or with no finished review of the sha, the typed model stands, with a note and evidence.roborev.verified = false in the record.

ddflow review of an item reviews the branch's commits plus tracked edits in its worktree. Untracked files are not sent to the reviewer; they are listed in the output and in the evidence's diff_source as "untracked, not reviewed" — commit a file to have it reviewed.

A router author is a set of families. Copilot CLI's HydraFusion is chosen like a model but routes each task across models from several providers, so [agent].routers maps its name to the families it draws on, and a reviewer counts as independent only when its family is outside the whole set. HydraFusion ships with an empty set — GitHub publishes no fixed roster — so complete --model hydrafusion refuses, naming the knob, until you list the families your plan routes to:

[agent]
routers = { hydrafusion = ["anthropic", "openai", "google"] }

$ ddflow complete P1.T1 --model claude-opus-5
cannot complete P1.T1 — 1 unmet condition(s):
  - reviewer independence not satisfied: every reviewer (rubber_duck) was family
    'anthropic', the same as the author. Same-family agreement is not independent evidence.

Every unmet condition is listed at once — a refusal that reveals one problem at a time trains an agent to reach for --force.

Proving a gate can fail at all

$ ddflow gate verify T1 unit_tests
  OK   src/calc.py: detected

unit_tests CAN fail: every registered mutation was caught.

A gate that cannot go red is worse than no gate — it reports success on every change, and everyone downstream reads that as evidence. gate verify breaks what the gate guards, using the mutations registered beside it, and requires the gate to notice:

[gate.unit_tests]
command = "python -m pytest -q"
cwd = "repo"
mutations = [ { file = "src/calc.py", old = "return a + b", new = "return a - b" } ]

Four things make it honest rather than ceremonial, and the last one is this feature's own bug, found by a cross-family review of it:

• A mutation that did not apply is a FAILURE, not a skip. If old is absent — or present twice, so the edit is ambiguous — the check fails. Skipping turns "the mutation never happened" into a green run, which reads as the opposite of the truth.

• The source is restored whatever happens, including on exception, or a failed verification leaves the tree broken and the next gate reports the verifier's fault.

• A gate with no registered mutations is reported as unproven. Declaring a check nobody has shown can fail is what this exists to catch. An agent gate says plainly that it has no command to mutate and rests on its evidence contract instead.

• A green baseline is required first. A gate already red for an unrelated reason — one pre-existing failing test, a tool that stopped being installed, a flake — reports failed for every mutation, so every mutation reads as detected and the gate is certified as able to fail when nothing has shown any such thing. The check written to catch the vacuous-pass class contained it. It now runs unmutated first and refuses without a pass.

A gate that reads a COMMITTED ref, as ci does (ddflow ci run merges the branch's commits in a scratch worktree), cannot see an uncommitted edit, so verifying it that way reported PASSED on mutated source. Such a gate (reads_ref = true, the default for ci) is verified against a throwaway commit holding the mutation, run in a scratch checkout of it; the branch, the index and the working tree do not move. The mutations are read from the item's own worktree as well as the primary, so one registered on a branch can be verified before it merges.

A gate that runs only on the work it concerns

[gate.<id>] applies_when = ["api/**", "schema/*.sql"] puts the gate in effect for an item only when one of those path patterns can cover one of the item's declared globs (--globs). For any other item ddflow gate status shows it as [-] <gate> -- not applicable: ..., naming the patterns and the globs it was judged against; nothing waits on it, so it is neither silent (completion does not demand an outcome), a required gate that is missing, nor ahead of the gates after it under gates.enforce_order. A gate that already has a recorded outcome always applies, and so does every gate of an item that declares no globs: a gate is never left out for want of information. It is a gate field, not a knob -- empty by default, so a project that never sets it sees no change.

A gate listed in [gates].required is the exception: the declared globs are agent-editable (ddflow update --globs, no approval), so they never excuse it. It reads [-] not applicable only when the ACTUAL change (the item's diff against its base) touches none of the patterns, and the reason names the changed paths. With no diff yet, an unreadable one or an empty one it stays owed. Every other gate is re-judged on the diff at complete: a change that touches its paths brings it back, and complete lists each gate the diff excused.

A gate that concerns a branch (needs_branch, set on merge) has the same shape: it does not apply to a phase with no branch and no worktree -- a phase has no branch of its own -- so it shows [-] merge -- not applicable: no branch ... and completes without a hand-recorded pass. What stands in for it is a fact: complete <phase> refuses while a finished task below it still has a branch the base lacks. An ordinary claimed task still needs merge.

A pass that must name its evidence

[gate.<id>] requires_evidence = ["report", "link"] and evidence_fields = ["intent", "edge cases"] refuse a bare pass: gate record ... --outcome passed fails, listing every unmet form at once, unless the evidence carries what the gate asks for. report -- an output digest (attach the output with --output-file; an empty or whitespace-only file does not count) or a report_digest; link -- a record id (a research note) whose item is this one, so a note filed against another item does not count; fields -- every evidence_fields name filled in with real content (an empty value, n/a, none or LGTM counts as missing). Listing evidence_fields implies the fields form. Only a pass is held to it -- a skip with a reason or unavailable stays available -- and, as with applies_when, these are gate fields, not knobs: empty by default, so a project that sets none sees no change. A requirement ddflow cannot check (a link where no record can be resolved) is not met rather than assumed.

When the exit code is not the verdict

Some tools say "I could not run" or "I only did part of it" with an exit code, and some exit 0 whatever happened. A command gate can say which:

[gate.critic]
command = "uv run scripts/critic_review.py --dirty -c configs/review_critic.toml"
unavailable_exits = [2, 143]      # endpoint down / SIGTERM: UNAVAILABLE, not failed
partial_exits = [3]               # reviewed part of the diff: PARTIAL
require_output = '^STATUS:'       # exit 0 without it = the tool did not do its job
fail_output = '^\s*- \[(HIGH|MEDIUM)\]'   # exit 0 WITH findings = failed

ddflow gate run also renews the caller's lease every [lease] heartbeat_s while the command runs, so a 25-minute suite does not outlive a 30-minute lease and read as abandoned work.

It keeps the run's whole output in .ddflow/runs/<item>/<gate>-<time>-<pid>.log (the newest ten per item and gate; the directory ignores itself, so nothing is committed) and records its path beside the digest, so the digest can be checked against bytes that still exist. The evidence also carries the suite's own verdict lines (summary: pytest's === 3 failed, 112 passed ===, unittest's FAILED (failures=2)) lifted from the whole output, because a gate that reruns its failures ends on the rerun's "112 passed" and the tail alone would hide the first pass's "115 failed". gate record --output-file keeps the file's path the same way.

The phase pipeline

research → [ task, task, task … ] → unit_tests → bug_hunt → dedupe
         → live_test → corrections → docs → merge

docs runs once per phase, before it merges: every user-visible change the phase made (commands, MCP tools, flags, knobs and defaults, output, setup) is checked against the README and the docs and updated there, with the files changed as its evidence. The per-commit stale-docs check catches a renamed or removed name; this catches the new feature nobody wrote down.

live_test is the one most often skipped and the one most worth keeping: a green unit suite and a working feature are different claims. Run the real thing on a small input and paste what it printed.

Human approval: a gate the agent cannot clear

Every other gate here is satisfied by the agent — it runs a command, or it asserts it did the thinking. That is right for work whose correctness is checkable afterwards, and wrong for a plan: by the time an agent has built the wrong thing, the cost is already paid.

A gate marked human = true is where the operator says yes, build that before the compute is spent.

# .ddflow/gates.toml
[gate.plan_approved]
title  = "Operator approves the plan"
human  = true
prompt = "Show the operator what you intend to build, then ask."

$ ddflow gate run T1 plan_approved
gate 'plan_approved' is a HUMAN-APPROVAL gate. It is not something you can run or
record — it is where the operator decides whether this work should proceed.
  ddflow approve T1 plan_approved
  ddflow approve T1 plan_approved --reject --reason '...'

$ ddflow gate record T1 plan_approved --outcome passed --evidence "looks fine"
'plan_approved' is a human-approval gate: it is cleared by a person, not by an agent
recording that it happened.                                            # exit 3

$ ddflow approve T1 plan_approved --note "read the plan, ship it"
T1.plan_approved approved by delian — read the plan, ship it

A rejection is a first-class outcome, not the absence of an approval: "the operator looked and said no" and "nobody has looked yet" are different states, and an item sitting in the second forever is how a checkpoint becomes a silent stall. --reject requires --reason.

There is deliberately no MCP tool for this, and ddflow/surfaces/exemptions.py records the exemption (a field of the command) with that reason. A human checkpoint reachable from the MCP surface is not a human checkpoint — it is a second gate record with a longer name. gate skip is refused too: "the operator does not need to approve this" is not the agent's call.

What this is, precisely. An audit trail and a speed bump, not a security boundary. An agent with shell access can run ddflow approve itself, and no design here changes that — the tool does not control the machine.

The guarantee, as narrowly as it holds: no MCP tool records a human outcome, and a clearance carries the OS user and a human flag, so a forged one is visible in the log rather than indistinguishable from a real one. (gate.<id>.human is also refused by the config writer, because two MCP calls — flip the flag, then record — used to clear the gate with no shell involved. Declare human gates in .ddflow/gates.toml, which no tool writes.)

Opt-in: the shipped pipeline has no human gate, and a test keeps it that way. When the merge gate itself is declared human, ddflow merge still lands the branch but records no outcome for that gate (merge_gate_human is true in its result, and the CLI names the ddflow approve <id> merge to ask for).

A required gate recorded partial because it is baseline-red (the failing tests also fail on the base branch, measured by gate run) blocks complete, and --force is not the way past. Re-run it once the base is green, or have a PERSON run ddflow approve <id> <gate> (decision D-accept-baseline-red-partial): that accepts the one recorded outcome by digest (approval.granted, refused under an agent identity, no MCP tool). The gate stays partial in the record and in gate status; complete warns that it settled only on that acceptance, and verify notes it. A re-run or re-record needs its own acceptance; failed, unavailable and a hand-recorded or partial_exits partial cannot be accepted this way.

Parallelism and coordination

$ ddflow next --phase P1
Ready (3 ready, 0 running, 1 blocked):
  P1.T1  persistent store
      writes: shortener/store.py, tests/test_store.py
  P1.T2  base62 encoder
      writes: shortener/encode.py, tests/test_encode.py

These are independent — run them in parallel worktrees.
  (blocked) P1.T3: deps — P1.T1 is open; P1.T2 is open

ddflow claim <ID> leases the item and binds it to a worktree.

What a claim holds is what you said it holds. claim --globs is recorded on the item, so no later heartbeat — live, or reviving a lapsed lease — resets the lease to the globs the item had before; the claim prints the globs it recorded. Globs are comma-separated or a JSON array, and the flag may repeat (every value is kept; over MCP a JSON array is read whole). A glob mangled by quoting is refused with the form every surface reads. update --globs replaces the item's globs, and a claimed item's lease with them; the result names what it dropped, and show prints the lease's globs beside the item's. --resources works the same way. ddflow wait --item X --globs .. judges the claim you are about to make rather than the stored globs, so READY means that claim will not be refused. One wait blocks for at most 1800 seconds on the CLI. Over MCP it is at most 1800 seconds when the call can send progress (a _meta.progressToken and [mcp].progress_s on: each beat resets the client's idle timer), and at most 1200 when it cannot — below the 1800 s a client gives a silent tool call before aborting it (Bfc43dae5b9); a longer --timeout is shortened (and says so on stderr), and you call again to wait longer.

Files every item touches — a changelog, a research log, a regenerated config — would make every pair of items collide. Declare them, and many leases may hold them at once:

[lease]
append_only_globs = ["docs/CHANGELOG.md"]   # every item adds lines
shared_globs = ["configs/default.toml"]     # generated: regenerate after merging

A path inside either is exempt from the overlap check (claim, update, next, wait) and counts as covered at commit time for anyone holding a live lease. For each append-only glob in the committed config ddflow writes <glob> merge=union to .gitattributes — when the knob is set through ddflow config --set / ddflow_configure, and again on ddflow init / adopt after a hand edit — and prints the line; commit it with the config. A generated file never gets union (it would be interleaved): doctor notes a shared glob git has no merge driver for (a document items edit by hand in sections, like README.md, is merged right by git's default text merge: declare it with a README.md merge=text line in .gitattributes and the note stops), and reports an append-only glob git does not union-merge. Git decides which driver a file gets (check-attr), globs match as .gitattributes does, and a union line never overrides a narrower driver the project set for a file inside the glob: it is written before that line, so the narrower one keeps winning. Only the committed config writes union lines; a .ddflow/local edit syncs nothing.

A contended file is served first come, first served. An agent refused for an overlap used to have no place in line: it polled, and whoever polled first after the release took the file, so a hot file (the one every change touches) starved its longest waiter for hours. Now a live ddflow wait --item X, or a claim that was refused and is asked AGAIN, is a place in line (kept in .ddflow/local/waits/, this machine's, never committed). While the oldest waiter could claim its item right now, a younger or unqueued claim of an overlapping file is refused with exit 3: TC is reserved for agent-b (waiting since 14:02:11Z for TB, which needs the same files); their place is held for 300s after they could claim. next lists such an item as blocked instead of offering it, and wait --item stays blocked on it. The waiter's own claim passes, and it spends the place. This is a best-effort queue, not a lock: the registry is checked when a claim is asked, not inside the lease transaction, so two claims landing in the very same instant at a release can still race.

The rule cannot hold anyone up for long: a waiter whose process died or whose wait deadline passed is dead, and blocks nobody; a waiter that was woken and does not come back loses its place [lease].waiter_reservation_s seconds later (default 300; 0 turns the queue off); a waiter still behind another holder, or waiting for files disjoint from yours, reserves nothing against you; and the order is strict (older first), so two waiters cannot reserve against each other. A refused claim keeps its place as long as it is asked again within the window; a SINGLE refusal reserves nothing (an agent that takes the alternative the refusal offers never comes back, and must not hide the item from next or block its owner). The holder is told who it holds up: heartbeat and release name the waiters, and brief adds a "Waiting on you" section. Releasing globs while only gates are pending is deliberately not offered: until the branch is merged another agent editing the same files would only meet the conflict at merge. The honest remedy is a short claim: claim when you are ready to edit, run the gates promptly, and do not sit on a lease waiting for a slow review (record partial and merge instead).

When two clones disagree. A merge can bring in a rival definition of one id (two clones added it) or two claims on one item whose windows overlapped. ddflow records the contest instead of letting the later event silently win: show prints a CONTESTED block with every rival whole, next withholds the item, and doctor names it. A person settles it with one event:

$ ddflow resolve P1.T1 --keep alpha            # a claim: by its holder, or its event id
$ ddflow resolve P1.T1 --keep 3fa9c2 --refile-as P1.T1b   # a definition; the loser is re-added

Keeping a claim releases every claim that overlapped it and starts the kept claim's lease window now: a claim that had already lapsed is not stretched back over the gap, so a claim another agent legitimately made in the meantime is not retroactively contested. A holder's own lapsed claim, every displaced claim (no cap), and a contestant's late renewal are all kept in the record. An unresolved contest always displays one of its claims (the latest, which may itself have lapsed -- show gives its time left): when a holder who took over one contestant's lapsed claim releases, that contestant is displayed again, and show (on its lease line) and status (on whichever list holds it -- usually Blocked, as a contested item) mark it previously taken over, first by <holder>; status --json lists it under taken_over (id, holder, taken_over_by, the first taker -- the record keeps one entry per claim, so a later takeover of the same claim is not named). Its holder may have moved on, so settle it with resolve. --keep may also name the current holder when it met no part of the contest. An item that is not contested is refused.

More than one person or clone: the multi-user model. The log is committed with the code, one shard per writer, so distinct writers merge without a conflict; a forge's merge button may not honour merge=union (unverified; it only matters when two branches append to one shard, and a local git pull --no-rebase clears it); offline divergence is detected (contested), not prevented, unless the opt-in remote claim lock (B192; not shipped yet, so claims are local-only today) is on; and committed session prompts, notes and evidence are readable by everyone with access to the repository.

A wrong worktree binding can be corrected. ddflow update <id> --worktree PATH rebinds the item, and your live lease on it, to that linked worktree and the branch checked out there. The field edit lands first and the rebind under the same lock, from a fresh read; an empty path is refused, and complete's tree check then follows the new tree.

If you are already in one, it adopts that one. Agent harnesses — Claude Code, Cursor — often isolate the agent themselves. Claiming from inside a linked worktree binds the item to that tree and branch rather than building a rival and telling you to leave the one holding your uncommitted work:

$ ddflow claim T1            # run from inside the harness's own worktree
claimed T1 (lease 1800s, renew every 300s)
  worktree: /work/agent-tree  (adopted — you were already in it)
  branch:   agent-work
  Carry on where you are.

ddflow never needed to have created the tree — it needs to know which tree an item is worked in, so recover can find stranded work and merge knows what to merge. An adopted tree is recorded as adopted, not created, so remove_on_merge will never delete something ddflow did not make. A tree already bound to another open item is refused: two items in one tree cannot be merged or recovered separately — unless that item has let go of it: merged, lease released, and nothing left in the tree (no uncommitted change, a HEAD its target already contains). worktree.adopt_existing = false restores the old behaviour; --no-worktree skips binding entirely. [worktree] local_files copies git-ignored, machine-local files (a .roborev.toml) from the primary into every tree a claim binds — one it creates, the harness tree it adopts, or the item's own tree on a re-claim — and never overwrites a file already there.

An item claimed --no-worktree has no branch of its own, so merge and review take one: the branch checked out in the worktree you run them from, or --branch <branch> from anywhere, and gate run runs in the worktree you stand in. Two places are not the item's and are refused rather than guessed: another open item's worktree, and — with worktrees on — the primary, whose working tree is nobody's in particular (other agents' event logs, anyone's uncommitted edit); there merge refuses, and review and gate run record UNAVAILABLE and say what to pass. A lone agent working in the primary says so with [worktree].enabled = false. The borrowed tree is never removed; uncommitted work in it is refused as for any merge; and paths the merge lands outside the item's globs are listed (outside_globs), since a borrowed branch can carry another item's commits too. The listing is exact for renamed files (both paths) and non-ASCII names; when git cannot list the landing at all, outside_globs_unknown is true and the CLI says so, rather than an empty list passing for a clean scope (it is null for an item landing its own worktree, whose paths are not listed).

$ ddflow merge B-fix --branch agent-work
  no worktree of its own: 'agent-work' was landed, no tree touched.
merged B-fix (1c2d3e4f) into main

What merge reports and what it leaves alone. sha is the commit the base points at after the landing — the merge commit, or the branch head on a fast-forward; the forge's merge commit in PR mode — and branch_head is the merged branch's own head, which under a squash is not on the base at all. A tree you are standing in is kept (its directory is your shell's cwd; deleting it fails your next command), and ddflow cleanup --apply removes it once the item is complete. Every removal of a worktree — merge, the PR flow, cleanup --apply and the onboarding sweep — is one guarded, logged step (cleanup.dispose_tree): under the log's lock with the leases re-read, a tree another live lease holds or an adopted one is kept and named, git's own check refuses uncommitted files and unmerged commits unless the caller has proof it may force, and a removal that worked appends worktree.removed. A merge that fails is aborted, so the primary checkout is never left mid-merge (doctor flags one that was left, with the git merge --abort that clears it). A branch with nothing ahead of its target is refused — it would record the item merged with nothing landed — unless --allow-empty.

A second agent is refused, and told what to take instead:

$ ddflow claim P1.T4 --agent gamma
P1.T4 writes 'shortener/store*.py' which overlaps 'shortener/store.py' held by alpha on P1.T1

You could take instead: P1.T2, P1.T5

Exit codes are the contract, and agents branch on them:

"Nothing is ready" and "everything is fine" are different facts. An agent that cannot tell them apart invents work.

Over MCP the code is the result's _meta.exit, from the same table the CLI exits with (core.outcome.exit_for): a refusal raised anywhere -- a lease held by someone else, a reviewer only a person may add, a log written by a newer ddflow -- is 3 with its remedy, not "bad arguments" or an internal error.

The critical path is reported, because it, not the task count, sets the wall-clock floor — adding a fifth agent to a phase whose runtime is a four-deep chain buys nothing. It walks nested sub-tasks: an umbrella's open sub-tasks count as steps before it, and a phase another depends on contributes its chain. Items held back only by a cap (the parallelism limit, worktree.max_parallel, or a resource's capacity) are counted by status and brief, and the cap's message says when a slot is free and, under auto, what limits it: "N more are ready but held by the parallelism cap (auto: limit 6, limited by load per core)". Both print one line about the limit in force, parallel: 6 (auto: ceiling 8; limited by load per core) or parallel: 4 (fixed) (status --json and ddflow_status carry it as parallel); "limited by independent work" means the offer already holds every ready item that can run beside what is in flight. A signal is named only while it is over its low mark (the one that lowered the limit, only while it still reads: a null one holds nothing); with every signal healthy the line says "limited by growth step (next in Ns)" while in flight has reached the limit (it rises one step per adapt_up_after_s), and "limited by demand (N in flight)" while it has not. In fixed mode the cap's words are exactly what they always were (schedule.max_parallel_tasks=N). A shrink never touches a running lease: in flight above the new limit simply leaves no slot until enough finish; the limit is never below 1, and a critical signal pauses new admissions for that evaluation only. What next offers is conflict-aware: the free slots are filled in priority order, but an item whose globs overlap one already offered in the same answer is held back as a conflict naming that item ("globs overlap X ... offered in this plan"), and the next independent item takes its slot; status counts these apart from the cap-held ones ("N overlap an offered item" vs "N held by the parallelism cap"). ddflow wait sleeps until something is ready, but when every blocker needs a An item tagged no-worktree (a review, a research task) is not held by worktree.max_parallel and is claimed without a tree. With [flow].claims = "remote" a claim also takes refs/ddflow/claims/<id> on the remote by compare-and-swap, so two clones that cannot see each other cannot both claim an item while online: the second is refused naming the holder, release and completion delete the ref, a heartbeat extends it, a lapsed one is replaced, and an unreachable remote refuses the claim rather than claiming locally. person — a dependency cycle, an expired lease under reclaim_policy = "report" — it refuses at once rather than sleeping to its timeout.

Lanes: who is working and what to do next

ddflow lanes is the coordinator's ledger of agent lanes (an agent identity is a lane). It is a projection of the event log, so it needs no registration and cannot drift, and it never starts, stops or resumes an agent and runs no daemon (D-sched-no-daemon). Acting on a lane (starting or resuming it) is a separate command, still to come.

$ ddflow lanes
LANE     HARNESS      MODEL    ITEM  HEARTBEAT  STATE                 FOR
holder   claude-code   opus     X     2m ago     working               14m
waiter   -            -        Q     8m ago     waiting-on-glob       8m
  waiter: blocked by holder
$ ddflow lanes advise
1. [stall] Y is blocked by lane holder (X: globs overlap X held by holder (f.py vs f.py)); idle: waiter. Ask holder to finish, narrow its globs, or release
   run: ddflow wait --item Y

States: working; stale (it holds its lease but has not beaten for lanes.stale_after_s, default lease.ttl_s); waiting-on-glob (names the holder) and waiting-on-dep; idle-with-ready-work; stopped (done), stopped (session ended, item unfinished) and stopped (reason unknown). The last is what a lapsed lease with no session.ended reads: the log does not say whether the context ran out or the process died, and lanes does not guess. --watch redraws when the log changes or every lanes.watch_interval_s.

ddflow lanes brief <lane> renders what a coordinator would otherwise write by hand as the lane's first prompt: the objective (the item's title and body), its phase, tags, needs and globs, the open lease and worktree, and the lane's previous run (its last session summary and notes). Every text an agent wrote is fenced as recorded data. ddflow lanes advise lists the moves, most urgent first (at most lanes.advise_max): a stopped lane with unfinished work, an item blocked by a named lane, lanes idle beside ready work, more working lanes than the adaptive parallelism limit. Over MCP it is ddflow_lanes (view = state, brief or advise; lane), in the full tool tier. [lanes] knobs: window_s, stale_after_s, watch_interval_s, advise_max, brief_notes, brief_text_chars.

Adaptive parallelism: configuration

How many items may be in flight adapts by default (decision D-adaptive-flow-accepted). In [schedule]:

The shipped marks (decisions D-adaptive-flow-accepted and D-unify 8); high shrinks the limit, critical pauses admission, and at or under low the signal counts as healthy again:

gate_failure_rate, merge_failure_rate, loop_findings and independent_ready ship with no marks: they are read and recorded but never move the limit until marks are set. A [schedule.signals] table in a config FILE that is not valid (an unknown signal, a missing or non-numeric mark, low over high) does not stop ddflow: it is reported on stderr and by doctor, and the strictest table applies instead -- every signal enabled, each mark the lower of the layers below and the shipped one, the bad table's own numbers ignored (D-enum-fallback-strict). config --set and ddflow_configure still refuse it.

worktree.max_parallel defaults to 0, which follows the schedule limit -- it is not unlimited; there is no unlimited setting, the ceiling always bounds it. A nonzero value stays an independent hard cap on worktrees. ddflow init and adopt no longer write the old explicit max_parallel = 4 / max_parallel_tasks = 4; a project that still carries them loads unchanged (parallel absent means auto, and max_parallel_tasks = 4 is then the start value), and ddflow doctor notes an explicit worktree.max_parallel = 4 -- it caps auto at 4 worktrees -- with the remedy ddflow config --set worktree.max_parallel 0.

Every knob is set from the CLI (ddflow config schedule.parallel fixed, ddflow config --local --set schedule.max_parallel_max 6) and MCP (ddflow_configure); the local layer wins over the committed one. A mark is a nested key: ddflow config --set schedule.signals.load_per_core.high 0.8 keeps the low mark, because the table merges layer by layer, mark by mark. In auto the range must hold max_parallel_min <= max_parallel_tasks <= max_parallel_max: an edit that breaks it is refused (exit 3); a file that already breaks it still loads, the controller clamps it, and doctor says so. An unknown signal name is refused with the list of signals. A signal with no marks (all but load_per_core, as shipped) is read but never moves the limit until marks are set for it.

Adaptive parallelism: the sample ring and the derived limit

The adaptive limit is derived and local, never committed. There is no daemon: every command that loads the project -- next, brief, claim and heartbeat among them, and heartbeats already run every lease.heartbeat_s on every platform -- appends at most one sample per schedule.signal_interval_s to .ddflow/local/flow/samples.jsonl: the host signals (below), the log-derived rates, how many items were in flight and whether the limit was binding. The ring lives under the git-ignored .ddflow/local/ (which also ignores itself, for a project whose .ddflow/.gitignore predates it), keeps the last six hours, and is folded through the controller with the [schedule] parameters whenever the limit is asked for, with the log-derived rates re-read at that moment. next, brief, status and wait plan with that limit. A corrupt or truncated line is skipped (and the file rewritten on the next sample), a clock that went backwards drops the sample, a short O_EXCL lock file keeps concurrent samplers to one sample per interval (the file's integrity does not depend on it: whole lines are appended and rewrites are atomic renames), and a directory that cannot be written leaves the limit at its start value with the reason. In fixed mode nothing is sampled. ddflow doctor notes a ring that cannot be read or written and a .ddflow/local/ that git does not ignore.

Adaptive parallelism: host signals

The adaptive parallelism controller steers by the host as well as by the log. ddflow/infra/signals.py samples three host signals with the standard library only; each one a platform cannot supply is unavailable (None), never 0, because a zero would read as a real measurement. An unavailable signal is neutral: it never lowers the limit and never justifies raising it, and with no host signal at all the limit holds at its start value. A sampler that raises or times out is unavailable too, with a one-line reason.

The controller reads them as load_per_core, memory_pressure (1 - memory_free_frac) and disk_pressure (1 - disk_free_frac), so that for every signal it sees higher is worse. ddflow doctor names each host signal that is unavailable on this machine, with its reason, as a note, not a problem.

Log-derived flow signals

Besides the host, the adaptive parallelism controller takes what the project's own event log shows as input signals (the module computes them; the controller's sampling path that consumes them is a separate piece of the adaptive-flow work). ddflow/core/flowsignals.py computes each as a pure function of the log and an injected clock, against baselines taken from the same project's history, so no number is specific to a machine or a repository. Each ends at "now"; a signal with too little history is unavailable (None), which is neutral: it never lowers the limit and never justifies raising it.

A fresh project simply reports the first four as unavailable until it has history; nothing needs configuring. A merge git tried and failed (a conflict) is recorded as a failed merge gate outcome, which is what merge_failure_rate counts; a merge refused before git tried (target checked out elsewhere, another merge in progress) is not. Reviewer latency is the elapsed_s that ddflow review records in the gate outcome's evidence: the reviewer's own time, not the agent's.

Gitflow, pull requests and version tags

Two independent axes in [flow], because teams combine them freely:

Which workflow that makes, and what is not covered:

Each of these is a workflow choice: the operator sets it, or an agent records it, and when nobody does the default is applied at first use and followed from then on.

The agent's loop does not change. next → claim → work → gates → merge. What merge means changes with the repository's policy:

• merge in PR mode pushes the branch and opens (or updates) a pull/merge request, then releases the lease and parks the item in REVIEW. The agent is free at once and takes the next task — nobody waits for a human. It refuses to open a request for work whose own pipeline is unfinished: a reviewer's time is the scarce resource, and a refusal after the merge could no longer stop anything.

• pr sync turns what reviewers did back into queue state. next runs it for you while anything is in review (sync_on_next):

  the forge says	ddflow does
  merged	passes the merge gate on the forge's evidence (URL, merge sha), completes the item, retargets anything stacked on it, removes its tree
  changes requested	returns the item to the queue with the review text (bodies and line comments); brief leads with it; a re-claim resumes the same tree and a re-merge updates the same request
  closed	parks it for a person — a "no" is not something to retry
  approved, checks green	merges it (pr_merge = "on_approval", pinned to the approved head)
  enqueued in a merge queue (GitHub; gh pr merge on a queue-protected branch enqueues)	stays in review and is said to be queued: queued with its position, pr status shows [merge queue #N], and it is not asked to merge again. A request that was queued and no longer is, still open, was ejected (queue_ejected); the queue landing it is an ordinary merged

• Review threads are data. Feedback still arrives as text in brief, but each line conversation is also a record with the forge's own id: pr threads T1 lists them live (id, file:line, author, the opening comment, resolved or not), and pr threads T1 --thread ID --reply "renamed in abc123" --resolve answers one and marks it resolved — so the reviewer sees which comments were addressed instead of re-reading the diff. GitHub (GraphQL review threads) and GitLab (resolvable discussions); the write is the forge's own, made as the operator who is logged in. MCP: ddflow_pr_threads.

• Stacking keeps work moving through review. While T1 waits in review, a task that needs it may start on top of T1's branch (stack = true); its request targets T1's branch and is retargeted to the real base when T1 merges. ddflow never merges a stacked request first — that would land it unreviewed inside T1's merge. A task depending on two unmerged branches waits: one branch cannot sit on two.

• Who presses merge is pr_merge: on_approval (default — a person's approval is still required, and branch protection still applies), auto (ask the forge to auto-merge when its own rules are met), or human.

Gitflow. Tasks fork from develop as feature/ or bugfix/ branches (by tag); a task tagged hotfix forks from production and lands on production and develop. Merges never switch a checkout: a target that is not checked out is merged in a throwaway worktree, and one checked out in someone else's tree is refused.

Versions. version show reads the highest v1.2.3 tag reachable from the release branch and computes the next version from Conventional Commits (feat → minor, fix → patch, !/BREAKING CHANGE → major; below 1.0.0 a breaking change bumps minor) and from the tags of items finished since (breaking, feature, bug, hotfix). version cut tags it — annotated, with generated release notes. Under gitflow it cuts release/X from develop, merges it into production, tags it and merges the tag back into develop; in PR mode it opens the release request instead, and pr sync tags the merge commit once a person merges it and opens the back-merge request.

A hotfix's back-merge is tracked. Under gitflow + pull requests the hotfix item completes once production has the fix, but the production → develop request it opens is recorded (pr status lists it under back_merges) and every later pr sync re-asks the forge until it merges ("back_merge: merged into develop") or is closed without merging, which is refused loudly: develop would not have a fix production already ships.

Version files. [flow.version_files] maps a path to a regex with exactly one capture group, the version text — 'pyproject.toml' = '^version = "([^"]*)"$' — and version cut replaces that group with the new version (no tag prefix) and commits it as chore: bump version to X, so the tag's commit says what the tag says. The commit is on the branch the tag names: the release branch under gitflow (in pr mode it travels in the release request), the release source for a trunk or maintenance cut. Everything is checked against that source first and a missing file, a pattern matching nothing or matching twice refuses the whole cut (exit 3) rather than guessing which line is the version; a trunk cut with integration = "pr" is refused too, because the bump would have no request to travel in.

Changelog with the cut. version cut --changelog also writes the new version's section into CHANGELOG.md (the path of [export.changelog], default CHANGELOG.md) through ddflow export changelog: the Unreleased entries become ## [x.y.z] - date with the right compare link, and an empty Unreleased remains. The file is committed on the branch the tag names, so the tag's commit holds it; in gitflow with pull requests it is a commit on the release branch, part of the release request, and the request body is the same section (the tag, cut later by pr sync, carries its usual message); otherwise the tag message is the section. Mode whole (the default) rewrites the file; mode region rewrites only the marked region and puts the new section below it, updating the [Unreleased]: link line when the file keeps one. Nothing is written without the flag (the tag message is then the commit-derived notes as before), --dry-run shows what would be written, a hand-edited file or uncommitted changes to it are refused (exit 3) unless --force, and a git or log failure is exit 2. Trunk or maintenance cuts with pull requests refuse --changelog (no request to carry the commit). The MCP tool takes changelog and force. Bumping version files is a separate task.

An item counts as shipped on a line when its landing merge commit or the branch it merged (the merge's second parent) is reachable from it: a hotfix landed on production reaches develop by the back-merge as that branch, with production's merge commit nowhere in it. A squash or fast-forward landing is the one commit it made. A squash landing has no merge parent to follow, so it is also found on another line by its change: a commit there with the same patch-id (a cherry-pick, or the back-merge squash), or the back-merge <id> into ... commit ddflow wrote for it -- so a hotfix squashed onto production counts as shipped on develop once it is back-merged. Only the newest 1000 commits of the line are searched.

ddflow holds no token: it drives gh or glab, logged in as the operator, so every permission question is answered by the forge. A forge that cannot be reached is exit 2 — "could not ask" is never reported as "nothing changed".

[flow]
model = "gitflow"          # or "trunk"
integration = "pr"         # or "merge"
pr_merge = "on_approval"   # or "auto" | "human"
pr_reviewers = ["alice"]

The research behind this is RESEARCH R16.

Several release lines: fixes to older majors

Projects that keep older majors alive — main is 3.x while 2.x and 1.x still get fixes — declare them as release lines, oldest first. The newest line is always the current one and follows model as above; a maintenance line lands straight on its branch.

[flow]
current_line = "3"
port_strategy = "cherry-pick"      # or "forward-merge" (the default)

[flow.lines]                        # oldest first
"1" = "maint/1.x"
"2" = "maint/2.x"

• An item belongs to a line: task add T --line 2, or phase add P --line 2 and every task in it inherits it. A line that does not exist is refused, never read as "the current one".

• A fix for several lines is one command: task add FIX --lines 1,2,3. ddflow writes it where the strategy says and generates a port task FIX@<line> for each other line — an ordinary task with its own branch, gates and merge or pull request. A port starts only once what it carries has landed (review is not enough):

  port_strategy	the fix is written on	each port…	ports run
  forward-merge (default)	the oldest line	merges the previous line's branch into its own (and passes through every line in between — a merge cannot skip one)	one after another
  cherry-pick	the newest line	applies exactly what the fix landed (the target's before→after range, whatever the merge strategy) with a three-way apply	in parallel

  What a fix landed is a range on the target, exact for a merge commit and a squash (merge_sha^1 is the old target). On GitHub a rebase-merge lands N commits, so ddflow reads the target's tip before it merges and checks the range against the request's commit count; a merge a person (or a queue) did is recognised from the change itself — the request's own diff has the same patch-id as the last N commits. If neither can be proven the old merge_sha^1 stays (GitLab reports no commit count, so it always does).

  A port carries what its fix landed. A follow-up fix to the same bug is a new item: task add FIX2 --port-of FIX gives it the lines FIX reached (naming one of FIX's ports works too), so FIX2's own ports carry what FIX2 lands.

• A conflicting port is work, not a failure. The claim leaves the conflict markers in the port's tree and names the files; the agent resolves, commits and carries on.

• Lines never collide. The same file on 2.x and on 3.x is two branches, so two agents may hold them at once; on the same line the glob check applies as always.

• Versions per line. version show --line 2 reads the highest tag reachable from maint/2.x; version cut --line 2 tags it there, and refuses a bump that would leave the 2.x major — a breaking change belongs on the current line.

Environment branches: promoting downstream

GitLab flow's environment branches — each mirroring what is deployed there — are declared in order, downstream of the current line's target (the base branch; under gitflow, production, so an environment receives what was released):

[flow]
environments = ["pre-production", "production"]
auto_promote = ["pre-production"]   # optional: continuous delivery to staging

• Work reaches an environment only by promotion, one step at a time. ddflow promote add pre-production files a task that merges main into pre-production; promote add production merges pre-production into production. So production only ever receives what the environment before it already has ("upstream first"). Nothing else targets an environment branch.

• A promotion is an ordinary task: claim it and its tree is made from the environment branch with the upstream branch already merged in (a conflict is left for the agent, like a port's). It runs gates.promotion_pipeline — unit_tests and merge by default, since what it carries already passed its own pipeline; add a human gate there for a person's sign-off on each deploy, and name it in gates.required to make that sign-off mandatory (every gates.*_pipeline counts, so a gate only the promotion pipeline runs is not an inert requirement). Where environments exist, ddflow workflow lists the promotion pipeline beside the task and phase ones. It needs no cross-family reviewer: it authors nothing.

• With integration = "pr" the promotion lands through a merge request into the environment branch — the approval is the deploy approval — and pr sync completes it.

• One open promotion per environment, and none when there is nothing to carry (exit 2).

• auto_promote lists environments ddflow next promotes to by itself when the branch upstream moves. Empty by default: a deploy is the operator's call.

• ddflow promote status shows each environment's head, how many commits it is behind the branch upstream of it, and any open promotion. The branch says what was promoted, not what is running: have the deploy hook call ddflow promote deployed production --sha "$GIT_SHA" (default: the branch head) and status adds live <sha> and how many commits of the branch are not yet deployed — "what is live" answered exactly.

Workflow choices: asked, recorded, defaulted on the record

ddflow supports several ways of working and never picks one silently. Each decision — model, integration, pr_merge, on_changes_requested, stack, port_strategy — is a choice, and its value comes from, in order:

1. the operator's config (.ddflow/config.toml, the machine-local .ddflow/local/config.toml, or env), which always wins;

2. a recorded choice — ddflow flow choose port_strategy cherry-pick --reason "2.x has diverged", by the operator or by an agent the operator left it to, attributed in the log;

3. the default — applied the first time the choice matters (the first claim, the first pull request, the first fix filed across lines) and recorded, so the project keeps following it even if a later ddflow ships a different default.

Until then, a relevant choice nobody made heads ddflow brief under Open workflow choices, so an agent asks at the start rather than discovering at the end that the project wanted something else. ddflow flow show lists every choice with its value, its options, and who decided — config, a named agent or person with their reason, or "DEFAULT (nobody chose)". A recorded choice that any config layer overrides is shown as such — and flow choose answers in_effect: false naming that layer — never silently ignored.

Research: RESEARCH R17.

Many agents, one server: identity, state and sharing

Several agents and subagents sharing one queue is the case this tool is for. Here is exactly how that works, because each of these has a wrong answer that looks right.

Is it stateless?

The queue is. The connection is not, in exactly one respect.

The append-only event log is the sole source of truth, and every read re-derives state from it — fold(read_all()), from scratch, on every call. Nothing is cached between requests, so there is no stale projection, no invalidation, and no divergence between two agents' views. Restart the server mid-task and nothing is lost: it never had anything the log did not.

The one piece of per-connection state is who you are (below). It is deliberately not in the log, because it is a property of the caller, not of the work.

Who is calling?

By default, identity is derived from the working tree. That is right for one agent per worktree, and silently wrong for several agents in one tree — they all resolve the same path to the same name, their events merge into one stream, brief answers with a sibling's task, and reviewer-independence compares an agent with itself and passes. Nothing errors. There is no signal that can tell them apart, so identity is declared:

Under a declared identity a call uses what that agent holds, on every surface: an MCP tool called with a foreign as_agent asks from the primary and ddflow_claim creates that agent's own tree instead of adopting the connection's harness worktree, and ddflow --agent X claim does not adopt a tree another identity is working in -- one it has worked in under its tree-derived name, or one it ADOPTED (worktree.adopted names the adopter, so a parent that always declares --agent still owns its tree once it has claimed from it; a parent that has never claimed from its tree is not yet detectable, and the first identity to adopt a tree owns it). A reviewer written through ddflow_configure is recorded under the per-call as_agent. ddflow_setup writes into the tree the server stands in, as adopt does; adopt run from a linked worktree writes the project's tracked files into that worktree (the event log and hooks stay shared).

A name you set yourself is never suffixed, so the same DDFLOW_AGENT in two clones is still one agent to ddflow; ddflow doctor notes a shard whose clock goes backwards, which is what that leaves behind after a merge.

Innermost wins. ddflow_identify is idempotent, persists for the connection, and refuses a name that could not be a log filename — it becomes one, and refusing at declaration time means the caller reads the reason rather than discovering it at the first write.

ddflow_identify also reaches the same harness's shell: the name is recorded (in the primary checkout's .git/ddflow-identity/, keyed by the harness's pid and start time), and a CLI run below that harness that names no identity (--agent, DDFLOW_AGENT) takes it, so a ddflow claim in the shell and a ddflow_heartbeat on the connection are one agent, and so is the commit hook. Linux only; ddflow_identify with no name withdraws it.

If more than one agent works one tree at once, declare identity. Everything that attributes work depends on it. A subagent must not call ddflow_identify on a shared connection — that renames its parent — and passes as_agent instead; without it, two subagents claiming the same file are one holder, and a holder's own leases never conflict with each other.

Can one server serve several projects?

No — one server process serves one repository, fixed at start from --repo, DDFLOW_REPO, or the working directory. No tool takes a repo argument, and a test asserts none ever does. Point a second agent at a second project by running a second server; they are cheap, and the isolation is the point.

One project shared by many agents is the supported case — and the one that needs no special setup beyond declaring identity:

• Writes never conflict, and they never block readers. Each agent appends to its own log shard, so there is no shared file to overwrite and no merge conflict to resolve. Writers do serialise briefly: one repo-wide lock is held across the clock allocation, the append and its fsync. Short, but not nothing — per-agent shards remove file contention, not lock contention.

• Reads take no lock at all, so a read-heavy agent cannot be starved by a write-heavy one, and a reader can never block a writer.

• File ownership is coordinated by globs. claim refuses an item whose writes overlap one already held, and names what to take instead — exit 3, not a failure.

• Lessons, decisions, research and bug history are shared by construction: they are events in the same log, so one agent's finding is immediately visible to every other.

Locking, contention and measured cost

Measured on this machine, single process, full read_all() + fold():

Linear, converging on ~8.7 µs/event; the higher figure at small sizes is fixed per-call overhead, not the fold. A project with 20,000 events pays ~175 ms for a state-reading call. Search and recall do not pay this — they run off a SQLite projection rebuilt only when the log's head moves.

tests/test_mcp_load.py runs 12 concurrent agents through the real MCP surface and asserts no deadlock, no lost append, no repeated Lamport value within an agent, and correct attribution for every event — not for a sample. Its thresholds are environment variables (DDFLOW_LOAD_AGENTS, DDFLOW_WRITE_LATENCY_BUDGET_S, DDFLOW_GROWTH_TOLERANCE, …) because a load test with a hardcoded budget either flakes on a shared runner or is too loose to fail.

The deadlock bound is a hard timeout: a wedged lock does not fail, it hangs, and an unbounded hang reads as a broken CI runner rather than as a bug.

Crash recovery

An agent is killed. Nothing is cleaned up, because in a real crash nothing runs.

$ ddflow recover
1 recoverable situation(s); 1 may contain work:

!! P1.T1  [expired_lease]  was: delta
     worktree /repo/.ddflow/worktrees/P1.T1
     INSPECT FIRST — 1 uncommitted file(s), 1 unmerged commit(s).
     `git -C .../P1.T1 diff main` then salvage,
     then `ddflow release P1.T1 --note salvaged`.

Four behaviours, each chosen against a specific way this goes wrong:

• While the lease is live, nothing happens. A dead agent is indistinguishable from a slow one until the lease expires, and guessing is how two agents end up in one tree.

• Recovery measures the tree — uncommitted files, unmerged commits — rather than trusting the recorded state. "Is there work in here?" is the only question that decides the remedy.

• An expired lease is never stolen silently, and recover --apply expires only trees it measured as empty.

• Adoption, not duplication: an agent resuming a recovered item gets the existing worktree back, not a second one beside it.

• The brief leads with it. ddflow brief names, before anything new, the trees holding work, the trees it COULD NOT MEASURE (treated as holding work) and the RUNNING items nobody holds -- up to five of each, the rest counted; leftovers with nothing to salvage are counted in one line. recover (its !! flag and "may contain work" count), status (recoverable and "may contain unsaved work") and doctor (a PROBLEM, not a note, for a tree) count the same three, so an unmeasurable tree is never reported as nothing to lose.

Reconstruction from logs alone

$ ddflow replay --out ./recovery-kit
wrote:
  recovery-kit/RECONSTRUCTION.md
  recovery-kit/QUEUE.md
  recovery-kit/LESSONS.md

RECONSTRUCTION.md is written as instructions to a fresh agent, not as a report about the past: every operator prompt in order, every research verdict, every lesson, the queue's shape — and every approach already tried and rejected, with the measurement that killed it.

It states its own limit, in the document: it reproduces the decisions, not the bytes. Model outputs are not deterministic, so replaying prompts will not recreate the original source. What it recreates is every input that produced it, which no other artefact holds.

The demo destroys an entire repository and rebuilds from 3.9 KB of JSONL, then checks nine specific fragments are present — including the operator's stated reason for a constraint, and the probe output behind a rejected design.

Secrets are redacted on the way in, not on the way out — the log is committed, so a scrub at read time is a scrub that git show walks straight past.

One shared Redactor (core/redact.py) serves the log, the views, the exports, upstream reports, each a named profile (redact_argv gives the shape of a command line). The log profile is the full one: besides secrets (which keep their surrounding words, api_key: [REDACTED]) it removes private addresses and hosts, this machine's hostname, home-directory paths, emails, and (once the [upstream] section exists) its redact_extra names from the free text of every NEW event, not only prompts and notes: EventLog applies it where every write passes: every string of an event's data is redacted unless it is a known lookup (core.redact.LOG_VERBATIM_NAMES: ids, digests, shas, paths, globs; LOG_VERBATIM_FIELDS: a few per-kind pairs, each with its reason), so gate evidence and output tails, bug summaries, lessons, decisions, task bodies, reasons, notes and a field the vocabulary has never heard of are all redacted. LOG_TEXT_FIELDS names the fields that are text even where their name is a lookup name. A new event kind must be classified or tests/test_log_redaction_all_kinds.py fails; a new field of a known kind that is a lookup must be added to those tables when the event-kind snapshot is regenerated, or it is redacted. A definition's digest (def.recorded) is of the redacted fields it stores, with every redaction marker ([REDACTED], [REDACTED:secret], ...) read as one token (core.digest.normalized), so clones on different redaction profiles agree on whether a definition changed; a digest recorded before that still matches (core.defs.same). The duplicate check's "identical text" digest does the same and also ignores case and whitespace (core.digest.normalize_text; the similarity index re-derives once). A trigger definition's fingerprint is core.digest.of_obj of its fields; a fire that carries the older spelling still counts as the same definition. Events already written are not rewritten.

Lessons that check themselves

A lesson can name the mistake in code, not just in prose:

$ ddflow lesson add --title "Never swallow a bare OSError" \
      --rule "Catch the specific error; a broad except turns a loud failure into a silent one" \
      --pattern "except OSError" --globs "*.py"
lesson La0c745cc recorded — inventory: 2 site(s) now

$ ddflow lesson verify          # later, after somebody adds a third
La0c745cc: 1 NEW site(s): c.py: except OSError:

Exit 1 names the file. That is the whole design, and it comes from a failure worth repeating: on the project ddflow was extracted from, a count-based clone ratchet sat red for ~350 commits. It was advisory so it never blocked, it reported a number so every reader learned to skip it, and 24 new clones arrived through that gap.

A count says "worse" and never "which".

A number cannot be acted on or reviewed. A list can: a new entry is a line somebody opens, and a disappeared entry is progress — reported, and never a failure, because the inventory may only shrink.

Three details that decide whether a ratchet survives contact with a real repository:

• A site is <path>: <matched text>, not path:line. Line numbers churn on every edit above a site, which would invent a matching pair of "new site" and "fixed site" findings out of an unrelated change — and a ratchet that cries wolf is one that gets switched off.

• An uncompilable pattern is refused, not stored. An empty inventory reads exactly like a clean repository, and would ratchet every real occurrence away the first time it ran.

• Exit 2 when no lesson declares a pattern. Not a pass. A corpus with zero ratchets should not be able to report "all clear".

Vendored and untracked files are never sites — matches in code nobody owns are findings nobody will act on. Most lessons stay prose, and a prose lesson produces no findings at all.

Checking that the checks are working

Three questions ddflow asks about itself, all derived from the log and all reported by ddflow doctor. They exist because an unmeasured mechanism is indistinguishable from a missing one.

Can the queue's work actually be picked up? Every other check counts the items that are present; this one asks whether any of them can be started. An open phase with no task under it is work ddflow next will never offer — a note by default (schedule.empty_phase, since a project that files phases before breaking them down lives there on purpose) — and a phase whose tasks are all finished while the phase stays open is always a problem, because that is a queue held open by an item nobody can act on.

Tasks cannot go missing here, and that is a property rather than an untested gap: two hypotheses about how one could were probed and both refuted, and a test now pins the invariant so a future filter cannot quietly reintroduce it.

Does a gate ever say yes? A gate that fails on everything is worse than no gate: it trains the next reader to skip it. A gate at or above gates.rate_max_fail once it has gates.rate_min_runs decisive runs is reported as flaky or as measuring a moving target — re-running it will not converge. A skipped gate is not a run, because counting skips as failures would make an unconfigured gate look like a broken one.

Did the periodic passes ever fire? The mechanism you did not measure is the one that is not running. Because a cadence here counts completions rather than wall-clock, this is exact rather than estimated: since is the completions elapsed since the pass last fired, which is the same quantity ddflow cadence uses to decide due-ness — deliberately, because two measures of "is this behind" that can disagree is a situation nobody can reason about. A pass more than cadence.max_missed scheduled runs behind is reported. Being merely due is not a finding (ddflow cadence already says that), and running early is not one either.

All three are notes, not problems: a defect in the machinery that checks the work must not block the work.

Reading the log, and why it is never compacted

The log only grows, so every state-reading call used to re-read and re-parse all of it. Measured at 20,000 events, that read costs 115 ms — and the breakdown is the whole design argument:

Parsing dominates, and an append-only file guarantees the bytes already parsed have not changed. So EventLog.read_all re-parses only the appended tail, and re-hashes the bytes it is re-using to prove they are still the same bytes:

A command like ddflow doctor — which reads four times — pays the full cost once instead of four times.

The sort and the de-duplication are maintained too (B169). read_all keeps the merged, Lamport-sorted, de-duplicated order between calls and folds each append into it by bisection, falling back to a full rebuild whenever the shortcut could not be proven identical (a shard whose bytes changed, a repeated event id, a batch of more than 64). Measured, one appended event, tests/bench_log_read.py: 20,000 events 14.8 ms to 3.8 ms; 100,000 events 75.9 ms to 19.5 ms. What remains is reading and hashing the log's bytes, which is the price of verifying them and is linear in the log.

A one-shot CLI call no longer parses everything (B166). Once a log has 5,000+ events, read_all writes a snapshot of the parsed events and their merged order to .ddflow/local/read-snapshot.bin (machine-local, git-ignored, never merged). A later process loads it instead of calling Event.from_json on every line: 100,000 events cold 685 ms to about 400 ms (1.7x), 20,000 events 125 ms to 63 ms (2x), median of three runs on a loaded machine. It is a cache and is trusted only as far as it verifies: per shard, the SHA-256 of the bytes it describes must still match the shard on disk (so a rewritten, truncated or switched-branch shard is re-parsed, and an appended or torn tail is parsed as usual); the file's own checksum, format, ddflow version and Event fields must match; any doubt means the snapshot is ignored and the log parsed. ddflow doctor's integrity check (EventLog.verify) never uses it, and a log opened only to read another repository (external sync, exports) neither reads nor writes one. It has the trust of the rest of .ddflow/local/: your own checkout's machine-local state, never fetched or merged. Deleting the file is always safe; DDFLOW_SNAPSHOT=0 turns it off for a process (or [log] snapshot = false). It covers up to max_cached_events (a larger log is parsed each time, as before). The table above is reproduced by python tests/bench_log_read.py.

Six knobs for reading the log, [log]:

The validity check is a content check, and that is the whole design. The consumed prefix is re-hashed on every read — 1.1 ms to read plus 4.3 ms to digest, against the 97 ms of parsing it avoids. The first version used st_ino instead, on the reasoning that "a git merge writes a temp file and renames, so the inode changes". That is false:

$ git checkout -q other && stat -c %i .ddflow/events/a1.jsonl
218500670
$ git checkout -q main  && stat -c %i .ddflow/events/a1.jsonl
218500670

Git rewrites tracked files in place. So switching between two branches that had diverged left a warm server serving events from the branch you left, silently losing the ones actually on disk, with the tail read starting mid-line — and because Store.rebuild takes its fingerprint from the real file while taking its events from the cache, that wrong state was written into the SQLite index stamped as current, which a fresh process would not rebuild away. A content digest makes a rewrite, a truncation, a git checkout, a git merge, a delete-and-recreate and a torn tail all one case, so there is no list of mechanisms to keep current.

A torn final line from an append that died mid-write is reported by ddflow doctor and re-read until the writer completes it, never marked consumed. Every guarantee here is mutation-verified in tests/test_log_read_cache.py — including that the digest covers the whole prefix rather than a trailing window of it, which a smaller fixture cannot tell apart.

Events that vanished from a shard

Events live in git history only, so a git push -f of an older main, a revert of the events commit, a git reset --hard or a deleted shard file takes events out of the tree, and a torn-line check cannot see an event that is simply gone. ddflow therefore keeps a shard high-water mark: .ddflow/local/shard-marks, a machine-local file that is never committed, with one row per (branch, shard) holding how many events this clone last saw in that shard on that branch, the id of the last one, and the file's size then. Every command raises the marks to what the shards hold and none lowers them (only --accept-lost-events, below, does), so ddflow's own operations (which only append) never shrink a shard. A mark is compared only on the branch it was taken on: checking out a branch that never had the events is not a loss. A detached HEAD takes and compares no mark.

When a shard now holds fewer events than its mark, ddflow doctor reports a PROBLEM (exit 1) and the brief opens with a "Lost events" section saying the same:

<shard> lost 4 event(s) since this clone last saw it (was 120, now 116); the last event it saw
was <event id>. They may still be in git: `git reflog`, `git log --all -- .ddflow/events/<shard>`,
`git show <sha>:.ddflow/events/<shard>`. Nothing was rewritten; `ddflow doctor
--accept-lost-events` records the new count once you have decided

A shard that grew back past its mark but no longer holds the last event this clone saw reads <shard> no longer holds the event this clone last saw in it (it held 120, now 125) instead. The hints are the way back: the lost events are usually still in the reflog or in another branch's history, and restoring them is an ordinary git operation. Nothing is rewritten or repaired by ddflow.

ddflow doctor --accept-lost-events is the operator's explicit acknowledgement that the loss is known and intended (the force-push was deliberate, or the events were restored elsewhere). It sets the marks of the shards that lost events to their present counts, so the PROBLEM clears, and adds a note per shard (<shard>: accepted the new count (116, was 120)); the log is not touched. It is a CLI-only action: there is no MCP parameter for it, because an agent that met this PROBLEM must put it to the operator rather than silence it.

Known limit. The mark is a count and the last event's id, not the set of ids. A rewrite that removes an interior event and then brings in enough new ones to reach the old count, with the old last event still present, all before any ddflow command looks at the shard, cannot be told from growth. Closing that needs a per-event record and belongs to the P-multiuser sync-check, not to this mark.

Managed definitions: the def.* event kinds

Doc types, schedules, triggers, skills, agents, research claims and rules are all definitions: named documents that are written, revised, retired, replaced or merged. They share one record (B-uni-def-records) instead of a set of events per family. Five event kinds carry them, each with the envelope {kind, id, digest, source, provenance}:

• def.recorded: a whole definition (fields). Recording an id again replaces its fields and brings a retired, superseded or merged one back.

• def.updated: some fields; a field set to null is removed.

• def.retired: with a reason.

• def.superseded: replaced by another active definition of the same kind (successor).

• def.merged: folded into another active definition of the same kind, which lists it as merged from.

The fold keeps each definition under kind:id, so a schedule and a skill may share a name, with its digest and its full history. All five kinds are provenance kinds, and ddflow replay renders each one. The one write path is ddflow.api def_record / def_update / def_retire / def_supersede / def_merge, with def_show and def_list to read. It refuses an unknown kind, an id that cannot be an id, or fields that are not plain JSON (exit 1). It also refuses to revise a definition that is missing or no longer active (exit 3). An update that changes nothing is exit 2. A new definition runs the add-time duplicate check when its kind is in [dedupe].kinds: rule is by default, and other families add their kind when they start filing. The CLI and MCP surfaces are B-uni-record-surface.

A log from a newer ddflow, and the link event kinds

Checkouts of one repository can run different ddflow versions, and a shard merged from a newer one may hold event kinds this code has no handler for. They are skipped, as an unknown config key is, but not silently: ddflow doctor adds a note naming each unknown kind and how many events were skipped (this log has events from a newer ddflow ... skipped: record.extended x2), and ddflow status carries them as skipped_kinds. Every number the older ddflow shows is computed without those events, so the remedy is to upgrade ddflow-mcp (and restart the MCP server) to at least the version the log was last stamped by (next section); the note names it. An event whose schema is newer than this code knows is counted the same way (phase.added (schema 2) x1).

Payload versions and upcasters. The log is never rewritten, so when an event's data has to change shape the kind's payload version goes up ("v": 2 in data; absent means 1) and a pure upcaster in ddflow/core/upcasters.py turns the old shape into the new one as the log is read: every handler sees the current shape, and the events every released ddflow wrote keep folding. A payload version newer than this code knows is counted like an unknown kind (phase.added (payload v2) x1), never guessed at. tests/fixtures/event_kinds.json is the committed snapshot of every kind's fields and types; a removed kind, a removed field or a changed type fails the suite unless it comes with a version bump, its upcasters and an entry in the compatibility contract. An added kind or field only has to be recorded (python tests/test_event_upcasters.py --write).

One malformed event no longer stops the fold. An event whose handler raises (a hand-edited shard, a ddflow bug) used to abort every read of the log. It is now recorded as a fold problem and the fold goes on; ddflow doctor adds a note naming each one (<event id> <kind> at lamport N by <agent>: <error>) with the advice to file it; what that event would have changed is missing from every number (what its handler changed before it raised stays). A strict fold (the tests' default) still raises.

The compatibility contract

What ddflow promises not to break is written down in docs/ddflow/compatibility.md (decision D-compat): which surfaces are interfaces (commands, flags, MCP tools and their results, --json fields, config keys, event kinds and payloads, template data, hook commands, on-disk formats); that every change is additive, deprecating (a hidden alias with a notice, kept until 1.0) or breaking (a declared migration and a minor bump while 0.x); that releases are numbered by impact; and that FORMAT_LEVEL in ddflow/__init__.py, not the version, says when an older ddflow can no longer read or write a project safely. An older ddflow writes what it understands, preserves the rest, and is refused only on a direct conflict. The skew guard below is the current, version-based form of that refusal.

Aliases and deprecations. A renamed command, command group, flag, MCP tool, tool argument or config key keeps working under its old name until 1.0 (D-compat). The new name is declared once in the command registry (ddflow/surfaces/registry.py: aliases, tool_aliases, deprecated_since, removed_in on Command and Param; renamed_from on a knob in config_sections/), and the declaration is refused if it names no release or lets the old name go before 1.0. An alias is hidden from --help, usage and tools/list and is always callable; the first use in a session says so in one line (ddflow: command 'doc' is deprecated since 0.1.17; use 'docs' ... on stderr for the CLI, a note: in the MCP result, a warning for a config key), and a second use says nothing. An old config key is read (the new key wins when both are set), and the next config write moves it to the new key, keeping comments; an old config --set key writes the new one. An unknown command, MCP tool or argument names the closest known one (Did you mean 'ddflow_probe'?). A command's alias is the same command to the parity test.

Upgrading ddflow

One command, ddflow upgrade, and one MCP tool, ddflow_upgrade, cover everything an operator asks about an upgrade (decision D-self-upgrade). Each mode below is a flag on the CLI and an argument over MCP, so "is ddflow up to date?" and "upgrade ddflow" can be put to an agent.

Check and proposal. The release check (at most once per [upgrade].check_interval_h, never a daemon) PROPOSES a newer release in the brief, status and doctor; it installs nothing. See "Is a newer ddflow out?" below. Self-upgrade installs the newer ddflow with the tool that installed this one and needs the operator's consent (--yes --reason; [upgrade].self); the version it replaced is remembered so --self --rollback reinstalls it. Containers and MCP servers compare their version with the project's stamp on start and run the safe migrations ([upgrade].on_start); they serve even while a migration waits. Backups are local (a copy under .ddflow/backups/) or snapshot (a git tag); --restore puts either back.

Monitor. --status prints the running version, the newest release and when it was last asked, this project's stamp, the plan with each step's state (pending, running, done, failed, unavailable, needs operator confirmation, waived), the backups, and the agents and MCP servers last seen on an older ddflow, which must restart (from the ddflow.seen stamps and skew.overridden events). It exits 0 when nothing is pending and 1 otherwise. --history lists the upgrades the log records. Both only read.

Orchestrate and resume. Every step of an --apply is an upgrade.applied event (a run id and status: started, one step per item, then the closing done or failed; no new event kind; an older ddflow reading the log shows these progress records as empty rows in its history), so replay explains the upgrade and an interruption (a crash, a closed terminal, a timeout) leaves a run --status calls interrupted. ddflow upgrade --resume finishes it: the steps the run settled are not run again, the rest are, what the operator confirmed before the interruption still counts, and a second --resume finds nothing (exit 2). A run that recorded a step in the last 3 minutes may still be applying, so --resume refuses it (exit 3) until it has been quiet that long. Over MCP an apply reports each finished step as a progress notification when the client sent a progressToken.

Rehearsing it first. scripts/ci/upgrade-dry-run --installed-version X (or --from-commit SHA, --previous-release) upgrades a throwaway clone of the project from the version you run now to this tree and prints a PASS/FAIL table; nothing touches the project, its .ddflow, the installed ddflow or a running MCP server (the clone gets a private HOME and no network, and the project's log, snapshot tags and backups are compared before and after). It reads the clone with the new code, shows the plan (the items you would be asked to confirm fail the row), applies it with a snapshot, applies again (a no-op), restores (the files return, every shard keeps its bytes), lets the OLD ddflow read the upgraded clone (its write is refused with the remedy or lands without disturbing a byte), connects the new MCP server to the old project (it proposes the upgrade, applies nothing until consent, then applies), runs two agents through a task, merges shards two old-version agents wrote and folds the old log through one registered event upcaster (the machinery has never met a real payload bump). The old ddflow is --old-cmd, --build-old (installed from the clone with uv) or --simulate-old VERSION; without one those rows say unavailable. Exit 0 all passed, 1 a check failed, 2 it could not run; --max-show N sets how many items the read pass renders with show (25; 0 is every item); --report FILE and --preview FILE write the Markdown report and the plan. scripts/release.sh and the publish workflow run it from the previous release and refuse a release whose upgrade is red; they pass --confirmations-ok, which names the items the operator would be asked to confirm in the preview without failing the row, because those come from this project's own config, not from the release (run without it, as an operator does, they fail the row).

Turning each off. ddflow config upgrade.release_check off (or DDFLOW_NO_UPDATE_CHECK=1) stops the release check, upgrade.on_start off the check at server or container start, upgrade.auto off the notice after an upgrade, upgrade.self operator keeps an agent from installing, and upgrade.backup none skips the originals. ddflow help upgrade is the same for an agent.

Upgrading a project: the whole picture

A project is brought up to a newer ddflow in three steps, and ddflow does only the first on its own.

1. Silent: it tells you. After ddflow is upgraded, the brief and the MCP handshake print one line, Upgraded ddflow A -> B: run ddflow upgrade --plan: the handshake once per version on this machine for this project, every brief until the project is upgraded. With the shipped [upgrade].auto = "check" the only file that writes is the git-ignored .ddflow/local/upgrade-notice.json. "safe" also refreshes ddflow's own files (hooks, driver docs, rules blocks) after a backup; "off" says nothing.

2. Read-only: you look. ddflow upgrade --plan lists what would change, by category, and writes nothing.

3. Consented: you apply. ddflow upgrade --apply does what the plan lists after saving what it rewrites. Anything an operator set by hand (a config value, a hand-edited driver doc) is never changed without --confirm KEY --reason WHY, which is recorded in the log.

On connect and on container start. A starting ddflow mcp server (and the container that runs it) does not wait for the brief: it compares its version with the project's stamp and, when it is newer, acts per [upgrade].on_start (safe default, check, off; the environment variable DDFLOW_UPGRADE_ON_START overrides it for one run, e.g. docker run -e DDFLOW_UPGRADE_ON_START=check; an unknown value counts as check). safe rebuilds the derived stores (the sqlite index) and, in a container only (running the newer image is the operator's choice), applies the hooks and instructions categories after a backup. On an MCP connect outside a container nothing else is applied without the operator: the handshake instructions and the first brief carry the PLAN as a question (ddflow ... newer than this project ... pending ... Ask the operator), and only on their yes does the agent call ddflow_upgrade with apply. Config values anyone set and every item marked for the operator still wait for confirm and reason. The start never refuses to serve: errors become a line in the report, work past [upgrade].start_timeout_s (10 s) is left running and the handshake says so in a NOTE (... an upgrade is PENDING but its plan is not known yet ... Ask the operator ..., never silence and never "current"); the plan it reaches later is kept in the marker, so the next connect proposes at once, and every brief repeats the note until the project is upgraded (ddflow doctor lists it as a NOTE too, with the same remedy, and nothing when the project is not behind). A read-only or foreign-owned project is reported and not written, and a runtime OLDER than the project writes nothing and says restart (the skew guard). The outcome is kept in the git-ignored .ddflow/local/upgrade-start.json; it replaces the one-line notice for that version. A server that was already running when ddflow was upgraded says restart the server in its replies.

Is a newer ddflow out? The release check. At most once per [upgrade].check_interval_h (default 24), when a brief or ddflow doctor runs, ddflow asks the package index it was installed from whether a newer release exists, and PROPOSES it: one line (ddflow 0.1.10 is available (you run 0.1.9): upgrade ddflow-mcp ... then restart the MCP server) in the brief and the MCP handshake once per version on this machine, and in status and doctor whenever one is known. It installs nothing. ddflow upgrade --check (MCP ddflow_upgrade with check) asks at once, whatever the interval, and prints the running version, the newest release, this project's stamp and the command that upgrades THIS install; exit 0 when this is the newest, 1 when a newer release exists, 2 when the check is off or the index is unreachable.

What leaves the machine: one GET <index_url>/ddflow-mcp/json (the public package index https://pypi.org/pypi/ddflow-mcp/json by default; [upgrade].index_url points a private mirror that serves the same document) with a 3 s timeout. It carries nothing about the project or the machine, and the answer is cached in the git-ignored .ddflow/local/release-check.json. Offline, a timeout or a bad reply is silent, and counts as a check: at most one request per interval. Pre-releases are ignored unless [upgrade].prereleases. A ddflow running from a source checkout is never proposed an index release by the periodic check. Turn it off with ddflow config upgrade.release_check off or, for one shell or container, DDFLOW_NO_UPDATE_CHECK=1: then no request is made at all. In a container the image registry tag is not consulted; the package index answers for every install.

Upgrading ddflow itself. ddflow upgrade --self [--to VERSION] [--dry-run] (MCP ddflow_upgrade with self = run, dry-run or rollback, and to) installs the newer ddflow with the tool that installed this one: uv tool upgrade ddflow-mcp for a uv tool, pipx upgrade ddflow-mcp for pipx, that interpreter's own pip install -U for a virtualenv (--to 0.1.10 pins a version with uv tool install --force / pipx install --force / pip install ddflow-mcp==...). Where ddflow cannot do it safely it does NOT try: an ephemeral uvx environment, a container image, a source checkout, an install this user cannot write, a system pip or a VCS install each print the exact command for the operator (pull the image and restart; git pull; reinstall) and exit 2. --dry-run names the command and runs nothing.

Installing new code is the operator's decision ([upgrade].self): ask (default) runs the installer only with --yes --reason WHY (MCP: confirm: ["self"] and reason), which is recorded in the upgrade.applied event (category self, from and to); without it the call is refused (exit 3) and shows the command it would run. operator refuses an agent over MCP altogether; auto asks for no consent but still needs --self. The installer runs as a subprocess with a 10-minute bound and its output captured; a failed or timed-out installer leaves the old version installed and working and says so (exit 1). The running process never replaces its own code mid-command: it finishes, says RESTART needed (an MCP server also says it in its replies), and the NEW code, in a fresh process, shows ddflow upgrade --plan (and under [upgrade].auto = safe applies hooks and instructions after a backup, which re-points a hook or MCP entry that pinned the old path). The version it replaced is remembered in the user's state directory ($XDG_STATE_HOME/ddflow/self-upgrade.json, default ~/.local/state/ddflow) because it belongs to the install, not to a project; ddflow upgrade --self --rollback reinstalls it from any directory, with the same consent.

Undo and downgrade. The log is append-only, so an upgrade is never undone by deleting events. ddflow upgrade --restore [NAME] puts back the files a backup holds (the default local backup, or the opt-in git snapshot tag: see "Backups: local or snapshot" above), and git revert of the commit that holds the upgrade's changes does the same for committed files. Going back to an older ddflow binary is a different matter: once a newer ddflow has stamped the log, the older one reads it but its writes are refused with the version to upgrade to (the skew guard, below), unless the operator passes --allow-older-version for that session.

Two versions at once. Teammates on different ddflow versions share one log. Each agent appends to its own shard under .ddflow/events/, the shards are merged by union (merge=union in .gitattributes), and the version stamp is an event like any other, so the highest stamp wins on every clone: after the first upgraded clone's shard arrives, an older ddflow can still read and ddflow upgrade --plan, but its writes are refused until it is upgraded. Two clones that each ran --apply before merging both appended an upgrade.applied event; the union keeps both, every step was idempotent, and the plan on the merged log is empty.

The version stamp and the skew guard

A project's log records which ddflow versions have worked on it, so an upgrade, or a checkout running an older ddflow than its teammates, is a fact instead of a guess (decisions D-upgrade-event-kinds and D-upgrade-skew-guard). Five event kinds, all skipped with a note by a ddflow that predates them:

The fold keeps the highest version stamped (State.ddflow_versions; ddflow status --json shows it in its ddflow_version field, and ddflow doctor names it when there is a skew); an older ddflow stamping later does not lower it. A machine-local, git-ignored marker .ddflow/local/seen.json holds the last version this machine acted under. It works the same on a fresh ddflow init and on a project that predates the stamp: an old log with no stamp reads clean and is stamped on its next write.

The skew guard. A write by a ddflow older than the log's highest stamp is refused (exit 3) before anything is written:

REFUSED: this project's log has been worked on by ddflow 0.2.0 (stamped by alice), and
this ddflow is 0.1.9, which is older: writing now could drop or misread what the newer
one recorded. Upgrade ddflow-mcp to >= 0.2.0 (`uv tool upgrade ddflow-mcp`, ...), then
restart the MCP server and retry ...

The remedy fits how this ddflow is installed (config_sections._compat.upgrade_advice): an installed ddflow is told to upgrade its package, a source checkout to merge main into it (Merge main into this checkout ..., then restart the MCP server (ddflow >= 0.2.0 is needed) and retry). The rest of the message does not change.

Reads are always allowed. The agent upgrades or asks the user; only if the user insists does it rerun with an explicit override: --allow-older-version --reason "<why>" on any CLI command, or the allow_older_version argument (the reason) on any MCP tool. That writes a skew.overridden event and marks every event of that session with the older version (older_ddflow in its data): ddflow history shows [older ddflow 0.1.9], ddflow replay says so, and ddflow doctor lists the overrides and how many events they cover, to be reviewed after upgrading. The override is per session, not per command: a new session, another agent, or a newer stamp is refused again. Only versions that ship the guard can refuse; releases before it cannot.