nightward
OfficialAn MCP server that lets an AI agent run nightward’s regression gate and inspect its boundary verdict, but never approve or reject changes.
nightward_runruns pytest, captures behaviors, and returns the boundary signal after code edits.The signal reports whether the boundary is
intact,breached,partial,incomplete,stale, orunknown, with changes grouped by name/kind/group.It includes diffs of moved behaviors to help distinguish intended changes from regressions; reading them does not mark them reviewed.
It returns warnings, pytest output tail, not-run/incomplete details, stale status, capture coverage, scrub-rule diagnostics, semantic-judge details, and environment mismatch warnings.
nightward_statusreads the last run’s verdict and diffs without running tests.It cannot approve changes; approval remains a human CLI action.
It gates only the store it was started with and refuses other stores;
pathandtimeoutcan constrain the run.
Provides a GitHub Action that runs the nightward gate in CI, writes verdicts and diffs to the job summary, and fails unless the boundary is intact.
Integrates as a pytest plugin and runs pytest capture suites, exposing a behavior fixture to capture behavior values for regression gating.
nightward
A regression gate for AI-made changes. Your AI agent can run the gate; opening it takes a human-approved baseline diff.

When an AI agent fixes A, it often moves B, C and D too, and the test suite stays green because no test pins down B, C or D. You find the side effects later, fix them, and the fixes move something else. An agent loop has no reliable way to know when it is done.
nightward gives that loop a definition of "done" that the agent can check but not change:
Your tests capture what the code does today with a
behavior(...)pytest fixture. You don't write expected values.A human reviews those captures and approves them once. That baseline is a set of JSON files committed next to your code.
After every change, nightward captures again and lists everything that moved, grouped by feature (the blast radius). The gate stays closed until a human approves each change as intended or the code is fixed.
The agent gets run and status, on the command line or as MCP tools. Approving
is a human CLI command, and its result is a file diff that goes through code review.
nightward run # capture behaviors and compare them with the approved baseline
nightward review # a human reads the diffs
nightward approve # the human accepts what review showed (intended changes only)
nightward gate # exit 0 only when nothing unapproved remains (CI, agent loops)nightward is not a test generator, and it has no idea what is correct. It only makes sure that no change to a captured behavior passes silently.
Compared with | How nightward differs |
NL→test generators | nightward gates changes, it doesn't author tests |
snapshot libraries (syrupy, inline-snapshot, ApprovalTests) | approval is a separate, review-gated step that never removes anything in bulk, and one run lists every moved behavior by feature with a verdict an agent can read. Detailed comparison |
ordinary regression tests | no hand-written expected values, and a semantic judge can gate nondeterministic AI text |
What's new in 0.3, and what changes for 0.2 users: see the CHANGELOG.
How a project uses it
Setup happens once. After that, every change goes around the loop, and only a human can take an "approve" edge:
flowchart TD
subgraph setup["Once per project"]
A["Write capture tests<br/>behavior(name, value, group=)"] --> B["nightward run"]
B --> C{"A human reads the first capture<br/>nightward review"}
C -- "a value is wrong" --> D["Fix the code"] --> B
C -- "this is what it should do" --> E["nightward approve<br/>commit .nightward/baseline/"]
end
E --> F
subgraph loop["Every change, by an agent or a human"]
F["Edit the code"] --> G["nightward run<br/>(MCP: nightward_run)"]
G --> H{"boundary"}
H -- "intact" --> I["Done: open the PR"]
H -- "breached, not intended" --> F
H -- "breached, intended" --> J["The agent stops and lists the changes"]
J --> K{"A human runs nightward review"}
K -- "approve" --> L["Baseline diff in the PR,<br/>reviewed by code owners"]
K -- "reject: a regression" --> F
end
I --> M["CI: nightward gate<br/>exit 0 only when intact"]
L --> MThe same loop as a timeline, for an agent change that moves something it shouldn't:
sequenceDiagram
actor Human
participant Agent
participant NW as nightward
participant CI
Human->>Agent: "add a coupon to checkout"
Agent->>Agent: edits the code
Agent->>NW: nightward_run
NW-->>Agent: breached: checkout_total (intended), loyalty_points (not intended)
Agent->>Agent: fixes the loyalty side effect
Agent->>NW: nightward_run
NW-->>Agent: breached: checkout_total only
Agent-->>Human: "checkout_total changed as asked - please review"
Human->>NW: nightward review, then nightward approve
Human->>CI: push (the baseline diff is in the PR)
CI->>NW: nightward gate
NW-->>CI: exit 0 (intact)Related MCP server: agent-invariants
Install
nightward needs Python 3.10+. Installing it also registers its pytest plugin, so
the behavior fixture is available in every test.
pip install nightward
pip install "nightward[mcp]" # + MCP server for AI agentsOther extras: [judge] (the Anthropic SDK, for the LLM judge) and [docs] (the
PDF, Word and Excel adapters).
Quickstart (5 minutes)
In an empty directory, create a tiny app and a test that captures its behavior.
app.py:
def checkout_total(items):
subtotal = sum(i["price"] * i["qty"] for i in items)
tax = round(subtotal * 0.1, 2)
return {"subtotal": subtotal, "tax": tax, "total": round(subtotal + tax, 2)}
def user_login(username):
return {"user": username, "status": "ok", "roles": ["member"]}test_app.py writes no expected values. It captures what the code returns
(for anything bigger than this, shape the payload so a human can review it:
payload design):
from app import checkout_total, user_login
CART = [{"price": 10.0, "qty": 2}, {"price": 5.5, "qty": 1}]
def test_checkout(behavior):
behavior("checkout_total", checkout_total(CART), group="billing")
def test_login(behavior):
behavior("user_login", user_login("alice"), group="auth")A test that captures one value can return it instead. @capture names the
behavior after the test (or name=), and each parametrized case keeps its id:
from nightward import capture
@capture(name="checkout_total", group="billing")
def test_checkout():
return checkout_total(CART)1. Approve the current behavior as the baseline.
git init # skip in an existing repo
nightward init # creates .nightward/ and git-ignores its per-run files
nightward run . # runs pytest and captures; lists what moved (names only)
nightward review # shows each diff; approve covers only what review showed
nightward approve --all
git add .gitignore .nightward # commit baseline/, rejected/, judge/ (init ignores per-run files)
git add app.py test_app.py && git commit -m "nightward baseline"run finds two NEW behaviors, review prints their JSON, and approve makes them
the baseline:
Boundary: breached (2 unapproved)
unchanged=0 changed=0 new=2 removed=0
...
approved checkout_total (c9ff6ed9)
approved user_login (b86be5a5)
Boundary: intact
unchanged=2 changed=0 new=0 removed=02. Make the kind of change an agent makes. In app.py, change the tax rate
0.1 to 0.2 (in bash: sed -i 's/subtotal \* 0.1/subtotal * 0.2/' app.py).
Then:
nightward run . # the tests still pass
nightward gate # exit code 1: not done
nightward status --json # the same verdict, for a script or an agent
nightward approve --all # refused: nobody has reviewed this change yet
nightward review # the diff a human decides onBoundary: breached (1 unapproved)
unchanged=1 changed=1 new=0 removed=0
group: billing
- [CHANGED] checkout_total
boundary breached (1 unapproved) (as of the last run, ...)
{"boundary": "breached", "unapproved": 1, "changes": [{"name": "checkout_total", "kind": "CHANGED", "group": "billing"}], ...}
error: checkout_total changed since your review (the capture now in the store was taken by `nightward run` at ...). Run `nightward review checkout_total`, then approve what it shows.
[CHANGED] checkout_total
--- approved
+++ received
@@ -1,5 +1,5 @@
{
"subtotal": 25.5,
- "tax": 2.55,
- "total": 28.05
+ "tax": 5.1,
+ "total": 30.6
}3. Decide. If the change is a regression, fix the code: put 0.1 back, run
nightward run . again, and it reports Boundary: intact, so nightward gate
exits 0. If the change is intended, run nightward approve --all (it now
succeeds, because review showed you this diff) and commit the updated
.nightward/baseline/checkout_total.approved.json.
4. Optional: see the blast radius in a browser. nightward view builds a
static, read-only dashboard and serves it on localhost. nightward view --no-serve only builds nightward-site/, which init already git-ignored.
For a bigger cascade (one tax tweak moves three customer-facing behaviors), run the petshop example from a clone of this repo.
What to commit
path | commit? | what it is |
| yes | the approved behaviors: the boundary |
| yes | changes a human confirmed as regressions |
| yes | the semantic judge's rulings (review them like the baseline) |
| no | per-run state, ignored by |
| no | the dashboard build: it contains your captured data |
In CI: the GitHub Action
Set up Python and your project's dependencies, then gate the PR with one step.
It runs your tests, writes the verdict and the diffs that moved to the job
summary, and fails the job unless the boundary is intact. It never approves:
intended changes are approved locally (nightward review, nightward approve)
and arrive in the PR as a baseline diff.
- uses: actions/checkout@v7
- uses: actions/setup-python@v7
with:
python-version: "3.12"
- run: pip install -e . # your project and its test dependencies
- uses: LunenLabs/nightward@v0.3.5
with:
path: tests # what pytest runs; default: [tool.nightward] path, else "."Inputs: path, dir (default .nightward), pytest-args (one per line),
allow-not-run, version, install, summary; output: boundary. A full
workflow with the dashboard artifact is in the
CI recipe.
Commands
command | what it does |
| create the store and the |
| run pytest, capture, compute the blast radius. Exit 1 if a capture test failed (the run is incomplete) |
| show the diffs, or with |
| promote reviewed NEW/CHANGED behaviors into the baseline. Bulk approval never removes anything |
| drop REMOVED behaviors: removal is always this explicit act |
| confirm a reviewed change as a regression. The gate stays breached |
| the verdict as an exit code: 0 intact; 1 breached, partial, stale or incomplete; 2 no store, no report, or an error. |
| the verdict with the change list. |
| explain what moved, and suggest scrub rules only for values that are volatile by evidence |
| the verdict from a capture that |
| build and serve the read-only dashboard |
| stdio MCP server for AI agents: run and status, never approve |
| the installed version |
gate and status report the last run, so run nightward run again after
every code edit (or use nightward gate --run). The full reference (removals, stale reports, merge conflicts,
locking, and using the plugin inside an existing pytest job) is in
docs/commands.md.
AI agents (nightward mcp)
An agent loop needs a definition of "done" that it can check but not change.
nightward mcp is a stdio MCP server with two
tools: nightward_run (run the tests, capture, recompute the verdict) and
nightward_status (read the last verdict). approve and reject are not
exposed: if the agent that makes a change could also approve it, the gate would
turn into a changelog. The judge and the store are the humans' committed choices,
so the agent can't pick a lenient judge or point the server at another store.
In Claude Code, install the plugin. It adds the MCP server and a skill that
tells Claude to run the gate after its edits and to hand intended changes to
you instead of approving them. A second skill, onboard, puts new code under the
gate: Claude reads the code without changing it, writes a
domain map
and the capture tests, and hands you the suspected defects and the first review. nightward itself must be installed in the
environment you start claude from (your project's virtualenv):
pip install "nightward[mcp]" # in the project's environment/plugin install nightward --marketplace LunenLabs/nightward/nightward:status then shows the verdict and what you need to review. Any other
MCP host can run the server directly:
claude mcp add nightward -- nightward mcp # or your host's equivalent; run it in the project rootBoth tools return the status --json shape. The agent is done only when
boundary is intact and stale is false:
| meaning | what the agent should do |
| no unapproved change | done |
| unapproved changes | fix the code, or stop and ask a human to approve |
| nothing unapproved among what ran, but approved behaviors were not checked (deselected, see | run them: remove the |
| nothing unapproved, but capture tests failed or errored | fix the failing tests (see |
| the baseline, capture or rejections moved since the report | call |
| no report yet | call |
Setup for other MCP hosts, every argument and returned field, and the rules for the loop are in docs/mcp.md.
Gating AI text: the semantic judge
Free-text AI output changes its wording on every run, so an exact comparison
breaches every time. Mark such a behavior semantic=True and commit a judge in
pyproject.toml: [tool.nightward] judge = "anthropic:claude-haiku-5-5". The
key-free persona:* judges are deterministic stand-ins for tests and CI: they
forgive changes of case, whitespace and punctuation in prose, not a reworded
sentence, so reworded AI text needs the LLM judge. The judge only decides whether two outputs mean the
same thing; approval stays human. Every ruling is recorded in the committed
ledger .nightward/judge/, and a judge that can't rule falls back to the exact
comparison. Details are in
docs/semantic-judge.md.
How it compares with snapshot testing
nightward is snapshot testing underneath: capture, approve, compare. If you already use syrupy, inline-snapshot or ApprovalTests and people make your changes, they may be all you need. nightward is built for the loop where an agent makes the change:
snapshot libraries | nightward | |
a mismatch | fails the test; a test stops at its first failing assertion | is recorded; one run lists every moved behavior, grouped by feature |
accepting changes | an update flag rewrites snapshots (syrupy's |
|
a machine verdict | the pytest exit code |
|
reworded AI text | a mismatch every run | optional semantic judge with a committed ledger |
Where they're better: inline-snapshot keeps the value next to the assertion, syrupy has serializers for many types and needs no extra command, and ApprovalTests exists for many languages. They can run side by side with nightward in one suite.
What "intact" means
"Boundary intact" means no captured behavior changed, nothing more:
The gate sees only what
behavior()captures, so capture the behaviors you care about.[tool.nightward] coverage = "src/app"lists the public functions no capture test ran.The gate is a convention, not a sandbox. An agent with a shell could run
approveor edit the baseline, but every bypass leaves a trace in git. Make that trace need a human: own the store with CODEOWNERS, require code-owner review onmain, and let CI run the gate from source (recipe).A semantic judge can be wrong, so its rulings are recorded for review.
Normalization defines "the same": timestamps and UUIDs are masked by default (opt out with
scrub=False), and dict key order is ignored.
Read the threat model and what nightward normalizes before you rely on a green gate.
Documentation
CHANGELOG: what's new in 0.3.0, and upgrading from 0.2
Commands and workflow: every command, removals, stale reports, merge conflicts, and the plugin in an existing pytest job
Designing payloads a human can review: aggregate by cause, record each input once, keep the machine out - read it before your first capture
What nightward normalizes: scrubbing, custom scrub rules, names, non-JSON values, and
doctorReview depth: per group, how much a review shows and when the agent asks - never whether a human approves
Domain maps: the rules each behavior guards, shown next to its diff; the onboard skill writes them
Document & artifact inputs: PDF, Word, Excel, text and raw bytes
Recipe: make "only a human approves" hold on GitHub: CODEOWNERS for the store, code-owner review, and an agent identity that isn't an owner
Scope
In scope: pytest capture, the blast-radius diff, the gate, the loop signal,
scrubbing and doctor, the static dashboard, the MCP agent gate (run and status,
never approve), and the semantic judge. Out for now: PR-comment summaries,
call-graph grouping, and other languages. See CONTRIBUTING.md.
MIT licensed.
Available Tools
2 toolsnightward_runA
Run the tests, capture behaviors, and return the boundary signal.
Call this after every code edit: nightward_status only reports the last run.
path: what pytest runs (relative to the server's working directory); omit it to
run the project's capture suite ([tool.nightward] path, else ".");
dir: omit it - the server gates the store a human started it with
(`nightward mcp --dir`), and any other store is refused; timeout: seconds
before pytest is stopped (the store is then left untouched and the last
report invalidated).
Returns {boundary, unapproved, changes: [{name, kind, group, judged,
judge_model, judge_reason, judge_replayed, rejected, rejected_by, depth,
ask_human, diff, diff_cut}], judged_same,
not_run (approved behaviors whose test was deselected: not checked), narrowed,
stale, incomplete, generated_at, judge, scrub_rules (custom scrub rules of the run
and what they replaced), env_mismatch (changed behaviors approved on another OS or
Python than this run, or null: tell the human - it never makes a breach
acceptable), capture_coverage ({roots, functions, executed, uncovered}: public functions
no capture test ran, when [tool.nightward] coverage is set, else null - a change
there can't breach the gate; never part of boundary), store (absolute path of the gated
store), path (what pytest ran), warnings: {skipped, failed, errors, deselected,
xfailed, scrubbed,
scrub_unmatched (custom scrub rules that matched nothing), scrub_rules (per custom
rule: values and behaviors it replaced, and whether that changed since the last run),
judge_added (judge ruling files this run added to the store's judge/ dir - a human
commits them with the baseline), pytest_returncode, pytest_output_tail}}.
boundary is one of:
"intact" done: no unapproved change;
"breached" unapproved changes: fix the code, or ask a human to approve;
"partial" nothing unapproved among what ran, but approved behaviors were
not checked (their test was deselected, see not_run): not done -
run them (remove the -k/-m selection); you can't waive this;
"incomplete" nothing unapproved, but capture tests failed or errored
(see incomplete and pytest_output_tail): fix the tests;
"stale" the baseline, capture or rejections moved since the report: run again;
"unknown" no report yet: run.
diff is what moved (old "-", new "+"), cut to 40 lines (diff_cut: lines left out;
diff is null once 400 lines are shown): read it to tell an intended change
from a regression. Reading it marks nothing reviewed; don't run `nightward
review` or `nightward view` for the diffs - they record that a human saw them.
Done means boundary == "intact" and stale is false. Behaviors approved with
semantic=True are judged by the judge the humans committed
([tool.nightward] judge in pyproject.toml, or `nightward mcp --judge`);
"judge" says which, and why it was unavailable if it was. This tool cannot
approve changes: a human does that with the nightward CLI.
Inputs: dir, path, timeout.
nightward_statusA
Read the boundary signal of the LAST nightward_run, without running tests.
It does not see code edits made since that run: after changing code, call
nightward_run for a fresh verdict. Same shape as nightward_run (diffs
included), without warnings. boundary is "intact" (done), "breached" (unapproved changes),
"partial" (approved behaviors not checked: their test was deselected),
"incomplete" (capture tests failed or errored; see incomplete), "stale"
(the baseline, capture or rejections moved since that run, so its verdict can't be
trusted: run again) or "unknown" (no report yet). generated_at says when
the run happened; store names the store read (dir: omit it, as for
nightward_run).
Inputs: dir.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
TDQS
Scored across 2 tools
The two tools have clearly distinct purposes: nightward_run executes tests and produces a fresh boundary verdict, while nightward_status only reads the last run's report. The descriptions explicitly clarify the relationship and warn that status cannot see edits made after the last run. There is no meaningful overlap or risk of misselection.
Both tools share the consistent nightward_ prefix and snake_case formatting, making them clearly part of the same server. However, nightward_run uses a verb while nightward_status uses a noun, so the pattern is consistent in namespace but not strictly verb_noun throughout.
Two tools is on the thin side, but each is essential to the narrow agent-facing purpose: run the gate and read the most recent result. The server deliberately avoids duplicating human-only CLI operations, so the small surface is reasonable rather than excessive.
For the stated MCP purpose, the surface covers the full agent workflow: trigger a fresh test/gate run, then read the last boundary report without rerunning. Approval and review remain intentionally human-only CLI actions, so their absence is not a gap in this server's role.
Maintenance
Related MCP Connectors
Git-native policy layer for AI agents: check_action verdicts against rules approved via PR.
Synthetic checks, nightly regression replay and model-drift alerts for AI agents
Browser-local and CLI static evidence for deployed AI model artifacts.
Checks AI-written code for leaked keys and made-up packages, on your machine. Never uploads code.
Related MCP Servers
- AlicenseCqualityDmaintenanceEnables acceptance gates for AI coding-agent runs by recording evidence, running deterministic validation, applying a quality gate, and rendering auditable outcomes.7Apache 2.0
- AlicenseAqualityBmaintenanceA deterministic behavior-compatibility layer for AI agents that checks normalized event traces against operating contracts and compares baselines with candidates to catch regressions in approval, stop, scope, recovery, and completion rules.4MIT
- AlicenseNot gradedqualityBmaintenanceEnables deterministic behavioral regression review of pull-request diffs by checking changed code against approved incident-derived contracts. It exposes read-only tools to assess diffs and pull requests, list and fetch contracts, and report outcomes over stdio without mutating repositories.MIT
- AlicenseNot gradedqualityBmaintenanceEnables AI coding assistants to scan a repository for code that cannot prove its usefulness, attribute it to AI-assisted or human commits, prove candidate deletions against the project's own tests, and pull a payoff report or run history, all without ever writing to the repository. The test command and any model judging stay under a person's control through environment configuration.1MIT