shipcheck

Deterministic release-quality audits, exposed as MCP tools, driven by Claude Code skills.

Your ticket tracker and your production branch slowly stop agreeing with each other. Before the release note goes out, somebody has to sit down and find the gaps. shipcheck is that somebody.

npm install
npm test
npm run audit:demo

No credentials, no network, no setup. The bundled synthetic release reproduces every failure mode the rules detect, so the third command prints a real report on a fresh clone.

What it finds

Six checks over a release snapshot, which is the issues in a release plus the merges that landed on the release branch:

Full rationale for each, and the criteria severity is assigned from, in docs/rules.md.

Related MCP server: sdlc-mcp

The part worth reading

An agent that audits your release process is itself untested, nondeterministic software. Point a model at a tracker and ask what is wrong with this release and you get a fluent answer that is right most of the time, which is the worst available failure mode: wrong often enough to matter, plausible enough that nobody checks.

So this repo is built around one question. Not how to connect a model to a tracker, but where the line sits between what must be repeatable and what must be judged, and how to enforce that line in code instead of asking a prompt nicely.

1. Rules are pure functions. (snapshot) => Finding[]. No I/O, no network, and no reads of the system clock: the observation time is a field on the snapshot. An audit of a snapshot captured in March produces the same findings today, so a finding can be reproduced and argued about rather than taken on faith. It also means the age-based rules are tested without mocking time.

2. Severity is computed. Priority is judged. The engine assigns severity from published criteria. It does not decide what the team should fix first, because that needs context no rule can see: who is on call, what ships Friday, whether this is the third week running. The agent ranks. The engine counts.

3. The agent cannot invent a finding. Not because the prompt asks it not to. Because there is no code path by which a model's output becomes a finding. An agent calls audit_release and receives findings the engine produced, typed, with evidence attached. It can read them, rank them and describe them.

That third property survives contact with the output, which is where this kind of design usually leaks. render_slack_post takes a snapshot path, not a report object. If it accepted a report, an agent could audit, quietly edit the findings, and ask for the edited version to be rendered. Re-running the audit server side closes that. The only thing an agent contributes to the rendered post is a narrative string, which the renderer labels "written by the review agent" while the findings block is labeled "computed, not generated". Whoever reads the Slack post can tell which half was computed and which half was judged.

Architecture notes, including why git rather than the tracker is the authority on what shipped, are in docs/architecture.md.

Usage

shipcheck audit --snapshot <path> [--format markdown|json|slack]
shipcheck rules [--json]

Exit codes: 0 no findings, 1 findings present, 2 usage or input error. The exit code makes it usable as a release gate in CI.

As an MCP server

npm run mcp

Four tools over stdio:

Registered in a Claude Code or Claude Desktop MCP config:

{
  "mcpServers": {
    "shipcheck": {
      "command": "npx",
      "args": ["tsx", "/absolute/path/to/shipcheck/src/mcp/server.ts"]
    }
  }
}

As a Claude Code skill

skills/release-review is the agentic layer: it runs the weekly audit, ranks findings by what they cost this team this week, drafts the Slack post, and hands it to a human to publish. The skill is explicitly forbidden from adding findings of its own, from changing a severity, and from posting to Slack itself.

That last constraint is deliberate. A human owns what appears in their team's channel.

Connecting a real tracker

The adapters in src/adapters/ are pure normalization functions, not network clients:

• jira.ts maps a Jira issue payload to the domain's ReleaseIssue

• git.ts parses git log --merges output into MergeRecord values

To wire up a live tracker, fetch the issues however you like (the Atlassian MCP server, the REST API, a CSV export), run them through fromJiraIssues, pair them with a parsed merge log, and write the result as a snapshot JSON. The audit is then the same code path the tests cover.

Keeping the fetch outside this repo is the reason the whole suite runs on a clone with no credentials, and the reason an audit is reproducible from a snapshot that can be committed, diffed and attached to an incident review.

Project layout

src/domain/      types only
src/rules/       one pure function per check, plus the registry
src/engine/      runs the registry, sorts, counts
src/adapters/    jira and git normalization, snapshot loading and validation
src/report/      markdown and Slack rendering
src/mcp/         MCP server
skills/          the Claude Code skill that drives the weekly review
fixtures/        synthetic release reproducing every failure mode
tests/           per-rule specs, engine regression net, adapter and docs specs

Adding a rule

Write the pure function, register it, test it, document it. Nothing in the engine, the renderers, the CLI or the MCP server changes. tests/docs.test.ts fails if the catalog in docs/rules.md falls behind the registry, so the documentation is a build artifact rather than a promise.

License

MIT. See LICENSE.