# shipcheck

Deterministic release-quality audits, exposed as MCP tools, driven by Claude Code skills.

Your ticket tracker and your production branch slowly stop agreeing with each other. Before
the release note goes out, somebody has to sit down and find the gaps. `shipcheck` is that
somebody.

```bash
npm install
npm test
npm run audit:demo
```

No credentials, no network, no setup. The bundled synthetic release reproduces every failure
mode the rules detect, so the third command prints a real report on a fresh clone.

## What it finds

Six checks over a release snapshot, which is the issues in a release plus the merges that
landed on the release branch:

| Rule | Severity | Fires when |
| --- | --- | --- |
| `stuck-in-ready-to-deploy` | high | Work merged to the release branch whose ticket still says it is waiting to deploy |
| `stale-resolution-after-reopen` | high | A ticket closed as "Won't Do", reopened, fixed, shipped, still resolved as "Won't Do" |
| `missing-fix-version` | medium | Work shipped with no release version, so "what was in release X" is unanswerable |
| `done-without-resolution` | medium | Done with no resolution recorded, evidence of a workflow path that skips the resolution screen |
| `merged-without-ticket` | medium | A merge reached production citing no ticket at all |
| `unreleased-done-backlog` | low | A Done column that never self-cleans |

Full rationale for each, and the criteria severity is assigned from, in
[docs/rules.md](docs/rules.md).

## The part worth reading

An agent that audits your release process is itself untested, nondeterministic software. Point
a model at a tracker and ask what is wrong with this release and you get a fluent answer that
is right most of the time, which is the worst available failure mode: wrong often enough to
matter, plausible enough that nobody checks.

So this repo is built around one question. Not how to connect a model to a tracker, but where
the line sits between what must be repeatable and what must be judged, and how to enforce that
line in code instead of asking a prompt nicely.

**1. Rules are pure functions.** `(snapshot) => Finding[]`. No I/O, no network, and no reads
of the system clock: the observation time is a field on the snapshot. An audit of a snapshot
captured in March produces the same findings today, so a finding can be reproduced and argued
about rather than taken on faith. It also means the age-based rules are tested without mocking
time.

**2. Severity is computed. Priority is judged.** The engine assigns severity from published
criteria. It does not decide what the team should fix first, because that needs context no rule
can see: who is on call, what ships Friday, whether this is the third week running. The agent
ranks. The engine counts.

**3. The agent cannot invent a finding.** Not because the prompt asks it not to. Because there
is no code path by which a model's output becomes a finding. An agent calls `audit_release` and
receives findings the engine produced, typed, with evidence attached. It can read them, rank
them and describe them.

That third property survives contact with the output, which is where this kind of design
usually leaks. `render_slack_post` takes a snapshot path, not a report object. If it accepted a
report, an agent could audit, quietly edit the findings, and ask for the edited version to be
rendered. Re-running the audit server side closes that. The only thing an agent contributes to
the rendered post is a `narrative` string, which the renderer labels "written by the review
agent" while the findings block is labeled "computed, not generated". Whoever reads the Slack
post can tell which half was computed and which half was judged.

Architecture notes, including why git rather than the tracker is the authority on what shipped,
are in [docs/architecture.md](docs/architecture.md).

## Usage

```bash
shipcheck audit --snapshot <path> [--format markdown|json|slack]
shipcheck rules [--json]
```

Exit codes: `0` no findings, `1` findings present, `2` usage or input error. The exit code
makes it usable as a release gate in CI.

### As an MCP server

```bash
npm run mcp
```

Four tools over stdio:

| Tool | Returns |
| --- | --- |
| `list_rules` | Every rule with id, title, severity and rationale |
| `explain_rule` | Full rationale for one rule and the evidence fields it emits |
| `audit_release` | The report: findings with evidence, counts by severity and by rule |
| `render_slack_post` | Slack Block Kit JSON, with an optional agent narrative clearly attributed |

Registered in a Claude Code or Claude Desktop MCP config:

```json
{
  "mcpServers": {
    "shipcheck": {
      "command": "npx",
      "args": ["tsx", "/absolute/path/to/shipcheck/src/mcp/server.ts"]
    }
  }
}
```

### As a Claude Code skill

[`skills/release-review`](skills/release-review/SKILL.md) is the agentic layer: it runs the
weekly audit, ranks findings by what they cost this team this week, drafts the Slack post, and
hands it to a human to publish. The skill is explicitly forbidden from adding findings of its
own, from changing a severity, and from posting to Slack itself.

That last constraint is deliberate. A human owns what appears in their team's channel.

## Connecting a real tracker

The adapters in `src/adapters/` are pure normalization functions, not network clients:

- `jira.ts` maps a Jira issue payload to the domain's `ReleaseIssue`
- `git.ts` parses `git log --merges` output into `MergeRecord` values

To wire up a live tracker, fetch the issues however you like (the Atlassian MCP server, the
REST API, a CSV export), run them through `fromJiraIssues`, pair them with a parsed merge log,
and write the result as a snapshot JSON. The audit is then the same code path the tests cover.

Keeping the fetch outside this repo is the reason the whole suite runs on a clone with no
credentials, and the reason an audit is reproducible from a snapshot that can be committed,
diffed and attached to an incident review.

## Project layout

```
src/domain/      types only
src/rules/       one pure function per check, plus the registry
src/engine/      runs the registry, sorts, counts
src/adapters/    jira and git normalization, snapshot loading and validation
src/report/      markdown and Slack rendering
src/mcp/         MCP server
skills/          the Claude Code skill that drives the weekly review
fixtures/        synthetic release reproducing every failure mode
tests/           per-rule specs, engine regression net, adapter and docs specs
```

## Adding a rule

Write the pure function, register it, test it, document it. Nothing in the engine, the
renderers, the CLI or the MCP server changes. `tests/docs.test.ts` fails if the catalog in
`docs/rules.md` falls behind the registry, so the documentation is a build artifact rather than
a promise.

## License

MIT. See [LICENSE](LICENSE).