Behaviorlock

Fails your CI when an agent's recorded behavior drifts after a model, prompt, memory, or tool change.

Behaviorlock compares two runs your harness already recorded, a baseline and a candidate, against a contract of behaviors that must stay stable: tool sequences, permission decisions, output structure, outcome verdicts, sets, ranks, and bounded numeric metrics. It is for teams that change the model, prompt, memory, policy, or tools under an existing agent. It does not call a model, judge prose, or inspect hidden reasoning.

If you maintain skills without writing code: Behaviorlock does not read your prompts or judge your texts. It compares values your tooling already records before and after a change — and tells you which of the ones you declared important moved further than you allow. Example: a newsletter skill after a model update. Your harness records the length, whether the call to action is present and which salutation was used. The contract says: length may vary by 20 %, the call to action must stay, the salutation must be one of the allowed forms. Behaviorlock reports each of these as compatible, drifted or unknown — in plain sentences you wrote yourself, such as "The call to action is still present" or "The permission decision does not weaken after upgrade." Behaviorlock does not interpret those sentences; the check itself is a short JSON rule next to each one (see The contract below).

representative scenarios
          │
          ├── baseline.trace.json  (model A / prompt v3)
          └── candidate.trace.json (model B / prompt v4)
                          │
                          ▼
                  behaviorlock.json
             selectors + deterministic matchers
                          │
                          ▼
      compatible · drifted · unknown + CI gate
                          │
                          ▼
        JSON · Markdown · HTML · SARIF · JUnit

Try it in two minutes

Behaviorlock is not on npm. The commands below use the package attached to the v0.1.0 GitHub release. Requires Node.js 20 or newer.

Claude Code (stdio MCP server):

claude mcp add behaviorlock -- npx -y https://github.com/christian140903-sudo/behaviorlock/releases/download/v0.1.0/behaviorlock-0.1.0.tgz serve

Any MCP client:

{
  "mcpServers": {
    "behaviorlock": {
      "command": "npx",
      "args": [
        "-y",
        "https://github.com/christian140903-sudo/behaviorlock/releases/download/v0.1.0/behaviorlock-0.1.0.tgz",
        "serve"
      ],
      "env": {
        "BEHAVIORLOCK_CONTRACT": "/absolute/path/to/behaviorlock.json"
      }
    }
  }
}

BEHAVIORLOCK_CONTRACT is optional. The server offers the prompt gate-agent-upgrade, which walks the agent through defining scenarios, recording a baseline, and comparing a candidate with behaviorlock_compare. The tools read the file paths the client passes to them.

See a pass and a drift on the bundled example

git clone https://github.com/christian140903-sudo/behaviorlock.git && cd behaviorlock
npm ci && npm run build
node dist/src/index.js compare \
  examples/baseline.trace.json \
  examples/candidate.trace.json \
  examples/behaviorlock.json --no-write

Behaviorlock PASS — Support agent upgrade
model-a / prompt-v3 → model-b / prompt-v4
6/7 compatible · 0 drifted · 1 unknown
Scenarios: 2 compatible · 0 drifted · 1 unknown

Between the two runs the candidate tightened a permission (ask → deny), answered in Markdown instead of JSON (both allowed), called the same two tools in a different order, and got 18 % slower (budget: 25 %). None of that breaks the contract. One scenario exists only in the candidate; it is reported as unknown, not as compatible. The default gate passes because every error assertion is compatible; --strict also requires the warning and info assertions and therefore fails on that unknown.

Now weaken the permission instead and run the same check:

sed 's/"deny"/"allow"/' examples/candidate.trace.json > drifted.trace.json
node dist/src/index.js compare \
  examples/baseline.trace.json drifted.trace.json examples/behaviorlock.json --no-write
echo "exit code: $?"

Behaviorlock FAIL — Support agent upgrade
model-a / prompt-v3 → model-b / prompt-v4
5/7 compatible · 1 drifted · 1 unknown
Scenarios: 1 compatible · 1 drifted · 1 unknown
exit code: 1

node dist/src/index.js explain permission-not-weaker examples/baseline.trace.json drifted.trace.json examples/behaviorlock.json prints the one assertion with both selected values (ask, allow) and the reason: Candidate rank is worse than baseline.

Without --no-write, compare also writes all five report formats to .behaviorlock/ next to the contract.

To get a behaviorlock command without a clone: npm install -g https://github.com/christian140903-sudo/behaviorlock/releases/download/v0.1.0/behaviorlock-0.1.0.tgz.

Related MCP server: Thread Contract MCP Server

Why it exists

When you swap the model under an agent, rewrite its prompt, or add a tool, most of its behavior may change, but some of it must not: it still asks before deleting, still calls tools in the approved order, still returns a format your code can parse. A manual spot check covers some of that once; Behaviorlock repeats the same declared checks on every change.

A golden-file diff of the two bundled traces flags every difference: the stricter permission, the new format, the reordered tools, the latency change, and the new scenario. It treats ask → deny (stricter, fine) and ask → allow (weaker, not fine) the same way: as a difference. Behaviorlock checks only what the contract declares, in the direction the contract gives.

Model-evaluation platforms are useful when a team wants to run providers, score semantic quality, or use an LLM judge. Behaviorlock owns a smaller layer: given two already-recorded runs, did the declared observable behavior remain compatible? In practice that means:

• no provider API keys, model adapters, prompts, or network calls;

• no judge model that can change the final answer;

• no hidden chain-of-thought capture;

• no arbitrary shell execution;

• the same JSON inputs always produce the same statuses and fingerprints;

• a new scenario without a baseline is unknown, not silently compatible.

Behaviorlock or agent-invariants? Both compare a baseline with a candidate, and both are mine. agent-invariants checks rules over an ordered event stream inside a run (approval before a payment tool, no work after a stop, retry limits). Behaviorlock compares values you chose to record (a decision, a tool list, a verdict, a number) between two runs. Use agent-invariants when the rule is about the order of events; use Behaviorlock when the question is whether a recorded value moved in a direction you declared unacceptable.

How it works

The portable trace

Any framework can emit the trace. Behaviorlock only requires scenario status and JSON observations:

{
  "$schema": "https://raw.githubusercontent.com/christian140903-sudo/behaviorlock/main/trace.schema.json",
  "traceVersion": 1,
  "run": { "id": "candidate-001", "candidate": "model-b / prompt-v4" },
  "scenarios": [
    {
      "id": "destructive-action",
      "status": "completed",
      "observations": {
        "permission": { "decision": "deny" },
        "tools": ["request_permission", "delete_item", "verify_absence"],
        "outcome": { "verdict": "satisfied" }
      }
    }
  ]
}

Trace metadata is excluded from the behavior fingerprint. Scenario order is normalized; array order inside observations remains behavior and is preserved.

Review and redact traces before storing them. Behaviorlock deliberately does not collect provider transcripts for you.

The contract

{
  "$schema": "https://raw.githubusercontent.com/christian140903-sudo/behaviorlock/main/behaviorlock.schema.json",
  "schemaVersion": 1,
  "project": { "name": "support-agent" },
  "scenarios": [
    {
      "id": "destructive-action",
      "assertions": [
        {
          "id": "permission-not-weaker",
          "statement": "The permission decision does not weaken after upgrade.",
          "severity": "error",
          "selector": "/observations/permission/decision",
          "matcher": {
            "op": "rank_not_lower",
            "order": ["allow", "ask", "deny"]
          },
          "limitations": [
            "This compares recorded decisions; it does not prove every destructive prompt was tested."
          ]
        }
      ]
    }
  ]
}

Selectors are RFC 6901 JSON Pointers evaluated against the whole scenario, so contracts can observe /status as well as /observations/....

Deterministic matchers

Relational matchers return unknown when the baseline selector is absent. Type mismatches that make a comparison undefined also return unknown.

CLI

behaviorlock init
behaviorlock validate behaviorlock.json baseline.json candidate.json
behaviorlock fingerprint candidate.json
behaviorlock compare baseline.json candidate.json behaviorlock.json
behaviorlock compare baseline.json candidate.json behaviorlock.json --strict
behaviorlock compare baseline.json candidate.json behaviorlock.json \
  --formats json,markdown,html,sarif,junit --out artifacts
behaviorlock explain permission-not-weaker baseline.json candidate.json behaviorlock.json

From a clone, use node dist/src/index.js in place of behaviorlock.

Exit codes:

• 0: every error assertion is compatible;

• 1: required behavior drifted or is unknown;

• 2: invalid input or runtime failure.

Reports and CI

• JSON carries the complete machine-readable comparison and report digest.

• Markdown is designed for upgrade review and pull requests.

• HTML is standalone, escaped, and marked noindex.

• SARIF lists drifted and unknown assertions as findings.

• JUnit maps drift to failures and unknowns to skipped tests.

A CI step in your own project, using the release package:

- run: >
    npx -y https://github.com/christian140903-sudo/behaviorlock/releases/download/v0.1.0/behaviorlock-0.1.0.tgz
    compare baseline.trace.json candidate.trace.json behaviorlock.json
    --formats markdown,sarif,junit --out behaviorlock-reports

The step fails when required behavior drifted or is unknown (exit code 1) and on invalid input (exit code 2).

MCP server

The stdio server (see Try it in two minutes) exposes five tools:

• behaviorlock_validate

• behaviorlock_compare

• behaviorlock_explain

• behaviorlock_fingerprint

• behaviorlock_render

It also exposes the contract schema, trace schema, bundled example, and the gate-agent-upgrade prompt. From a clone, you can point a client at node /absolute/path/to/behaviorlock/dist/src/index.js serve after npm ci && npm run build.

TypeScript API

After npm install https://github.com/christian140903-sudo/behaviorlock/releases/download/v0.1.0/behaviorlock-0.1.0.tgz:

import { compareBehavior, renderReport } from 'behaviorlock';

const report = await compareBehavior(
  './baseline.trace.json',
  './candidate.trace.json',
  './behaviorlock.json',
);

console.log(report.summary.gatePassed);
console.log(renderReport(report, 'markdown'));

Verify it yourself

git clone https://github.com/christian140903-sudo/behaviorlock.git && cd behaviorlock
npm ci && npm test        # expected: 73 passing (last verified 2026-10-08, Node 22, fresh clone)

What the 73 tests cover: all nine matchers including their unknown cases and the default and strict gates; JSON Pointer selection and schema validation of contracts and traces; the comparison runtime, fingerprints, and all five report formats; the CLI commands with exit codes 0, 1 and 2; and the stdio MCP server end to end (tools, resources, prompt, a comparison over stdio).

What they do not cover: no real agent or model run (all traces are small hand-written fixtures); no CLI test with a drifted error assertion (drift is tested at the function level; the drift example above runs it through the CLI); SARIF and JUnit output are checked for content, not validated against the official schemas or uploaded to a code-scanning service; CI runs on Linux only (Node 20, 22, 24); no tests with large traces.

The drift example in Try it in two minutes triggers the failure case on your machine. Two more checks:

npm run test:coverage   # same tests with Node's coverage report
npm run smoke:pack      # packs the package, installs it in a temp project, runs the CLI and imports the SDK

What it does not do

• It does not record traces or run your agent. Your harness writes the JSON, and Behaviorlock trusts it: it does not attest that a trace came from a particular model, prompt, harness, or production run.

• It does not judge quality. No semantic equivalence, factual truth, safety, fairness, or hidden reasoning. A passing gate covers only the scenarios and assertions in the contract; missing scenarios and weak selectors create false confidence.

• It does not prevent drift; it reports it. Whether a drifted run blocks a release depends on how you wire the exit code into CI.

• Numbers are single values, not benchmarks. number_delta compares one recorded value per selector, with no statistics.

• Fingerprints are content digests, not signatures or provenance.

• Known issue in 0.1.0: compare --strict prints PASS in the summary line when only the default gate passed, but correctly exits with 1. Rely on the exit code.

Details: limitations, security model, contract reference.

How this was built

Most of the code was written by AI coding agents under my direction. I wrote the specification, set the constraints, decided what to test, reviewed the result and rejected what did not hold. Release decisions and every claim in this README are mine.

The public history starts at the finished 0.1.0 code (one commit on 2026-07-17); the agent sessions behind it are not published. The idea comes from a private agent system of mine; this repository extracts the general mechanism without that system's rules, memory, or logs (ORIGINS.md).

Status

0.1.0 as a GitHub release (2026-07-17), not on npm · experimental · single maintainer · no known external users · last verified 2026-10-08.

Referenced by MediaPlural/agent-proven: its agent-regression skill points to Behaviorlock for deterministic trace comparison. That is a pointer in another project's documentation, not a usage report.

Version 0.1.x is an early public schema; incompatible changes will be documented before a stable release. Please report security issues through the private process in SECURITY.md.

License

MIT © Christian Bucher