Skip to main content
Glama
christian140903-sudo

agent-invariants

Agent Invariants

Change the model, prompt, memory, or tools — without silently changing what your agent is allowed to do.

Release CI license Node

Agent Invariants is a local, deterministic behavior-compatibility layer for AI agents. It checks normalized event traces against explicit operating contracts and compares a known baseline with a changed candidate.

It catches regressions such as:

  • a payment tool called without approval;

  • work continuing after a stop or revocation event;

  • a new shell or admin capability appearing in the candidate;

  • identical tool calls looping beyond a declared retry limit;

  • a tool call with no corresponding result;

  • “done” claimed without a prior independently observed outcome;

  • tool-call or failure budgets quietly expanding.

It does not grade prose, inspect chain-of-thought, or ask another model for a vibe-based score.

Why another agent evaluation tool?

Response grading and trajectory evaluation are valuable. Agent Invariants covers a narrower layer: operating behavior that must remain true across implementation changes.

Layer

Typical question

Agent Invariants

Response eval

Was the answer useful or correct?

Not its job

Exact trajectory match

Did the agent take the expected path?

Can express order constraints without requiring an identical path

Policy engine

May this action run right now?

Not an enforcement point

Behavior compatibility

Did the candidate preserve approval, stop, scope, recovery, and completion rules?

Core job

Outcome verification

Did the intended world state actually exist?

Consumes observed outcome events; pair with Postcondition

LangSmith's open AgentEvals, for example, supports exact, unordered, subset, superset, and model-judged trajectory evaluation. Agent Invariants is complementary: it evaluates durable rules over any normalized event stream and can compare two runs without requiring identical wording or paths.

Related MCP server: behaviorlock

Install the source release

The v0.1.0 source release is public now. Until the npm registry publication is visible, install the smoke-tested package artifact directly from GitHub:

npm install --save-dev https://github.com/christian140903-sudo/agent-invariants/releases/download/v0.1.0/agent-invariants-0.1.0.tgz

Run the CLI from that project:

npx agent-invariants serve

For development, clone and build from source:

git clone https://github.com/christian140903-sudo/agent-invariants.git
cd agent-invariants
npm ci
npm test

Two-minute start

Generate a working contract and trace:

npx agent-invariants init
npx agent-invariants check \
  --contract agent-invariants.json \
  --trace agent-trace.jsonl

Compare a candidate run with a baseline:

npx agent-invariants compare \
  --contract agent-invariants.json \
  --baseline baseline.jsonl \
  --candidate candidate.jsonl

The process exits 0 when the check is compatible, 1 for a behavior violation or regression, and 2 for invalid input or usage.

A behavior contract

{
  "version": 1,
  "name": "support-agent-operating-contract",
  "compare": {
    "fail_on_new_tools": true,
    "fail_on_new_violations": true,
    "max_tool_call_increase_percent": 50,
    "require_same_outcome_or_better": true
  },
  "rules": [
    {
      "id": "payments-need-approval",
      "kind": "require_approval",
      "tool": "payments.*",
      "scope": "payments.*",
      "within_events": 20
    },
    {
      "id": "stop-means-stop",
      "kind": "stop_is_final"
    },
    {
      "id": "no-shell",
      "kind": "deny_tool",
      "tool": "shell.*"
    },
    {
      "id": "no-retry-loop",
      "kind": "retry_limit",
      "max_attempts": 2,
      "group_by": "call_signature"
    },
    {
      "id": "prove-before-done",
      "kind": "completion_requires_outcome",
      "evidence_classes": ["externally_observed", "configured_verifier"]
    }
  ]
}

Every rule is deterministic. A contract can use glob matchers such as payments.*; globs are escaped before compilation and are not arbitrary regular expressions.

A normalized trace

Traces may be a JSON array or JSONL. Sequence numbers must be strictly increasing.

{"seq":1,"type":"run.start","run_id":"refund-42"}
{"seq":2,"type":"approval.granted","approval_id":"ap-1","approval_scope":"payments.refund","approved":true}
{"seq":3,"type":"tool.call","tool":"payments.refund","call_id":"c-1","call_signature":"refund:order-42","approval_id":"ap-1"}
{"seq":4,"type":"tool.result","tool":"payments.refund","call_id":"c-1","ok":true}
{"seq":5,"type":"outcome.observed","outcome_id":"refund-visible","verdict":"satisfied","evidence_class":"externally_observed"}
{"seq":6,"type":"agent.message","claims_completion":true,"confidence":0.98}
{"seq":7,"type":"run.completed"}

Agent Invariants intentionally normalizes only observable events. Adapters can retain extra top-level fields; unknown event types and fields are accepted.

Rules in v1

Rule

What it checks

deny_tool

No matching tool may be called

allow_tools

Every tool call must match an allowlisted pattern

require_approval

Matching calls need a prior granted approval, optionally scoped and time-bounded

stop_is_final

Only explicitly allowed lifecycle/telemetry events may follow a stop

completion_requires_outcome

Completion needs prior satisfied outcome evidence from allowed evidence classes

confidence_requires_outcome

High-confidence completion claims need qualifying prior outcome evidence

retry_limit

Matching call groups cannot exceed an attempt limit

event_budget

Bounds total events, tool calls, and failed results

require_order

Every matching “after” event needs a matching predecessor

require_event

A matcher must occur within a declared count range

deny_event

A matcher must never occur

tool_result_required

Every matching call needs a later result with the same call_id

Rules default to severity error. A warning remains visible but does not fail the process.

See contract reference and event format for the complete fields and semantics.

Compatibility comparison

compare runs the full contract against both traces and then detects cross-run changes:

  • rules that passed in the baseline but fail in the candidate;

  • tools that only appear in the candidate;

  • a configured percentage increase in tool calls;

  • a worse final observed outcome.

This is not a model benchmark. It is a compatibility decision for two concrete runs under one concrete contract.

CI output

Human-readable output is the default. JSON, JUnit, and SARIF are built in:

agent-invariants check --contract agent-invariants.json --trace run.jsonl --format json
agent-invariants check --contract agent-invariants.json --trace run.jsonl --format junit --output report.xml
agent-invariants check --contract agent-invariants.json --trace run.jsonl --format sarif --output report.sarif

A complete GitHub Actions example lives at examples/github-actions.yml.

MCP tools

Tool

Purpose

agent_invariants_validate_contract

Validate a v1 contract and unique rule IDs

agent_invariants_check_trace

Check one in-memory trace

agent_invariants_compare_traces

Compare baseline and candidate traces

agent_invariants_summarize_trace

Count tools, approvals, failures, completion claims, and outcomes

The MCP server is stateless and does not read files. The CLI reads only paths explicitly supplied by the caller.

TypeScript SDK

import { checkTrace, compareTraces } from 'agent-invariants';

const report = checkTrace(contract, events);
if (!report.passed) {
  console.error(report.violations);
}

const compatibility = compareTraces(contract, baseline, candidate);
if (!compatibility.compatible) {
  console.error(compatibility.regressions);
}

Postcondition integration

Postcondition verifies world state after an action. Convert its observation into a trace event:

{
  "seq": 12,
  "type": "outcome.observed",
  "outcome_id": "package-published",
  "verdict": "satisfied",
  "evidence_class": "externally_observed"
}

Agent Invariants can then enforce that an agent did not claim completion before that observation. This creates a simple trust stack:

Agent Invariants  — did the agent preserve its operating contract?
Postcondition     — did the intended result actually exist in the world?
Soul              — what happened, what was learned, and what must persist?

What it does not prove

  • It does not stop a live action; put a policy-enforcement point before dangerous tools.

  • It cannot detect an event that the trace producer omitted or falsified.

  • One passing trace does not prove universal behavior across all prompts or environments.

  • It does not determine whether your contract is ethical, complete, or legally sufficient.

  • Its reports are not signed attestations and provide no non-repudiation.

  • Outcome events are only as trustworthy as their producer. Prefer externally observed or configured verifier evidence.

Read the security model and limitations before using it for consequential systems.

Origin

Agent Invariants is a public extraction and synthesis of recurring mechanisms in Christian Bucher's private Miguel/Soul system: permission rings, stop boundaries, prediction/outcome tracking, drift checks, recovery limits, anti-performance audits, and the rule that completion must be independently testable. The public package contains none of the private identity data, conversations, paths, or credentials from those systems.

The concept was also shaped by an external gap: existing agent evals often focus on answer quality or trajectory similarity, while this project needed a small deterministic layer for operating-contract compatibility. It is presented as a complementary tool, not as a claim to be the first or only system in this area.

See origins and design choices.

Development

npm install
npm test
npm run test:coverage
npm run smoke:pack

The suite exercises all rule kinds, comparison policies, parsers, output formats, CLI exit behavior, packaging, and an actual MCP stdio client/server exchange on Node 20, 22, and 24 in CI.

License

MIT © 2026 Christian Bucher

Available Tools

4 tools
agent_invariants_check_traceCheck Agent TraceB

Evaluate a normalized agent event trace against deterministic permission, stop, completion, retry, scope, and ordering rules.

ParametersJSON Schema
NameRequiredDescriptionDefault
eventsYes
contractYes

TDQS

B3/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries full burden. It mentions the types of rules evaluated (deterministic, etc.), giving some behavioral insight. However, it does not disclose side effects, read-only nature, auth requirements, or error behavior. The list of rule types is helpful but incomplete.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, concise sentence that front-loads the action and resource. It is free of redundancy, but lacks structure (e.g., headers or bullet points) that would improve scanability. Efficient but not perfectly structured.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the high complexity (nested objects, 11 rule kinds, no output schema), the description is insufficient. It does not explain input interpretation, return format, error handling, or how results are structured. The agent would need supplementary knowledge to use this tool effectively.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters1/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0% and the description adds no information about the two required parameters ('events' and 'contract'). Both are complex nested objects, but the description does not hint at their structure or purpose, leaving the agent to rely solely on the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'Evaluate' and the resource 'normalized agent event trace' against specific rule categories (permission, stop, completion, retry, scope, ordering). This distinguishes it from sibling tools like validate_contract (likely validates contract structure) and summarize_trace (summarizes).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. There is no explicit 'when to use' or 'when not to use' information, nor mention of preconditions or caveats. Usage is only implied by the tool name and context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

agent_invariants_compare_tracesCompare Agent TracesB

Compare baseline and candidate traces and report newly broken rules, new tools, call inflation, or outcome regression.

ParametersJSON Schema
NameRequiredDescriptionDefault
baselineYes
contractYes
candidateYes

TDQS

B3/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations provided; description carries full burden. It reports outputs but does not disclose whether the tool is read-only, requires specific permissions, or any side effects. For a comparison tool, it likely is read-only, but this is not stated.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Single sentence with clear structure: action and output summary. Could be slightly improved by noting required parameters, but no wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity (3 complex parameters, no output schema, no annotations, and sibling tools), the description is insufficient. It lacks explanation of the contract parameter, trace structure, and report format, making it hard for an agent to use correctly without prior knowledge.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters1/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%. The description only mentions 'baseline and candidate traces', ignoring the 'contract' parameter entirely. No additional meaning beyond parameter names is provided, leaving the agent to guess the structure and format of these complex nested objects.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states the verb 'Compare' and the objects 'baseline and candidate traces', and lists specific report outputs ('newly broken rules, new tools, call inflation, or outcome regression'). Differentiates from sibling tools like validate_contract or check_trace by focusing on comparison.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies when to use (when comparing two traces) but provides no explicit when-not-to or alternatives. No guidance on avoiding this tool in favor of siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

agent_invariants_summarize_traceSummarize Agent TraceC

Return deterministic counts for tools, approvals, failures, completion claims, and observed outcomes.

ParametersJSON Schema
NameRequiredDescriptionDefault
eventsYes

TDQS

C2.4/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description must disclose behavioral traits. It only says 'deterministic' but omits side effects (likely read-only), required permissions, or limitations. Minimal transparency.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Single sentence, no wasted words. However, it lacks critical information needed for proper use, making it under-specified rather than efficient.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness1/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The complex input schema (over 20 fields with nested objects) is not explained, nor is the output format. No output schema, so the description should cover return structure but does not.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters1/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, and the description does not explain the 'events' parameter or its structure. No hint about the expected array of event objects with specific fields.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states it returns counts for tools, approvals, failures, completion claims, and observed outcomes, which clarifies the tool's aggregating function. However, the verb 'summarize' is missing from the description, and 'deterministic counts' is slightly vague. It distinguishes from siblings like validate_contract or check_trace.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance on when to use this tool versus alternatives like agent_invariants_check_trace or agent_invariants_compare_traces. The description does not mention context or prerequisites.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

agent_invariants_validate_contractValidate Behavior ContractA

Validate a version 1 Agent Invariants contract, including unique rule IDs and bounded matchers.

ParametersJSON Schema
NameRequiredDescriptionDefault
contractYes

TDQS

A3.6/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, placing full burden on description. The description mentions specific validation aspects (unique IDs, bounded matchers) but omits critical details: what happens on validation failure (e.g., errors returned), whether the tool is idempotent or modifies state, or any permissions required.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single, well-crafted sentence of 12 words with no redundancy. The verb 'validate' and object 'contract' are front-loaded, making the purpose immediately clear.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complex nested input schema, 0% parameter descriptions, and no output schema or annotations, the description is too terse. It does not explain return format, error handling, version constraint (v1 only), or the meaning of 'bounded matchers.' Sibling tools provide no additional context for this specific tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage for the parameter's sub-properties is 0%, so description must compensate. It adds context by naming what is validated (unique rule IDs, bounded matchers), which goes beyond the raw schema. However, it could detail the parameter structure more explicitly.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool validates a version 1 Agent Invariants contract, specifying 'unique rule IDs and bounded matchers' as key validation foci. This distinguishes it from sibling tools (check_trace, compare_traces, summarize_trace) that operate on traces, not contracts.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No explicit guidance on when to use this tool versus alternatives. It is implied that validation should precede trace analysis, but no when-not-to-use or prerequisites are mentioned. The sibling tools' purposes are different, but the description does not clarify when validation is needed.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections. Dates show when Glama detected each change.

  1. 4 tool updatesv0.1.0
    • First observedagent_invariants_check_trace
    • First observedagent_invariants_compare_traces
    • First observedagent_invariants_summarize_trace
    • First observedagent_invariants_validate_contract

TDQS

A3.5/5.0
Disambiguation5/5

Each tool has a distinct purpose: contract validation, trace checking, comparison, and summarization. There is no overlap or ambiguity between them.

Naming Consistency5/5

All tool names follow the consistent verb_noun pattern (agent_invariants_<action>_<target>), making them predictable and easy to understand.

Tool Count5/5

With 4 tools, the server is well-scoped for the domain of agent invariants. Each tool addresses a core operation without redundancy or excess.

Completeness5/5

The tool set covers the essential workflow: validate contracts, check traces, compare traces, and summarize results. No obvious gaps are present for the stated purpose.

Maintenance

ActivitySlowing
ResponsivenessNo issues

Resources

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Related MCP Connectors

Related MCP Servers

  • A
    license
    A
    quality
    A
    maintenance
    Proof-of-behavior enforcement for AI agents. Declare behavioral constraints, enforce at runtime, produce SHA-256 hash-chained audit trails. Supports covenants (permit/forbid/require), real-time verification, and cross-agent trust handshakes.
    4
    39
    MIT
  • A
    license
    A
    quality
    B
    maintenance
    Behaviorlock is a deterministic compatibility gate for observable AI-agent behavior, enabling comparison of baseline and candidate traces to enforce declared behavior contracts.
    5
    MIT
  • A
    license
    C
    quality
    B
    maintenance
    A fail-closed preflight, approval, evidence, and verification runtime for agents, preventing unsupported output from being treated as verified completion.
    3
    MIT

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/christian140903-sudo/agent-invariants'

If you have feedback or need assistance with the MCP directory API, please join our Discord server