Skip to main content
Glama
io.github.justrunme/agent-security-contract

io.github.justrunme/agent-security-contract

by justrunme

agent-security-contract

CI License

Fail-closed security contract for AI agents and MCP tool calls.

ASC is a platform gate: given an approved policy and a tool-call trace, it emits a CI-grade evidence envelope with verdict ALLOW | BLOCK | REQUIRE_APPROVAL. It is backed by an executable abuse-case corpus. Numbers in docs/EVIDENCE.md are generated from results/results.json — never hand-typed.

Claims are scoped: metrics and pass/fail statements apply on this corpus of N scenarios only. They do not imply general agent safety.

Maintainer site: https://justrunme.com

Why

Agent and MCP stacks expand the attack surface beyond prompt text: tool misuse, poisoned tool descriptions, argument injection, and unapproved high-impact actions. ASC turns an explicit capability contract into a repeatable pre-execution CI / admission check.

Aligned with (complementary, not a fork):

Related MCP server: @palveron/mcp-server

Positioning (honest)

ASC’s differentiator is pre-execution ENFORCEMENT, not detection: the gate prevents the tool call (or requires approval) and proves the decision in CI via an evidence envelope.

Project

Primary job

When it runs

Relation to ASC

ASC (this repo)

Fail-closed policy gate + evidence envelope

Before tool execution / in CI admission

Enforcement oracle

Promptfoo (MCP red teaming)

Attack generation & detection findings in CI

Eval / red-team jobs

Complementary detection; does not replace an admission deny

OWASP Agent Security Regression Harness

Regression suite for agent security behaviors

CI regression

Complementary testing; ASC can act as an enforcement profile

LiteLLM / guardrails

Request/response filtering & provider controls

Proxy / runtime path

Complementary runtime filters; ASC focuses on tool-capability contracts + digest pinning

Quickstart

Requires Python 3.11+.

python -m pip install -e ".[dev]"

# Evaluate one trace — prints a full evidence envelope with --json
python -m asc check \
  --policy policy/demo-agent.policy.yaml \
  --trace corpus/scenarios/benign_read_workspace_file.json \
  --json

# Verify an envelope (rejects invalid/tampered envelopes)
python -m asc check \
  --policy policy/demo-agent.policy.yaml \
  --trace corpus/scenarios/benign_read_workspace_file.json \
  --json --envelope-out /tmp/env.json
python -m asc verify --envelope /tmp/env.json --policy policy/demo-agent.policy.yaml

# Run the full corpus and refresh evidence
python -m asc corpus \
  --policy policy/demo-agent.policy.yaml \
  --corpus corpus/scenarios \
  --results results/results.json \
  --summary results/summary.md \
  --evidence docs/EVIDENCE.md

Exit codes (asc check)

Verdict

Exit code

ACS disposition

ALLOW

0

ALLOW

BLOCK

1

DENY

REQUIRE_APPROVAL

2

ASK

Reusable GitHub Action

- uses: justrunme/agent-security-contract@v0.2.0
  with:
    policy: policy/demo-agent.policy.yaml
    corpus: corpus/scenarios
    results: results/results.json
    summary: results/summary.md
    evidence: docs/EVIDENCE.md

Runtime integration

Call the evaluator before tool execution (library or HTTP sidecar). See docs/INTEGRATION.md. On sidecar/client timeout, fail closed (BLOCK); asc.client.evaluate_remote() does this for you.

python -m asc.sidecar --policy policy/demo-agent.policy.yaml --port 8788

MCP server

ASC exposes its policy engine as a Model Context Protocol server. MCP clients (Claude Desktop, Cursor, etc.) can call ASC tools to enforce security contracts before executing tool calls.

Install

pip install agent-security-contract

Or from source:

python -m pip install -e ".[dev]"

Configure for Claude Desktop

Add to ~/Library/Application Support/Claude/claude_desktop_config.json (macOS) or %APPDATA%\Claude\claude_desktop_config.json (Windows):

{
  "mcpServers": {
    "agent-security-contract": {
      "command": "asc-mcp"
    }
  }
}

Configure for Cursor

Add to Cursor MCP settings (Settings → Features → MCP Servers):

{
  "mcpServers": {
    "agent-security-contract": {
      "command": "asc-mcp"
    }
  }
}

Run standalone (stdio)

asc-mcp

Run as streamable HTTP server

asc-mcp --transport streamable-http --host 127.0.0.1 --port 8765

Available tools

  • check_tool_call: Check a proposed tool call against a policy contract. Returns PASS/BLOCK/REQUIRE_APPROVAL verdict with reasons. Fail-closed: any error yields BLOCK.

  • verify_corpus: Run the full abuse-case corpus and return metrics (critical false-ALLOW, benign false-BLOCK rate, pass/fail).

  • verify_scenario: Verify a single corpus scenario file against a policy.

  • list_policy_tools: List loaded policies, capability catalog, digest, and per-agent tool allowlists.

All tools are read-only (they inspect policy and traces but do not modify state).

External auth on Kubernetes

Envoy ext_authz in front of an MCP server, calling the real ASC decision API (kind + script): examples/external-auth/README.md. Integration evidence: docs/EVIDENCE-EXTERNAL-AUTH.md (on this corpus).

Policy contract

YAML policy (policy/demo-agent.policy.yaml) declares:

  • Capability catalog — tool name, description, input schema

  • Capability digest — sha256: over the canonical catalog; must match the approved digest

  • Per-agent allowlists — which tools an agent may call

  • Argument constraints — type, pattern, deny patterns, allowlisted hosts, max length, and per-arg required (default true); undeclared args are rejected; omitted optional args are allowed

  • Path canonicalization — path args are percent-decoded, NUL-checked, slash-collapsed, and ..-resolved before allowlist checks

  • High-impact tools — require approved: true on the call or the verdict is REQUIRE_APPROVAL; destructive paths that resolve to a parent/root still BLOCK

  • Rate limits — max calls per trace and per tool

Evidence envelope

Every verdict emits JSON conforming to schemas/evidence-envelope.schema.json:

Field

Meaning

schema_version

Envelope schema (1.0.0)

policy_version

Policy document version

policy_sha256

SHA-256 of the policy file bytes

scenario_id

Corpus / caller scenario id

verdict / reason

Decision

trace_id

Trace identifier

commit_sha

ASC_COMMIT_SHA / GITHUB_SHA / unknown (CI sets ASC_COMMIT_SHA to the PR head or push SHA)

timestamp

UTC ISO-8601

envelope_sha256

SHA-256 over the canonical JSON of all other fields

asc verify rejects missing fields, invalid values, disposition/verdict inconsistency, any edit that no longer matches envelope_sha256, and (when --policy is set) a policy_sha256 that differs from the policy file.

envelope_sha256 is an unkeyed integrity digest, not a signature: it catches accidental or naive edits, but someone who can rewrite the file can also recompute it. Treat envelopes as trustworthy only when they come from a CI run you control (for example, the uploaded workflow artifact).

Corpus metrics (generated)

See docs/EVIDENCE.md for the current run. Metrics are always phrased on this corpus of N scenarios.

Positive coverage is first-class: legitimate ALLOW scenarios are at least as numerous as attack scenarios; benign false-BLOCK is reported next to critical false-ALLOW.

Threat-model limits (honest)

ASC evaluates static traces (and pre-execution hook payloads) against a declared policy. It is not:

  • a signer: evidence envelopes carry an integrity digest but no signature or attestation yet

  • a full ACS wire Guardian (no session audit chain, no MODIFY/DEFER dispositions yet)

  • a runtime OS sandbox or network enforcer by itself

  • a prompt-injection detector inside the model

  • a replacement for identity, secrets, or production change-management controls

  • a claim of general agent safety beyond this corpus of N scenarios

Use it as a CI / admission gate alongside runtime controls, Promptfoo-style red teaming, and ACS-compliant Guardians.

Upstream

Possible OWASP upstream contributions: docs/UPSTREAM.md.

Development

python -m pip install -e ".[dev]"
python -m ruff check src tests
python -m pytest --collect-only -q
python -m pytest -q

CI (.github/workflows/ci.yml) runs a clean-install pytest --collect-only gate, lint, unit tests (including fail-closed and envelope negatives), the full corpus, writes results/results.json + markdown summary as artifacts, regenerates docs/EVIDENCE.md, and runs the Envoy external-auth kind quickstart job. Workflow permissions: contents: read; Actions pinned by commit SHA.

Research and book

The preprint Fail-Closed Authorization for Tool-Using AI Agents: A Capability Contract and Case Study (Andrey Lesnikov, Zenodo, 2026) describes the contract and its evaluation. Additional context and analysis: Fail-Closed Authorization for AI Agents.

The book Prove It Before Production: Evidence-Driven Infrastructure Engineering for Kubernetes and AI (Amazon author page) is about proving that an infrastructure change is safe before it reaches production: reconstruct live state, rehearse the change, emit an explicit ALLOW / BLOCK / REQUIRE_APPROVAL verdict, and fail closed when evidence is missing (Chapter 7). Part IV (Chapters 11-14) applies the same contracts to AI agents on the control plane, including agent security contracts (Chapter 12), inference budgets and GPU admission.

Live demo: Hugging Face Space

Author

Andrey Lesnikov — Senior DevOps & AI Platform Engineer, independent researcher in fail-closed authorization for AI agents. 3rd place, Cloud Native Rockstar Awards 2026 (Cloud Native Conference 2026, Frankfurt).

Links: justrunme.com • GitHub • ORCID

Security

See SECURITY.md.

Available Tools

4 tools
check_tool_callA

Check a proposed tool call against an ASC policy contract.

Returns PASS/BLOCK/REQUIRE_APPROVAL verdict with reasons. Fail-closed: any error, missing policy, or unknown input yields BLOCK. Use this tool before executing any tool call to enforce a security contract.

Args: policy_path: Path to the ASC policy YAML file trace_data: Tool call trace mapping with agent_id, trace_id, calls, optional tool_catalog scenario_id: Identifier for this check (for reporting)

Returns: Mapping with verdict, reason, scenario_id, blocked_call_id, and details

Inputs: trace_data (required), policy_path (required), scenario_id. Returns: object

list_policy_toolsB

List loaded policies and their configured tools/rules.

Returns the capability catalog, capability digest, and per-agent tool allowlists from an ASC policy file.

Args: policy_path: Path to the ASC policy YAML file

Returns: Mapping with version, capability_digest, capabilities, and agents configuration

Inputs: policy_path (required). Returns: object

verify_corpusA

Verify/run the abuse-case corpus against an ASC policy.

Runs all scenario JSON files in the corpus directory and returns metrics including critical false-ALLOW count, benign false-BLOCK rate, and per-scenario results.

Args: policy_path: Path to the ASC policy YAML file corpus_dir: Directory containing scenario JSON files output_results: If True, return full per-scenario results (may be large)

Returns: Mapping with passed (bool), metrics, and optionally scenarios list

Inputs: corpus_dir (required), policy_path (required), output_results. Returns: object

verify_scenarioA

Verify a single corpus scenario file against an ASC policy.

Loads the scenario JSON, runs the trace through the policy evaluator, and returns whether the actual verdict matches the expected verdict.

Args: policy_path: Path to the ASC policy YAML file scenario_path: Path to a single scenario JSON file

Returns: Mapping with match (bool), expected, actual, reason, and scenario metadata

Inputs: policy_path (required), scenario_path (required). Returns: object

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    TDQS

    A3.8/5.0

    Scored across 4 tools

    Disambiguation4/5

    The tools have largely distinct purposes: runtime enforcement (check_tool_call), corpus-level verification (verify_corpus), single-scenario verification (verify_scenario), and policy inspection (list_policy_tools). The boundary between check_tool_call and verify_scenario is slightly blurry because both evaluate traces against a policy, but the descriptions make the intended contexts clear.

    Naming Consistency5/5

    All tool names use a consistent snake_case verb_noun pattern: check_tool_call, verify_corpus, verify_scenario, list_policy_tools. The conventions are predictable and readable throughout.

    Tool Count5/5

    Four tools is well-scoped for a security-contract enforcement server. Each tool covers a distinct workflow: runtime checking, whole-corpus verification, single-scenario verification, and policy inspection.

    Completeness4/5

    The surface covers the core lifecycle: enforce a policy on a proposed tool call, test one scenario, test the whole corpus, and inspect configured policies/tools. A minor gap is the absence of a standalone policy-validation or policy-diff operation, but agents can work around this through the existing check and list tools.

    Maintenance

    ActivityMaintained
    ResponsivenessNo recent community issues

    Related MCP Connectors

    Related MCP Servers

    • A
      license
      Not graded
      quality
      C
      maintenance
      Policy enforcement gateway for MCP tool calls, evaluating every tool invocation against declarative YAML policies (allow/deny/escalate-to-human), generating cryptographic hash-chained audit receipts, and including built-in content safety scanning.
      2
      MIT
    • A
      license
      A
      quality
      C
      maintenance
      Provides advisory AI-governance checks for MCP hosts and coding agents, enabling policy verdicts, PII masking, and audit traces before executing tool calls.
      3
      82 npm
      MIT
    • A
      license
      Not graded
      quality
      C
      maintenance
      Enforces fine-grained, context-aware access control on MCP tool calls, with a tamper-evident, replayable audit log that records denials and verifies every decision.
      MIT
    • A
      license
      Not graded
      quality
      B
      maintenance
      Provides a governance layer for MCP servers, enforcing policy-based allow/deny/approval rules, requiring human approval for risky calls, and recording an auditable trail of every AI agent tool call.
      MIT