💥 agent-crashtest

Security for AI agents in your CI.

Claude Code, Codex and other coding agents now run in GitHub Actions on issues, PR comments and fork PRs, with your API keys and a write token in the environment. One wrong line in the workflow, and anyone on GitHub can open an issue that tells your agent what to do.

agent-crashtest finds those workflows and blocks them in review, before they reach main.

# .github/workflows/agent-crashtest.yml
name: agent-crashtest
on: [pull_request]
permissions:
  contents: read
jobs:
  scan:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: glibey/agent-crashtest@v1

Findings show up as annotations on the PR diff and as a table in the job summary, and the check fails on high or worse.

What a risky agent workflow looks like

on:
  issues:
    types: [opened]                         # anyone on GitHub can open an issue
jobs:
  triage:
    runs-on: ubuntu-latest
    steps:
      - uses: anthropics/claude-code-action@v1
        with:
          anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
          allowed_non_write_users: "*"      # turns off the write-access check
          claude_args: --allowedTools "Bash,Read"   # any shell command

 CRITICAL  Anyone on GitHub can drive the agent in .github/workflows/triage.yml
 CRITICAL  Agent may run any shell command in .github/workflows/triage.yml

The issue text becomes the agent's instructions, and the agent can run any command next to your secrets. Narrow the tools to what the job needs (--allowedTools "Bash(gh issue edit:*),Read") and it drops to the low-risk setup the action's own security docs recommend.

Related MCP server: CodeAudit MCP

What it checks in CI

It understands the guards people actually use, so it doesn't cry wolf: author_association and role allowlists, gate jobs, maintainer-only labels, merged-PR-only triggers, and schedule-only workflows.

Your local agent setup too

The same scanner checks the agent on your machine: Claude Code, Codex, Cursor, MCP servers and instruction files.

npx agent-crashtest          # scan this project and your global agent config
npx agent-crashtest --fix    # apply the safe fixes, each shown as a diff first

--fix changes only what has one right answer: it denies agent reads of secret files, pins npx MCP servers to the current version, and removes enableAllProjectMcpServers and skipDangerousModePermissionPrompt. Files in your home directory are asked about separately and never changed with --yes.

Inside your agent

Let the agent scan, explain each finding for your project, and fix it one change at a time with your OK.

• Claude Code (skill, plus a warning at session start when something is critical): /plugin install agent-crashtest --marketplace glibey/agent-crashtest

• Codex, Cursor, Copilot, Gemini CLI and other agents with Agent Skills support: npx skills add glibey/agent-crashtest

• Any MCP client (a read-only scan tool):

{
  "mcpServers": {
    "agent-crashtest": { "command": "npx", "args": ["-y", "agent-crashtest@0.5.2", "mcp"] }
  }
}

For Codex, in ~/.codex/config.toml:

[mcp_servers.agent-crashtest]
command = "npx"
args = ["-y", "agent-crashtest@0.5.2", "mcp"]

For teams

Running AI agents in CI across an organization? We're building a Team version:

• a GitHub App for private repos that reviews every PR touching an agent workflow or agent config

• one policy for what agents may do in CI across all repos

• an inventory of every workflow that runs an AI agent, and who can trigger it

👉 Join the early access list, free during the beta.

Accuracy

A security check that cries wolf gets removed from CI. The rules were run against thousands of public repositories that run AI agents in GitHub Actions or commit agent config, and every critical finding was reviewed by hand. Patterns that turned out safe, such as allowlist-gated and schedule-only workflows, Bash(npx eslint:*), curl localhost and defensive rules in CLAUDE.md, are regression tests now.

Found a false positive or a missed risk? Open an issue.

Privacy and trust

• The scan runs locally or on your runner. No network calls, no telemetry. Secret values are never printed; .env files are read for variable names only.

• The Action builds from the ref you pin and never downloads the package at run time.

• No runtime dependencies. npm releases are published from GitHub Actions with provenance.

Badge

npx agent-crashtest --badge

prints a snapshot badge with your grade. For one that stays current, use the workflow badge of the Action; the job summary prints the exact Markdown.

Options

npx agent-crashtest [options]
npx agent-crashtest mcp           run as an MCP server over stdio

--dir <path>        project to scan (default: current directory)
--project-only      skip global config in your home directory
--fix               show safe fixes as diffs and apply them after you confirm
--yes               with --fix: apply project changes without asking (never touches files in ~)
--badge             print a README badge with your grade
--json              machine-readable output
--fail-on <level>   exit 1 on findings at or above: critical, high, medium, low
--ci                same as --fail-on high
--github            GitHub Actions mode: PR annotations, job summary, step outputs
--demo              scan a deliberately vulnerable sample project

Action inputs: path (default .), fail-on (default high). Outputs: score, grade, critical, high.

Roadmap

• GitHub App for private repos (early access)

• Policy file (.agent-crashtest.yml): declare which workflows may run agents, with which tools, enforced in CI

• More CI agents: GitHub Copilot coding agent, Gemini CLI action, GitLab CI

• --fix for CI workflows: narrow tools and drop outsider access in agent workflows

Ideas and requests: open an issue.

MIT