Skip to main content
Glama

killdiff

Zero-config polyglot diff-only mutation tester that catches hollow AI-generated tests in seconds. CLI, stdio MCP server (verify_diff_tests), and GitHub Action with zero runtime dependencies.

License: MIT Node >=24 Zero Dependencies


Catch Hollow AI Tests in Seconds (Zero Setup)

npx killdiff --demo

Spins up a temporary git repo with a simulated refund-approval PR (src/auth.mjs + weak unit tests that pass baseline), mutates only the changed diff lines in parallel isolated sandboxes, and flags every surviving logic bug where tests failed to fail (25% score: 3 killed, 9 survived out of 12 mutants).


Related MCP server: testgen-agent

Quick Start

# Mutate only lines changed against main
npx killdiff main...HEAD

# Enforce an 80% mutation score gate in CI with Markdown PR table
npx killdiff origin/main...HEAD --max 25 --fail-under 80 --markdown

# Output SARIF 2.1.0 for GitHub Code Scanning
npx killdiff main...HEAD --sarif > killdiff.sarif

# Override test runner explicitly
npx killdiff main...HEAD --cmd "pytest -q"

Connect to Your Coding Agent (Stdio MCP)

killdiff mcp exposes verify_diff_tests so Antigravity, Claude Code, Codex, Cursor, and OpenCode can verify their own newly written tests actually kill logic mutations before handing a PR back to you.

{
  "mcpServers": {
    "killdiff": {
      "command": "npx",
      "args": ["-y", "killdiff", "mcp"]
    }
  }
}

Reusable GitHub Action

- uses: flipcyde101/killdiff@v0.1.0
  with:
    range: origin/main...HEAD
    max-mutants: 25
    fail-under: 80

How killdiff Stays Fast & Safe

  • Diff-only scope: Parses git diff --unified=0 so only executable code on changed lines is mutated (skips test files, comments, strings, imports, and type declarations).

  • Polyglot on Day 1: Mutates TypeScript/JavaScript, Python, Go, and Rust across boundary (>=/>, <=/<), equality (===/!==, ==/!=), logical (&&/||, and/or), boolean (true/false, True/False), and arithmetic (+/-) operators.

  • Isolated parallel sandboxes: Never edits your real working tree. Creates ephemeral temp workspaces symlinking/junctioning node_modules/.venv/target for fast setup and cleans up automatically.

  • Baseline & timeout guards: Verifies baseline tests pass before mutating, caps runs with --max 25, and terminates infinite-loop mutants cleanly as TIMEOUT (counted as killed).


Honest Comparison

Feature

killdiff

StrykerJS / mutmut

cargo-mutants

Static AI-slop linters

Languages

TS/JS, Python, Go, Rust

Single-language silos

Rust only

Regex/AST heuristics

Config required

Zero config (auto-detects runner)

Per-project config file

Minimal

Minimal

Closed-loop agent MCP (verify_diff_tests)

Yes

No

No

No

Mathematically proves tests fail on logic bugs

Yes

Yes

Yes

No

Where competitors win

(Lexical diff-scoped speed)

Full compiler-AST mutators across entire repositories

Deep Rust compiler integration & AST mutants

Instant millisecond feedback without running test suites


Architecture

flowchart LR
    Diff["git diff --unified=0\n(Changed lines only)"] --> Lexer["Comment & String-Aware\nPolyglot Mutator\n(TS/JS · Py · Go · Rust)"]
    Lexer --> Pool["Parallel Isolated Sandboxes\n(junction/symlink deps +\n2.5x baseline timeout)"]
    Pool --> Report["Terminal · Markdown PR · SARIF 2.1.0\n+ Stdio MCP (verify_diff_tests)"]

Local Agent Reliability Suite

  • kvdiff: Zero-code local proxy pinpointing the exact UTF-8 byte breaking your LLM prompt cache (Anthropic, OpenAI, Gemini/Antigravity, llama.cpp/vLLM).

  • boxcheck: Local DOM geometry, CSS cascade, and layout-collision oracle (CLI + MCP) over raw Node 24 WebSocket CDP.

  • killdiff (this repo): Zero-config polyglot diff-only mutation tester catching hollow AI-generated tests (CLI + MCP + GitHub Action).

License

MIT

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    Related MCP Connectors

    Related MCP Servers

    • A
      license
      Not graded
      quality
      B
      maintenance
      A six-tool stdio MCP server that lets Claude Code, Codex and other agents submit a candidate change for before/after test execution and bounded StrykerJS mutation analysis inside resource-limited offline Linux containers. It returns explicit pass, fail or inconclusive verdicts bound to exact source, dependency, policy and image hashes, plus findings for removed, skipped or excluded tests.
      1
      MIT