Fang
Provides a composite GitHub Actions integration for running Fang as a PR gate, including diff-scoped mutation testing, mutant sampling, minimum-score enforcement, and uploading HTML/JSON/SVG report artifacts.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@Fangrun diff-scoped mutation testing and show me the surviving mutants"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
Coverage tells you which lines ran. Fang tells you which lines your tests would notice.
A test suite can execute every branch, assert almost nothing, and look great in a coverage report. Fang asks a blunter question: if the code were subtly wrong, would anything fail? It answers by generating small, valid mutations of the code in your diff, running the suite against each one, and reporting the mutations that survived.
$ python -m fang run examples/shop --all
FANG · mutation report
examples/shop · all
────────────────────────────────────────────────────────────────────────────
SCORE 61.9% ██████████████░░░░░░░░ C · thin
39 killed · 24 survived
10 tests · 0.11s baseline · 0.11s/mutant · 63 mutants in scope
────────────────────────────────────────────────────────────────────────────
SURVIVORS (24) mutations your tests did not notice
shop/inventory.py:19 · Inventory.take [comparison]
- if count > available:
+ if count >= available:
`>` -> `>=` in `count > available` · Add a test at the boundary where this
comparison flips.
shop/inventory.py:22 · Inventory.take [return]
- return True
+ return None
`return True` -> `return None` · Assert the return value; calling the function
is not enough.
...
verdict thin · unittest · 10 testsWhy this exists
Mutation testing is the honest measure of a suite, but the classic tools are built for nightly jobs: they mutate an entire repository and run the suite thousands of times. That is too slow for the loop most of us actually work in.
Fang takes the opposite position:
Only the diff. It parses
git diff -U0, maps added lines onto AST nodes, and mutates nothing else.Fail fast. Pytest runs with
-x, unittest with-f. A mutant is usually killed by the first test that notices.Sample when the diff is large. Evaluate 120 mutants and report a 95% Wilson interval instead of pretending a sample is exact.
Cache aggressively. A mutant is keyed by its edit, the source file, the test suite, and the config. Reruns are nearly free.
Isolate workers. Each parallel worker gets a temporary copy of the project, mutates one file, restores it, and repeats.
Report like a product. A dark HTML report with filters, a copyable hint per survivor, an SVG badge, a PR-comment markdown file, and machine-readable JSON.
Speak MCP. Coding agents can call
fang_check_diffafter writing tests and get the survivors back as structured data.
No LLM. No API keys. No vector database. Just AST edits and test processes.
Related MCP server: CodePeel MCP Server
Quickstart
Requires Python 3.9+ and git. Nothing else.
git clone <this-repo> fang
cd fang
# Run the included shop demo (10 tests, 63 mutants, ~4-7s on 4 workers)
python -m fang run examples/shop --all
# Or install the `fang` command
python -m pip install .
# pipx install .The demo is deliberately under-tested. It scores 61.9% and leaves 24 survivors, including an untested express shipping branch, an untested exactly-at-threshold case, and a take() return value nobody asserts.
Open the live report or docs/report.html locally. The terminal output is a summary; the HTML has every mutant, filterable by status, file, function, or text.
The PR gate
python -m fang run . --diff origin/main --sample 120 --min-score 80--diff origin/mainmutates only lines added versus that revision. Untracked files count; deleted files cannot be mutated.--sample 120evaluates a deterministic sample of 120 mutants and prints the 95% Wilson interval.--min-score 80makes the process exit1when the score is below the threshold. Exit2means setup failed (no tests, red baseline, bad revision), not "tests found bugs".The HTML report and JSON are written even when the gate fails, so CI can upload them.
A green baseline is mandatory. If the suite is already red, Fang refuses to run: a failing mutant and a pre-existing failure would be indistinguishable. That is a feature.
Add it to GitHub Actions with the included composite action:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: actions/setup-python@v5
with:
python-version: "3.12"
- uses: ./
with:
path: .
diff: origin/${{ github.base_ref }}
sample: "120"
min-score: "80"
report: fang-report.html
- uses: actions/upload-artifact@v4
if: always()
with:
name: fang-report
path: |
fang-report.html
fang-badge.svg
fang-comment.mdFor a monorepo, pass --paths packages/api and --tests packages/api/tests. For an agent-written PR, --sample 60 is often enough to tell whether the new tests assert anything.
Coding agents (MCP)
Fang ships a stdio MCP server with four tools:
Tool | What it returns |
| Score, counts, and surviving mutants for the current diff, each with a hint |
| The score only, for a diff or the whole project |
| Paged survivors with file, line, operator, before/after, hint |
| Detected runner, test command, file counts, config |
Typical agent loop:
Agent edits code and adds tests.
It calls
fang_check_diff(sample: 40).Fang returns survivors such as
count > available→count >= available.The agent adds the boundary test that the hint describes.
It calls again. The survivor disappears; the score rises.
Claude Desktop / generic MCP config:
{
"mcpServers": {
"fang": {
"command": "python",
"args": ["-m", "fang", "mcp", "/absolute/path/to/project"]
}
}
}OpenCode:
{
"mcp": {
"fang": {
"type": "local",
"command": ["python", "-m", "fang", "mcp", "."]
}
}
}Run python -m fang mcp . and speak newline-delimited JSON-RPC 2.0 over stdio. The server only writes protocol messages to stdout; logs never pollute the stream.
How it works
git diff -U0
│ added line numbers
▼
AST parse ──► candidate mutations ──► compile() check
│ valid only
▼
one file spliced per mutant
│
┌────────────────────┴────────────────────┐
▼ ▼
worker copy #1 worker copy #N
mutate → run suite (-x) → restore mutate → run suite → restore
│ │
└────────────────────┬────────────────────┘
▼
killed / survived / timeout / crash
│
score, Wilson interval, HTML, JSON, MCPMutation operators are intentionally high-signal and small in number:
Operator | Example | What a survivor usually means |
|
| A boundary is untested |
|
| Only one side of a condition is exercised |
|
| The numeric result is not asserted exactly |
|
| A magic value is not observed |
|
| The return value is ignored |
|
| One branch has no test |
|
| A side effect is unasserted |
Score. detected / (detected + survived), where detected includes kills, timeouts, and mutant-induced crashes. When sampled, the report shows a 95% Wilson interval so the number is not over-claimed.
Speed. The shop demo generates 63 mutants and runs them in roughly 4-7 seconds on 4 workers. Fang's own mutation engine has 403 mutants; a 60-mutant sample runs in about 5 seconds with a 0.2s suite. The cost is fundamentally mutants in scope × suite time / workers; --diff, --sample, and fail-fast are the levers.
Cache. .fang/cache.json stores killed/survived/timeout verdicts keyed by the mutant, source hash, test-suite fingerprint, and config. Editing a test invalidates the cache. Worker failures are never cached.
Isolation. Each worker copies the project (excluding .git, virtualenvs, caches, and build output), applies one mutation, runs the suite, and restores the file. The baseline runs in the real project. Tests must be hermetic and path-relative; use --jobs 1 if they are not.
Accuracy and limits
Fang is a sharp tool, not an oracle.
Survivors are questions, not bugs. Some mutations are equivalent: the program cannot observe the difference. Changing a rounding precision from
2to3on values that are already exact is the canonical example. Read the hint, then decide.A kill can be weak. A mutant that crashes on import is "detected", but it did not exercise a meaningful assertion. The report separates timeouts and crashes; use them as a signal to inspect.
Sampling is an estimate. A 95% Wilson interval is not a guarantee that the unsampled mutants look the same. Increase
--samplefor release gates.Python only, for now. The target project must parse with the Python running Fang. Python 3.9+ is supported; code using newer syntax needs a newer interpreter.
The suite must be green and reasonably fast. Fang runs the entire suite per mutant. A 10-minute suite makes full mutation testing impractical; narrow the scope or the test command.
Tests are not mutated by default. They run the suite, but
tests/is excluded from the candidate set.--mutate-testsexists if you want to audit the tests themselves.
Self-audit
Fang was pointed at its own mutation engine:
$ python -m fang run . --all --paths fang/mutators.py \
--test-command "python -m unittest tests.test_mutators" \
--sample 60 --jobs 4
SCORE 73.3% ████████████████░░░░░░ B · decent
44 killed · 16 survived
23 tests · 0.20s baseline · sampled 60 of 403 · 95% CI 61.0%-82.9%The first run scored 48.3%. The survivors were not noise: tests never asserted the return value of the internal _add helper, never exercised visit_AnnAssign, and never checked every entry in the operator tables. Adding those tests raised the sample to 73.3%. The remaining survivors are real: return-value paths, a dont_inherit flag, and truncation widths that no test observes. The full report is in docs/self-audit.html and live on GitHub Pages.
That is the intended workflow: run Fang, read the survivors, decide which ones matter, add tests, watch the score move.
Where Fang fits
mutmut, cosmic-ray, and Stryker are mature, general-purpose mutation testers. If you want an exhaustive nightly mutation run with dozens of operators and deep framework integrations, use them.
Fang is for the inner loop and the PR gate: only the diff, a time budget, a defensible interval, and a result a human or an agent can act on in the next five minutes. It also happens to have no installation surface beyond Python itself.
Commands
Command | Purpose |
| Mutate uncommitted changes, or the whole project when clean |
| Mutate every Python file in scope |
| Mutate added lines versus a git revision |
| Mutate staged changes versus HEAD |
| Mutate a range, useful while developing a function |
| Deterministic sample with a 95% interval |
| CI gate; exit 1 below the threshold |
| HTML report (default |
| PR-comment markdown |
| SVG score badge |
| Full machine-readable report on stdout |
| Write a starter |
| Show the detected runner, test command, and counts |
| Start the MCP stdio server |
| Open the latest HTML report |
Config lives in .fang.json at the project root. CLI flags win over config; fang init writes a template.
Project layout
fang/
source.py byte-offset-aware source splicing
mutators.py AST visitors → compile-checked mutants
diffmap.py git diff -U0 → added lines
discovery.py file walking, test-runner detection
runner.py subprocess execution, timeouts, killer parsing
engine.py scope, baseline, sampling, cache, parallel workers
scoring.py mutation score + Wilson interval
term.py ANSI terminal report
report_html.py self-contained HTML, badge, markdown
mcp.py stdio JSON-RPC MCP server
cli.py commands, gates, artifacts
examples/shop/ under-tested demo project
tests/ 89 tests, stdlib unittest only
docs/ generated reports, badge, screenshot, architecture notesDevelopment
python -m unittest discover -s tests -v # 89 tests, under a minute
python -m compileall -q fangThe test suite includes a check that every import in fang/ is standard library. That check is part of the product promise, not an afterthought.
See docs/architecture.md for the data flow, caching rules, and failure modes, and AGENTS.md for the agent-facing workflow.
License
MIT. See LICENSE.
Available Tools
4 toolsfang_check_diffA
Diff-scoped mutation testing. Mutates only added Python lines versus a git revision (default HEAD), runs the test suite, and returns the mutants the tests failed to detect. Use this to verify that new tests actually bite.
| Name | Required | Description | Default |
|---|---|---|---|
| diff | No | Git revision to diff against (default: HEAD). | |
| jobs | No | Parallel workers (default: 2). | |
| root | No | Project root (default: server root). | |
| sample | No | Max mutants to evaluate (default: 40). | |
| timeout | No | Per-mutant timeout in seconds. | |
| survivor_limit | No | Max survivors returned (default: 25). |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It does disclose the core mechanics (only added lines are mutated, the test suite is executed, survivors are returned), but says nothing about cost/runtime, whether the working tree must be clean, or what side effects (temp files, test execution) occur.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with the scoping constraint, and every clause carries information an agent needs. Nothing is padded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema exists, but the description states what is returned (mutants the tests failed to detect), and all six optional params are schema-documented. It stops short of covering runtime expectations or prerequisites for a test-suite-executing tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so every parameter including defaults is already documented in the schema. The description adds no syntax or behavioral nuance beyond that, so baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource chain: mutates only added Python lines vs a git revision, runs the test suite, returns undetected mutants. The 'diff-scoped' framing implicitly separates it from fang_mutation_score (whole-project scoring), but no sibling is named explicitly.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
'Use this to verify that new tests actually bite' gives a clear motivating scenario, but there is no explicit when-not guidance and no routing to fang_mutation_score or fang_list_survivors for the whole-repo or follow-up cases.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
fang_doctorC
Inspect the project: detected test runner, Python files, test files, and config.
| Name | Required | Description | Default |
|---|---|---|---|
| root | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. 'Inspect' implies a read-only diagnostic, but it never states whether it writes anything, requires an existing config, or fails on a non-project directory. It also omits what the agent should do with the results.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
One front-loaded sentence with no filler. The colon-list structure is efficient, though the tense mixing ('Inspect' vs 'detected') reads slightly awkwardly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema exists, so the enumeration of reported items does useful work in signaling return contents. However, with no annotations and an undocumented 'root' parameter, the definition leaves key calling details unresolved.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0% and there is a single 'root' parameter the description never mentions. It does not say what root defaults to, whether it is a path or project name, or that omitting it inspects the current directory — a meaningful gap for the only input.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Names a specific verb (Inspect) and resource (the project) and enumerates what it reports: test runner, Python files, test files, config. This makes it distinguishable from the mutation-testing siblings (fang_mutation_score, fang_list_survivors), though it never explicitly contrasts itself with them.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no statement of when to call this versus fang_check_diff or fang_mutation_score, no mention of prerequisites, and no exclusions. The agent must infer the workflow position of a 'doctor' tool from its name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
fang_list_survivorsC
Run the analysis and return surviving mutants only, with file, line, operator, before/after code, and a hint. Supports offset/limit pagination.
| Name | Required | Description | Default |
|---|---|---|---|
| all | No | ||
| diff | No | ||
| jobs | No | ||
| root | No | ||
| limit | No | Default: 25. | |
| offset | No | Default: 0. | |
| sample | No | Default: 60. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full behavioral burden. It discloses that the tool runs analysis and returns survivors with specific fields and pagination, but it does not explain runtime cost, prerequisites such as a configured mutation setup, whether tests are executed, or side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loaded with the main action and output, with no filler. Its brevity is appropriate structurally, even though it leaves behavioral and parameter gaps.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 7-parameter analysis tool with no annotations and no output schema, the description is too incomplete. It partially compensates by listing returned survivor fields, but it omits key parameter semantics and behavioral context needed to invoke the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is only 43%, so the description should compensate for undocumented parameters. It mentions offset/limit pagination, which is already in the schema, but does not explain all, diff, jobs, root, or sample beyond their names and basic types.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific action and resource: run analysis and return surviving mutants only, including concrete output fields. It is clear enough to distinguish from a mutation score tool, though it does not explicitly name the sibling tools it differs from.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies this is for listing surviving mutants, but it gives no explicit when-to-use guidance, no exclusions, and no reference to alternatives such as fang_mutation_score or fang_check_diff. The agent must infer the routing from the tool name and brief purpose.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
fang_mutation_scoreA
Return only the mutation score and counts for a diff or the whole project. Cheaper than fang_check_diff when you just need the number.
| Name | Required | Description | Default |
|---|---|---|---|
| all | No | Audit the whole project instead of a diff. | |
| diff | No | Git revision (default: HEAD). | |
| jobs | No | ||
| root | No | ||
| sample | No | ||
| timeout | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It usefully discloses that it is cheaper and returns only score and counts, but it does not describe that mutation testing is computationally expensive or that parameters like jobs, sample, and timeout affect runtime and resource use. It provides some behavioral context but leaves significant operational traits unexplained.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two tight sentences with no waste. The core purpose and the cost trade-off are front-loaded, and every clause earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Because there is no output schema, the description appropriately states what is returned ('mutation score and counts'). However, for a six-parameter mutation-testing tool with no annotations, it lacks enough detail about execution cost, parameter effects, and prerequisites to be fully self-sufficient.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is only 33%, and the description only implicitly maps to the 'all' and 'diff' parameters via 'a diff or the whole project.' It says nothing about jobs, root, sample, or timeout, leaving half the parameters undocumented in both the schema and the description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb and resource ('Return only the mutation score and counts') and scope ('a diff or the whole project'). It also distinguishes itself from the sibling fang_check_diff by explaining it is the cheaper, number-only variant, so an agent can choose correctly without opening the schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives a clear selection rule: use this when you 'just need the number' and want something cheaper than fang_check_diff. It lacks explicit when-not guidance, such as what to do when you need detailed survivor information (fang_list_survivors), but the named alternative makes the main trade-off clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
4 tool updates
v0.1.0- First observed
fang_check_diff - First observed
fang_doctor - First observed
fang_list_survivors - First observed
fang_mutation_score
TDQS
Scored across 4 tools
fang_check_diff and fang_list_survivors both run analysis and return surviving/failed-to-detect mutants, making their boundary somewhat unclear despite differing detail/pagination emphasis. fang_mutation_score and fang_doctor are clearly distinct, but the core two tools overlap enough to risk misselection.
All tools use the fang_ prefix and snake_case, which is consistent and readable. The verb/noun pattern is mostly followed (check_diff, list_survivors), though mutation_score and doctor are minor deviations.
Four tools is a reasonable, focused scope for a mutation-testing server. The set is neither bloated nor extremely thin, though the check_diff/list_survivors overlap slightly weakens the case for each tool earning a fully distinct place.
The surface covers diff-scoped mutation checks, mutation scores for diff or whole project, survivor listing, and project inspection. Minor gaps exist around whole-project detailed mutant reporting or configuration, but core mutation-testing workflows are covered.
Maintenance
Related MCP Connectors
Generates unit tests for Python code with coverage before/after reports and concrete edge cases.
Change-aware CI validation and affected-test guidance for coding agents.
Paste code + tests, get a strong/weak verdict with killed/survived mutants. Free API + MCP.
Evidence-backed architecture-quality analysis for Python agent applications.
Related MCP Servers
- FlicenseNot gradedqualityDmaintenanceProvides deterministic Python code quality analysis using flake8, mypy, McCabe, and vulture, enabling LLMs to access real linting and type checking results.1-

CodePeel MCP Serverofficial
AlicenseAqualityDmaintenanceEnables AI agents to review code diffs for bugs, security issues, and bad patterns, and generate fixes.4MIT- AlicenseNot gradedqualityBmaintenanceEnables delegating coding tasks to sandboxed agents, returning changes as auditable diffs for human review before applying or discarding them.1MIT
- AlicenseNot gradedqualityBmaintenanceProvides a code-quality backend MCP tool that automatically writes and repairs pytest tests, runs them in a sandbox, validates their effectiveness via mutation testing, and surfaces suspected source bugs for human review.MIT