Rework

What it is

When an AI agent works through a task list, "I finished X" is unverifiable by default — the agent could have written a tautological test (assert True), a test that never ran, or a test that passes regardless of whether the actual logic is correct.

Rework closes that gap. It forces every completion claim through automated proof — real FAIL_TO_PASS/PASS_TO_PASS verification plus mandatory mutation-checking — or an explicit, recorded human/AI judgment call when automated proof isn't possible. Different kinds of evidence (an automated proof, a judgment call, a fallback) are never blended into one deceptively simple percentage.

It exposes exactly 3 MCP tools:

Related MCP server: AI Workbench MCP

The core loop

1. project_define(project_id, goal, dod, criteria_inputs)

Turns a vague goal into Given-When-Then acceptance criteria, but only after real interrogation:

• Every criterion must trace back to something the user actually said (grounding field) — nothing is invented from nothing.

• Every criterion is self-audited on two axes before submission: how underspecified it is, and how gameable it is. Score too high on either and it's rejected.

• If more than half the submitted criteria fail this audit, the whole batch is rejected — that signals the interrogation itself was too shallow, not that individual criteria need patching.

• Given-When-Then text is scanned for unquantified adjectives ("secure," "fast," "robust," etc.) — rejected because they're not measurable, whole-word matched so it doesn't false-positive on things like "fastener."

• Duplicate criterion IDs and empty submissions are rejected outright; malformed input produces a clean structured error, never a crash.

2. project_check(project_id, criterion_id, ...) — the load-bearing tool

Runs actual verification for one criterion, called once per criterion as real work happens:

• FAIL_TO_PASS / PASS_TO_PASS verification (SWE-bench style): a test must be shown to have failed before the fix and pass after, either via a real pre-change command re-run, or an explicit (trusted, not verified) assertion.

• Mutation checking is mandatory before a criterion's first real verification. Rework deliberately breaks the target code (flips a comparator, inverts a negation) and confirms the test actually catches the break. A test that can't catch a broken implementation isn't real verification — it might be tautological.

• Works across languages: Python (tokenize-based), JS/TS (hand-tuned tree-sitter, fully proven), and a generic tree-sitter-language-pack dispatcher for Go/Rust/Java/C/C++/C# (Java is fully proven end-to-end; the rest are structurally verified but not proof-of-execution verified — disclosed honestly, not oversold).

• Non-functional-risk gating: criteria flagged security/crypto/concurrency never pass on a plain green test alone — they require an explicit property test or human review.

• Editing the test file itself to make it pass triggers an automatic fail — the classic way agents cheat verification.

• When mutation-checking can't cleanly resolve, the tool doesn't guess — it routes to human_review or claude_review.

• claude_review is a real evaluation, not a rubber stamp. It requires an actual verdict (trustworthy / not_trustworthy) plus specific written reasoning — not just a label. A trustworthy verdict reaches its own status, verified_complete_reviewed, kept deliberately separate from automated verified_complete so a judgment call is never confused with a proof.

3. project_status(project_id)

Read-only progress reporting. percent_complete is calculated strictly from automated verified_complete criteria — never blending in verified_complete_reviewed (judgment calls) or fallback_complete (env errors, human-approved fallbacks). All buckets are reported separately. Criteria with corrupted/unrecognized status values are surfaced with a warning instead of silently vanishing from the count.

Proven on real work

• Tracked its own build, a bug-fix patch set, and a from-scratch PDF-to-Markdown converter built specifically to exercise the full loop end-to-end — including a real case where claude_review correctly flagged a test gap as not_trustworthy before it was fixed and re-verified honestly.

• A fresh integration pass plus 10 simulated projects (26 assertions total) covering mixed outcomes, gating, fallback routing, rejection boundaries, and corrupted-data handling — all passed.

Setup

python -m venv venv
venv/Scripts/activate   # or source venv/bin/activate on macOS/Linux
pip install -r requirements.txt

Register with Claude Code:

claude mcp add rework -- <path to venv python> <path to server.py>

See AGENT.md for the full operating guide — how to interrogate a goal, drive project_check, resolve mutation-check failures, and the known limitations of each language checker.

Project structure

server.py         MCP entrypoint (FastMCP), registers the 3 tools
core/             criteria schema/validation, mutation checkers (Python/JS-TS/generic), storage
tools/            project_define / project_check / project_status implementations
tests/            pytest suite
AGENT.md          operating manual for an agent driving Rework
REF SHEET- WHEN UNSURE/   original design spec this implements

Running tests

venv/Scripts/python -m pytest tests/

Author

Built and maintained by Santhosh Stalin.