verdict
It is an MCP server that gives coding agents structured, sandboxed verification feedback for Python projects.
Run
verifyto execute pytest tests affected by your working-tree diff, or explicitly scope to 'all' or a single test file.Tests run in an isolated container (podman/docker) with the worktree mounted read-only, so your host environment is never mutated.
Results are compact typed verdicts with failure fingerprints, a
preexistingflag, and aselection_notewhen impact selection falls back.Use
explain_failureto get the full traceback/output for a specific failing check on demand, instead of in the bulk summary.Use
historyto see when a failure fingerprint was first/last seen and how often, distinguishing new regressions from long-standing breakage.Run
run_checksfor ruff/mypy lint and type checks, normalized into the same verdict schema.Optional
verdict.tomlconfig supports package discovery, custom runner images, setup commands, and max-failure limits.Works with Claude Code, Cursor, or any MCP client; can be run via
uvxwithout a permanent install.
Runs pytest tests in an isolated container, with impact-based test selection and returning compact typed verdicts that include failure fingerprints and pre-existing failure flags.
Runs ruff lint checks as part of the verification workflow, normalizing results into the same verdict schema used for test failures.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@verdict verify tests affected by my latest changes"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
verdict
https://github.com/user-attachments/assets/62923912-98aa-4246-8bc8-4adf55ae3ff8
Structured, sandboxed verification feedback for coding agents.
An MCP server that replaces your agent's pytest shell-outs with something built for the agent inner loop: impact-selected tests, run in an isolated environment, returning compact typed verdicts instead of 40,000 tokens of raw runner output — with failure fingerprints that tell the agent whether a failure is its regression or was broken all along.
raw pytest dump: ~40,000 tokens, unstructured, run un-sandboxed on your machine
verdict: ~400 tokens, typed JSON, run in a rootless container, with memoryWhy
The highest-frequency tool call in agentic coding is verification — and it's the least structured. Agents re-run whole suites when one module changed, burn context parsing ANSI-coded tracebacks, run arbitrary code directly on your machine, and routinely misdiagnose pre-existing breakage as their own regression (then "fix" code that wasn't broken). verdict fixes all four.
Related MCP server: Checkride
Tools
Tool | What it does |
| Selects tests affected by your working-tree diff (static import graph via grimp), runs them via podman/docker with the worktree mounted read-only, returns typed failures with fingerprints and a |
| Full traceback for one failure, on demand — bulk never rides in the summary |
| First seen / last seen / times seen — regression vs. long-standing breakage |
| Lint and type checks, normalized into the same verdict schema |
Every failure carries a fingerprint: a stable hash of the normalized failure signature (volatile tokens — addresses, tmp paths, ids, durations — collapsed). Same logical failure, same fingerprint, across runs and refactors. Fingerprints are what give verdict memory.
Quickstart
No install step needed — uvx fetches it on first use. (Or uv tool install verdict-mcp / pip install verdict-mcp for a permanent verdict-mcp command.)
Claude Code — .mcp.json in your project root:
{
"mcpServers": {
"verdict": {
"command": "uvx",
"args": ["verdict-mcp"],
"env": { "VERDICT_PROJECT": "." }
}
}
}Cursor — same shape in .cursor/mcp.json.
Optional verdict.toml in your repo root:
[project]
packages = ["your_package"] # for impact selection (auto-guessed if omitted)
[runner]
image = "ghcr.io/you/yourproj-test" # prebuilt env with your deps
setup_cmd = "pip install -e .[test]" # or install on the fly (runs with network; tests don't)
# prefer = "local" # escape hatch if you have no container runtime
[limits]
max_failures = 10Try it without an agent:
cd examples/demo_project
VERDICT_PROJECT=. verdict-mcp # then connect any MCP client, or use the MCP inspectorSandbox posture (v0.1)
Checks run in an ephemeral container (podman preferred, docker fallback): worktree mounted read-only at /src, copied to a writable /work inside the container, --network=none for the check run. Your host environment is never mutated by a test run. If setup_cmd is configured, that step runs with network before the check; prefer a prebuilt image for a tighter posture. No container runtime → explicit prefer = "local" fallback runs checks against a temp copy of your worktree (still never in place). See SECURITY.md for the full threat model and known limitations.
Troubleshooting: if a verdict says container engine 'podman' could not start the check, run the suggested podman pull <image> by hand — the engine's own error is the answer. One known trap on macOS: a "credsStore": "gcloud" line in ~/.docker/config.json makes podman call the gcloud credential helper for every registry, including docker.io; an expired gcloud login then breaks all pulls. Fix with gcloud auth login or remove that line.
Honest limitations
Impact selection uses the static import graph — approximate by design. Dynamic imports, fixture-by-name resolution, and data-driven tests can be missed;
verify(scope="all")is always available and verdict says inselection_notewhenever it falls back.Python/pytest only today, plus ruff/mypy. The adapter interface is small and documented — vitest and
go test -jsonadapters are the most-wanted contributions (CONTRIBUTING.md).Flake detection and coverage-map-based selection are v0.2 (roadmap).
Roadmap
v0.2: coverage-based impact maps (precise selection), flake detection via fingerprint alternation, devcontainer.json support, result cache keyed on (tree hash, check, image digest). Later: vitest/jest, go test, cargo test adapters; per-repo verdict daemon mode.
License
Apache-2.0
Available Tools
4 toolsexplain_failureA
Full traceback and output for one failing check from a previous verify run.
Inputs: check_id (required), run_id. Returns: object
historyA
When was this failure fingerprint first/last seen, and how many times?
Use this to distinguish a regression you introduced (unknown fingerprint) from long-standing breakage (fingerprint seen across many runs/commits).
Inputs: fingerprint (required). Returns: object
run_checksB
Run lint/type checks ('ruff', 'mypy') in the sandbox; same verdict shape as verify.
Inputs: checks. Returns: object
verifyA
Verify the current working-tree changes by running affected pytest tests in a sandbox.
Inputs: base, scope. Returns: object
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
TDQS
Scored across 4 tools
Each tool has a clearly distinct purpose: verify runs unit tests, run_checks runs static checks, explain_failure provides failure details, and history shows recurrence patterns. No overlap or ambiguity exists between tools.
Most tools follow a verb-first convention (verify, explain_failure, run_checks), but 'history' is a noun used as a query command. The style is otherwise consistent with snake_case and lowercase, making it readable, but the mix prevents a perfect score.
With only 4 tools, the server is tightly scoped to verification and failure analysis. Each tool earns its place, covering the necessary actions without bloat or redundancy.
The tool surface fully covers the core workflow: running tests/lint checks, explaining failures, and checking historical recurrence. There are no obvious missing operations for this domain, and the lifecycle is complete for its stated purpose.
Maintenance
Related MCP Connectors
Build, test and cache insights for Xcode, Gradle and Bazel projects, for coding agents.
QA platform for agents: coverage signals, in-repo test plans, verified tests and release governance.
Test analytics for AI agents: test history, flaky tests, failure clusters, coverage.
Pre-execution governance for AI agents. Deterministic PASS/FAIL/REVIEW verdicts, replayable proof.
Related MCP Servers
- AlicenseAqualityDmaintenanceProvides an isolated workspace for testing candidate code, runs tests, and returns deterministic pass/fail verdicts. Enables automated grading of software engineering solutions by ensuring reproducible test runs.5MIT
- AlicenseAqualityDmaintenanceEnables AI coding agents to run fast, sandboxed pre-flight validation—tests, type-checking, linting, security audits, Git safety scans, and fix suggestions—before committing or pushing code, with instant caching and optional Docker isolation.1012 npmMIT
- AlicenseAqualityBmaintenanceProvides coding agents with visibility into test health through tools for flaky test detection, test quality linting, and LLM evaluation harness, enabling them to triage failures, review test quality, and check prompt changes for regressions.9Apache 2.0
- AlicenseAqualityBmaintenanceEvidence-backed, read-only codebase diagnostics for coding agents: secrets, dependency, RLS, and source-impact audits with a verify loop and PR review verdicts. Never writes files or uses the network by default.7101 npmMIT