verdict
verdict
面向编码代理的结构化、沙箱化验证反馈。
一个 MCP 服务器,用专为代理内循环打造的工具取代代理的 pytest shell 调用:影响范围选中的测试,在隔离环境中运行,返回紧凑的类型化裁决,而不是 40,000 个 token 的原始运行器输出——并附带失败指纹,告诉代理该失败是它自己引入的回归,还是一直就存在的损坏。
raw pytest dump: ~40,000 tokens, unstructured, run un-sandboxed on your machine
verdict: ~400 tokens, typed JSON, run in a rootless container, with memory为什么
代理式编码中调用频率最高的工具是验证——而它恰恰是最缺乏结构化的。代理在只改了一个模块时重跑整个测试套件,浪费上下文解析 ANSI 编码的 traceback,直接在你的机器上运行任意代码,还经常把早已存在的损坏误判为自己的回归(然后去"修复"本来没坏的代码)。verdict 解决了全部四个问题。
Related MCP server: Checkride
工具
工具 | 功能 |
| 通过静态导入图(grimp)选择受工作树 diff 影响的测试,通过 podman/docker 运行,工作树以只读方式挂载,返回带指纹和 |
| 按需返回单个失败的完整 traceback——批量内容从不混入摘要中 |
| 首次出现 / 最后出现 / 出现次数——区分回归与长期存在的损坏 |
| 将 lint 和类型检查归一化到相同的裁决模式中 |
每个失败都携带一个指纹:归一化失败签名(易变 token——地址、临时路径、id、时长——已折叠)的稳定哈希。同一逻辑失败,同一指纹,跨运行、跨重构保持一致。指纹正是 verdict 拥有记忆的原因。
快速开始
无需安装步骤——uvx 会在首次使用时自动获取。(或者用 uv tool install verdict-mcp / pip install verdict-mcp 安装一个常驻的 verdict-mcp 命令。)
Claude Code —— 项目根目录下的 .mcp.json:
{
"mcpServers": {
"verdict": {
"command": "uvx",
"args": ["verdict-mcp"],
"env": { "VERDICT_PROJECT": "." }
}
}
}Cursor —— 在 .cursor/mcp.json 中使用相同结构。
仓库根目录下可选的 verdict.toml:
[project]
packages = ["your_package"] # for impact selection (auto-guessed if omitted)
[runner]
image = "ghcr.io/you/yourproj-test" # prebuilt env with your deps
setup_cmd = "pip install -e .[test]" # or install on the fly (runs with network; tests don't)
# prefer = "local" # escape hatch if you have no container runtime
[limits]
max_failures = 10不通过代理直接试用:
cd examples/demo_project
VERDICT_PROJECT=. verdict-mcp # then connect any MCP client, or use the MCP inspector沙箱姿态(v0.1)
检查在临时容器中运行(优先 podman,docker 兜底):工作树以只读方式挂载到 /src,复制到容器内可写的 /work,检查运行期间使用 --network=none。测试运行绝不会改动你的宿主机环境。如果配置了 setup_cmd,该步骤会在检查前带网络运行;更严格的做法是优先使用预构建镜像。没有容器运行时 → 显式 prefer = "local" 回退方案会针对工作树的临时副本运行检查(仍然不会原地执行)。完整威胁模型和已知限制见 SECURITY.md。
故障排查: 如果裁决显示 container engine 'podman' could not start the check,请手动运行提示中的 podman pull <image>——引擎自身的报错就是答案。macOS 上有一个已知陷阱:~/.docker/config.json 中的 "credsStore": "gcloud" 一行会让 podman 对每个 registry(包括 docker.io)都调用 gcloud 凭据助手;过期的 gcloud 登录随后会导致所有拉取失败。用 gcloud auth login 修复,或删除该行。
诚实的局限
影响范围选择基于静态导入图——按设计就是近似值。动态导入、按名称解析的 fixture 以及数据驱动测试可能被遗漏;
verify(scope="all")始终可用,且 verdict 会在回退时于selection_note中明确说明。目前仅支持 Python/pytest,外加 ruff/mypy。适配器接口很小且有文档——vitest 和
go test -json适配器是最受欢迎的贡献(CONTRIBUTING.md)。抖动检测和基于覆盖率映射的选择属于 v0.2(路线图)。
路线图
v0.2: 基于覆盖率的影
Available Tools
4 toolsexplain_failureA
Full traceback and output for one failing check from a previous verify run.
Inputs: check_id (required), run_id. Returns: object
historyA
When was this failure fingerprint first/last seen, and how many times?
Use this to distinguish a regression you introduced (unknown fingerprint) from long-standing breakage (fingerprint seen across many runs/commits).
Inputs: fingerprint (required). Returns: object
run_checksB
Run lint/type checks ('ruff', 'mypy') in the sandbox; same verdict shape as verify.
Inputs: checks. Returns: object
verifyA
Verify the current working-tree changes by running affected pytest tests in a sandbox.
Inputs: base, scope. Returns: object
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
TDQS
Scored across 4 tools
Each tool has a clearly distinct purpose: verify runs unit tests, run_checks runs static checks, explain_failure provides failure details, and history shows recurrence patterns. No overlap or ambiguity exists between tools.
Most tools follow a verb-first convention (verify, explain_failure, run_checks), but 'history' is a noun used as a query command. The style is otherwise consistent with snake_case and lowercase, making it readable, but the mix prevents a perfect score.
With only 4 tools, the server is tightly scoped to verification and failure analysis. Each tool earns its place, covering the necessary actions without bloat or redundancy.
The tool surface fully covers the core workflow: running tests/lint checks, explaining failures, and checking historical recurrence. There are no obvious missing operations for this domain, and the lifecycle is complete for its stated purpose.
Maintenance
Related MCP Connectors
Build, test and cache insights for Xcode, Gradle and Bazel projects, for coding agents.
QA platform for agents: coverage signals, in-repo test plans, verified tests and release governance.
Test analytics for AI agents: test history, flaky tests, failure clusters, coverage.
Pre-execution governance for AI agents. Deterministic PASS/FAIL/REVIEW verdicts, replayable proof.
Related MCP Servers
- AlicenseAqualityDmaintenanceProvides an isolated workspace for testing candidate code, runs tests, and returns deterministic pass/fail verdicts. Enables automated grading of software engineering solutions by ensuring reproducible test runs.5MIT
- AlicenseAqualityDmaintenanceEnables AI coding agents to run fast, sandboxed pre-flight validation—tests, type-checking, linting, security audits, Git safety scans, and fix suggestions—before committing or pushing code, with instant caching and optional Docker isolation.105 npmMIT
- AlicenseAqualityBmaintenanceProvides coding agents with visibility into test health through tools for flaky test detection, test quality linting, and LLM evaluation harness, enabling them to triage failures, review test quality, and check prompt changes for regressions.9Apache 2.0
- AlicenseAqualityBmaintenanceEvidence-backed, read-only codebase diagnostics for coding agents: secrets, dependency, RLS, and source-impact audits with a verify loop and PR review verdicts. Never writes files or uses the network by default.7101 npmMIT