コーディングエージェントのための構造化・サンドボックス化された検証フィードバック。
エージェントの pytest シェルアウトを、エージェントのインナーループ向けに作られたものに置き換える MCP サーバーです。影響を受けたテストを選択し、隔離された環境で実行し、生のランナー出力の40,000トークンではなくコンパクトな型付き判定を返します。さらに失敗フィンガープリントにより、失敗が自分のリグレッションなのか、元々壊れていたのかをエージェントに伝えます。
raw pytest dump: ~40,000 tokens, unstructured, run un-sandboxed on your machine
verdict: ~400 tokens, typed JSON, run in a rootless container, with memoryエージェント型コーディングで最も頻度の高いツール呼び出しは検証ですが、最も構造化されていません。エージェントは1つのモジュールが変更されただけでスイート全体を再実行し、ANSIコード化されたトレースバックを解析してコンテキストを消費し、任意のコードを直接マシン上で実行し、既存の破損を自分のリグレッションと誤診して（壊れていないコードを「修正」して）しまうことがよくあります。verdict はこれら4つすべてを修正します。
Related MCP server: Checkride
ツール | 機能 |
| 作業ツリーの差分に影響を受けるテストを選択し（grimpによる静的インポートグラフ）、podman/dockerで作業ツリーを読み取り専用でマウントして実行し、フィンガープリントと |
| 1つの失敗の完全なトレースバックをオンデマンドで提供。大量の情報はサマリーに含まれません |
| 初回検出日時 / 最終検出日時 / 検出回数 — リグレッションか長期間の破損かを判別 |
| リンターと型チェックを実行し、同じ判定スキーマに正規化します |
すべての失敗にはフィンガープリントが付与されます。これは正規化された失敗シグネチャの安定したハッシュです（揮発性トークン — アドレス、一時パス、ID、期間 — は省略されます）。同じ論理的失敗は、実行やリファクタリングをまたいでも同じフィンガープリントになります。フィンガープリントこそが verdict に記憶を与えます。
インストール手順は不要です — uvx が初回使用時に取得します。（または、uv tool install verdict-mcp / pip install verdict-mcp で永続的な verdict-mcp コマンドをインストールできます。）
Claude Code — プロジェクトルートの .mcp.json:
{
"mcpServers": {
"verdict": {
"command": "uvx",
"args": ["verdict-mcp"],
"env": { "VERDICT_PROJECT": "." }
}
}
}Cursor — .cursor/mcp.json に同じ形で配置します。
リポジトリルートに任意の verdict.toml:
[project]
packages = ["your_package"] # for impact selection (auto-guessed if omitted)
[runner]
image = "ghcr.io/you/yourproj-test" # prebuilt env with your deps
setup_cmd = "pip install -e .[test]" # or install on the fly (runs with network; tests don't)
# prefer = "local" # escape hatch if you have no container runtime
[limits]
max_failures = 10エージェントなしで試す:
cd examples/demo_project
VERDICT_PROJECT=. verdict-mcp # then connect any MCP client, or use the MCP inspectorチェックは一時コンテナ（podman優先、dockerフォールバック）で実行されます。作業ツリーは /src に読み取り専用でマウントされ、コンテナ内の書き込み可能な /work にコピーされ、チェック実行時は --network=none が設定されます。ホスト環境がテスト実行によって変更されることはありません。setup_cmd が設定されている場合、そのステップはチェック前にネットワークありで実行されます。より厳格な姿勢にはプリビルドイメージを推奨します。コンテナランタイムがない場合、明示的な prefer = "local" フォールバックにより、作業ツリーの一時コピーに対してチェックが実行されます（それでも元の場所では実行されません）。完全な脅威モデルと既知の制限については SECURITY.md を参照してください。
トラブルシューティング: 判定で container engine 'podman' could not start the check と表示された場合は、提案された podman pull <image> を手動で実行してください。エンジン自身のエラーが答えです。macOSでの既知の落とし穴: ~/.docker/config.json の "credsStore": "gcloud" 行により、podman がすべてのレジストリ（docker.ioを含む）で gcloud 資格情報ヘルパーを呼び出します。gcloud のログインが期限切れだと、すべてのプルが失敗します。gcloud auth login で修正するか、その行を削除してください。
影響選択は静的インポートグラフを使用します — 設計上近似です。動的インポート、フィクスチャ名による解決、データ駆動テストは見逃される可能性があります。verify(scope="all") は常に利用可能で、verdict はフォールバック時に selection_note でその旨を伝えます。
現在はPython/pytestのみ、およびruff/mypy。アダプターインターフェースは小さく文書化されています — vitest と go test -json アダプターが最も要望の多いコントリビューションです（CONTRIBUTING.md）。
フレーク検出とカバレッジマップベースの選択は v0.2 です（ロードマップ）。
v0.2: カバレッジベースの影響マップ（正確な選択）、フィンガープリントの交互によるフレーク検出、devcontainer.json サポート、(ツリーハッシュ、チェック、イメージダイジェスト) をキーとする結果キャッシュ。その後: vitest/jest、go test、cargo test アダプター、リポジトリごとの verdict デーモンモード。
Apache-2.0
Full traceback and output for one failing check from a previous verify run.
| Name | Required | Description | Default |
|---|---|---|---|
| run_id | No | optional; defaults to the most recent run containing this check. | |
| check_id | Yes | the failure's check_id, e.g. 'pytest::tests/test_x.py::test_y'. |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full disclosure burden. It reveals the return content (traceback and output) and the source (previous verify run), which is helpful. Yet it does not explicitly declare read-only behavior or lack of side effects, nor what happens if the check_id is not found. This is a moderate gap given the absence of annotation support.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficiently worded sentence that front-loads the core purpose and key constraint (from a previous verify run). There is zero wasted text, making it easy for an agent to parse quickly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the output schema is present and parameters are fully described, the description provides sufficient context for the tool's purpose and usage. It mentions the prerequisite of a prior verify run and the focus on a specific check. Minor omissions like error handling or explicit read-only status are not critical for basic invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with both run_id and check_id already documented, including the default behavior of run_id and an example for check_id. The description adds no additional semantic value beyond the schema, such as clarifying the format of output or the relationship between the parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description precisely states the tool's function: retrieving the full traceback and output for a single failing check from a prior verify run. This clearly distinguishes it from siblings like 'verify' (which runs checks) and 'history' (which lists runs), as it focuses on failure details.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It provides context by specifying the tool is for a 'previous verify run', implying it should be used after a verification execution. However, it does not explicitly name alternatives or state when not to use it, leaving some room for inference about choosing it over history or re-running verify.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
When was this failure fingerprint first/last seen, and how many times?
Use this to distinguish a regression you introduced (unknown fingerprint) from long-standing breakage (fingerprint seen across many runs/commits).
| Name | Required | Description | Default |
|---|---|---|---|
| fingerprint | Yes |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It reveals that the tool returns historical information (first/last seen, count) and implies a read-only query, but it does not explicitly state read-only status, error handling, or any side effects. For a simple query tool, this is acceptable but not exhaustive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences with zero filler. The first sentence directly states the query, and the second explains the use case. It is front-loaded with the core purpose and efficiently conveys essential information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (one required parameter, output schema exists) and the description's coverage of purpose and usage, it is largely complete. It could explicitly mention that the tool is read-only, but that omission is minor given the lack of annotations and the straightforward nature of the operation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema defines 'fingerprint' as a plain string with no description (0% coverage). The tool description adds meaning by referring to it as a 'failure fingerprint' and using it in context, clarifying that it identifies a specific failure pattern. This compensates for the lack of schema documentation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states what the tool returns (first/last seen, count) and identifies the resource (failure fingerprint). It also gives a specific use case—distinguishing new regressions from long-standing breakage—which clearly differentiates it from siblings like verify or explain_failure.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides a clear when-to-use: when you need to know if a fingerprint is unknown (possible regression) or has been seen across runs/commits (long-standing breakage). It gives a concrete scenario but does not explicitly name alternatives or state when not to use the tool, so it falls just short of a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Run lint/type checks ('ruff', 'mypy') in the sandbox; same verdict shape as verify.
| Name | Required | Description | Default |
|---|---|---|---|
| checks | No |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must carry the behavioral disclosure burden. It states execution is 'in the sandbox' (an environmental detail) and references the verdict shape of 'verify', but it does not disclose whether the tool is read-only, whether it modifies anything, what side effects exist (if any), or any authorization requirements. For a tool with no annotation coverage, this is a significant gap.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, front-loaded with the key action, and contains zero filler. Every element ('run lint/type checks', specific tools, sandbox, shape reference) earns its place. Excellent conciseness and structure.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
While the tool is simple (one optional param, output schema present), the description leaves critical gaps: it does not explain the 'checks' parameter, relies on the reader knowing 'verify's verdict shape' (which may be defined elsewhere), and does not clarify whether all checks run or a subset can be requested. An agent may not be able to call the tool correctly without additional inference.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The only parameter, 'checks', is entirely undocumented in the schema (0% coverage). The description mentions the tools 'ruff' and 'mypy' but never explains that 'checks' likely selects or filters which of these to run, nor does it describe the expected values. The description adds no meaning beyond the schema, leaving the parameter's role ambiguous.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('run lint/type checks') and specifies the exact tools involved ('ruff', 'mypy'), and notes the 'same verdict shape as verify' to differentiate from at least one sibling. It gives a specific, unambiguous purpose.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies a context (running lint/type checks) and points to a related tool ('verify') for shape, but it does not explicitly state when to use this tool over 'verify' or the other siblings. There is no direct 'when not to use' guidance, though the intent is somewhat inferable.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Verify the current working-tree changes by running affected pytest tests in a sandbox.
| Name | Required | Description | Default |
|---|---|---|---|
| base | No | git ref to diff against (default: HEAD, i.e. uncommitted changes). | |
| scope | No | None for impact-based selection (default), 'all' for the full suite, or a path like 'tests/test_api.py' to run one file. |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It reveals that tests run 'in a sandbox', implying isolation, which is useful. However, it does not mention whether the tool modifies files, requires network access, or has any side effects on the workspace. Since it does not contradict annotations (none exist) and provides one meaningful behavioral detail, a middle score is appropriate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single, front-loaded sentence that immediately communicates the action, target, and method. Every word contributes value, with no redundancy or filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool is relatively simple: two optional params, no required fields, and an output schema exists (so return format need not be explained). The description covers the core behavior adequately. While it could elaborate on how 'base' and 'scope' interplay, the schema already documents those, so the description does not need to repeat them. Minor gap: it does not state whether the tool is read-only, but the sandbox hint mitigates that.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% – both 'base' and 'scope' have descriptions in the schema. The tool description adds no additional parameter semantics beyond what the schema already provides. Per the baseline rule for high schema coverage, a score of 3 is given.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the specific action ('Verify'), the target ('current working-tree changes'), and the method ('running affected pytest tests in a sandbox'). It distinguishes itself from siblings: explain_failure is for diagnosing failures, history is for past actions, and run_checks is a broader term that might overlap, but this tool is explicitly about pytest verification, making its niche unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies the tool is for verifying changes before committing, but it does not explicitly state when to prefer this over run_checks or provide exclusion criteria. There is no mention of alternatives or conditions that would make this tool inappropriate, leaving the routing decision to the agent's inference.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
v0.1.0explain_failurehistoryrun_checksverifyScored across 4 tools
Each tool has a clearly distinct purpose: verify runs unit tests, run_checks runs static checks, explain_failure provides failure details, and history shows recurrence patterns. No overlap or ambiguity exists between tools.
Most tools follow a verb-first convention (verify, explain_failure, run_checks), but 'history' is a noun used as a query command. The style is otherwise consistent with snake_case and lowercase, making it readable, but the mix prevents a perfect score.
With only 4 tools, the server is tightly scoped to verification and failure analysis. Each tool earns its place, covering the necessary actions without bloat or redundancy.
The tool surface fully covers the core workflow: running tests/lint checks, explaining failures, and checking historical recurrence. There are no obvious missing operations for this domain, and the lifecycle is complete for its stated purpose.
QA platform for agents: coverage signals, in-repo test plans, verified tests and release governance.
Change-aware CI validation and affected-test guidance for coding agents.
Pre-execution governance for AI agents. Deterministic PASS/FAIL/REVIEW verdicts, replayable proof.
Run AI evaluations, record agent traces and gate releases on evidence, with scoped org access.
