Структурированная, изолированная обратная связь по проверке для агентов кодирования.
Сервер MCP, который заменяет вызовы pytest вашего агента на что-то, созданное для внутреннего цикла агента: тесты, выбранные по влиянию, запускаемые в изолированной среде, возвращающие компактные типизированные вердикты вместо 40 000 токенов необработанного вывода раннера — с отпечатками сбоев, которые сообщают агенту, является ли сбой его регрессией или был сломан изначально.
raw pytest dump: ~40,000 tokens, unstructured, run un-sandboxed on your machine
verdict: ~400 tokens, typed JSON, run in a rootless container, with memoryСамый частый вызов инструмента в агентном кодировании — это проверка, и она наименее структурирована. Агенты перезапускают целые наборы тестов, когда изменился один модуль, тратят контекст на разбор ANSI-кодированных трассировок, выполняют произвольный код прямо на вашей машине и регулярно ошибочно диагностируют ранее существовавшую поломку как собственную регрессию (а затем «исправляют» код, который не был сломан). verdict решает все четыре проблемы.
Related MCP server: Checkride
Tool | What it does |
| Выбирает тесты, затронутые вашим diff в рабочем дереве (статический граф импортов через grimp), запускает их через podman/docker с рабочей директорией, смонтированной только для чтения, возвращает типизированные сбои с отпечатками и флагом |
| Полная трассировка для одного сбоя по запросу — массово никогда не включается в сводку |
| Первый раз / последний раз / сколько раз — регрессия против давней поломки |
| Линтинг и проверки типов, нормализованные в ту же схему вердиктов |
Каждый сбой несет отпечаток: стабильный хэш нормализованной сигнатуры сбоя (изменчивые токены — адреса, временные пути, идентификаторы, длительности — свернуты). Один и тот же логический сбой, один и тот же отпечаток, при разных запусках и рефакторингах. Отпечатки дают verdict память.
Установка не требуется — uvx загрузит его при первом использовании. (Или uv tool install verdict-mcp / pip install verdict-mcp для постоянной команды verdict-mcp.)
Claude Code — .mcp.json в корне вашего проекта:
{
"mcpServers": {
"verdict": {
"command": "uvx",
"args": ["verdict-mcp"],
"env": { "VERDICT_PROJECT": "." }
}
}
}Cursor — та же структура в .cursor/mcp.json.
Необязательный verdict.toml в корне репозитория:
[project]
packages = ["your_package"] # for impact selection (auto-guessed if omitted)
[runner]
image = "ghcr.io/you/yourproj-test" # prebuilt env with your deps
setup_cmd = "pip install -e .[test]" # or install on the fly (runs with network; tests don't)
# prefer = "local" # escape hatch if you have no container runtime
[limits]
max_failures = 10Попробуйте без агента:
cd examples/demo_project
VERDICT_PROJECT=. verdict-mcp # then connect any MCP client, or use the MCP inspectorПроверки выполняются в эфемерном контейнере (предпочтителен podman, запасной вариант — docker): рабочее дерево монтируется только для чтения в /src, копируется в доступную для записи /work внутри контейнера, --network=none для запуска проверки. Ваше хост-окружение никогда не изменяется при запуске теста. Если настроен setup_cmd, этот шаг выполняется с сетью перед проверкой; для более строгого режима предпочтителен предварительно собранный образ. Нет контейнерного рантайма → явный запасной вариант prefer = "local" запускает проверки против временной копии вашего рабочего дерева (по-прежнему не на месте). См. SECURITY.md для полной модели угроз и известных ограничений.
Устранение неполадок: если вердикт говорит container engine 'podman' could not start the check, выполните предложенную команду podman pull <image> вручную — собственная ошибка движка и есть ответ. Одна известная ловушка на macOS: строка "credsStore": "gcloud" в ~/.docker/config.json заставляет podman вызывать помощник с учетными данными gcloud для каждого реестра, включая docker.io; истекший вход в gcloud затем ломает все загрузки. Исправьте с помощью gcloud auth login или удалите эту строку.
Выбор по влиянию использует статический граф импортов — приблизителен по замыслу. Динамические импорты, разрешение фикстур по имени и тесты, управляемые данными, могут быть пропущены; verify(scope="all") всегда доступен, и verdict сообщает в selection_note, когда происходит откат.
Сегодня только Python/pytest, плюс ruff/mypy. Интерфейс адаптеров небольшой и документирован — адаптеры vitest и go test -json являются самыми востребованными вкладами (CONTRIBUTING.md).
Обнаружение флейков и выбор на основе карты покрытия — это v0.2 (roadmap).
v0.2: карты влияния на основе покрытия (точный выбор), обнаружение флейков через чередование отпечатков, поддержка devcontainer.json, кэш результатов с ключом (хэш дерева, проверка, дайджест образа). Позже: адаптеры vitest/jest, go test, cargo test; режим демона verdict для каждого репозитория.
Apache-2.0
Full traceback and output for one failing check from a previous verify run.
| Name | Required | Description | Default |
|---|---|---|---|
| run_id | No | optional; defaults to the most recent run containing this check. | |
| check_id | Yes | the failure's check_id, e.g. 'pytest::tests/test_x.py::test_y'. |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full disclosure burden. It reveals the return content (traceback and output) and the source (previous verify run), which is helpful. Yet it does not explicitly declare read-only behavior or lack of side effects, nor what happens if the check_id is not found. This is a moderate gap given the absence of annotation support.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficiently worded sentence that front-loads the core purpose and key constraint (from a previous verify run). There is zero wasted text, making it easy for an agent to parse quickly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the output schema is present and parameters are fully described, the description provides sufficient context for the tool's purpose and usage. It mentions the prerequisite of a prior verify run and the focus on a specific check. Minor omissions like error handling or explicit read-only status are not critical for basic invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with both run_id and check_id already documented, including the default behavior of run_id and an example for check_id. The description adds no additional semantic value beyond the schema, such as clarifying the format of output or the relationship between the parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description precisely states the tool's function: retrieving the full traceback and output for a single failing check from a prior verify run. This clearly distinguishes it from siblings like 'verify' (which runs checks) and 'history' (which lists runs), as it focuses on failure details.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It provides context by specifying the tool is for a 'previous verify run', implying it should be used after a verification execution. However, it does not explicitly name alternatives or state when not to use it, leaving some room for inference about choosing it over history or re-running verify.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
When was this failure fingerprint first/last seen, and how many times?
Use this to distinguish a regression you introduced (unknown fingerprint) from long-standing breakage (fingerprint seen across many runs/commits).
| Name | Required | Description | Default |
|---|---|---|---|
| fingerprint | Yes |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It reveals that the tool returns historical information (first/last seen, count) and implies a read-only query, but it does not explicitly state read-only status, error handling, or any side effects. For a simple query tool, this is acceptable but not exhaustive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences with zero filler. The first sentence directly states the query, and the second explains the use case. It is front-loaded with the core purpose and efficiently conveys essential information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (one required parameter, output schema exists) and the description's coverage of purpose and usage, it is largely complete. It could explicitly mention that the tool is read-only, but that omission is minor given the lack of annotations and the straightforward nature of the operation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema defines 'fingerprint' as a plain string with no description (0% coverage). The tool description adds meaning by referring to it as a 'failure fingerprint' and using it in context, clarifying that it identifies a specific failure pattern. This compensates for the lack of schema documentation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states what the tool returns (first/last seen, count) and identifies the resource (failure fingerprint). It also gives a specific use case—distinguishing new regressions from long-standing breakage—which clearly differentiates it from siblings like verify or explain_failure.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides a clear when-to-use: when you need to know if a fingerprint is unknown (possible regression) or has been seen across runs/commits (long-standing breakage). It gives a concrete scenario but does not explicitly name alternatives or state when not to use the tool, so it falls just short of a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Run lint/type checks ('ruff', 'mypy') in the sandbox; same verdict shape as verify.
| Name | Required | Description | Default |
|---|---|---|---|
| checks | No |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must carry the behavioral disclosure burden. It states execution is 'in the sandbox' (an environmental detail) and references the verdict shape of 'verify', but it does not disclose whether the tool is read-only, whether it modifies anything, what side effects exist (if any), or any authorization requirements. For a tool with no annotation coverage, this is a significant gap.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, front-loaded with the key action, and contains zero filler. Every element ('run lint/type checks', specific tools, sandbox, shape reference) earns its place. Excellent conciseness and structure.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
While the tool is simple (one optional param, output schema present), the description leaves critical gaps: it does not explain the 'checks' parameter, relies on the reader knowing 'verify's verdict shape' (which may be defined elsewhere), and does not clarify whether all checks run or a subset can be requested. An agent may not be able to call the tool correctly without additional inference.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The only parameter, 'checks', is entirely undocumented in the schema (0% coverage). The description mentions the tools 'ruff' and 'mypy' but never explains that 'checks' likely selects or filters which of these to run, nor does it describe the expected values. The description adds no meaning beyond the schema, leaving the parameter's role ambiguous.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('run lint/type checks') and specifies the exact tools involved ('ruff', 'mypy'), and notes the 'same verdict shape as verify' to differentiate from at least one sibling. It gives a specific, unambiguous purpose.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies a context (running lint/type checks) and points to a related tool ('verify') for shape, but it does not explicitly state when to use this tool over 'verify' or the other siblings. There is no direct 'when not to use' guidance, though the intent is somewhat inferable.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Verify the current working-tree changes by running affected pytest tests in a sandbox.
| Name | Required | Description | Default |
|---|---|---|---|
| base | No | git ref to diff against (default: HEAD, i.e. uncommitted changes). | |
| scope | No | None for impact-based selection (default), 'all' for the full suite, or a path like 'tests/test_api.py' to run one file. |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It reveals that tests run 'in a sandbox', implying isolation, which is useful. However, it does not mention whether the tool modifies files, requires network access, or has any side effects on the workspace. Since it does not contradict annotations (none exist) and provides one meaningful behavioral detail, a middle score is appropriate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single, front-loaded sentence that immediately communicates the action, target, and method. Every word contributes value, with no redundancy or filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool is relatively simple: two optional params, no required fields, and an output schema exists (so return format need not be explained). The description covers the core behavior adequately. While it could elaborate on how 'base' and 'scope' interplay, the schema already documents those, so the description does not need to repeat them. Minor gap: it does not state whether the tool is read-only, but the sandbox hint mitigates that.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% – both 'base' and 'scope' have descriptions in the schema. The tool description adds no additional parameter semantics beyond what the schema already provides. Per the baseline rule for high schema coverage, a score of 3 is given.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the specific action ('Verify'), the target ('current working-tree changes'), and the method ('running affected pytest tests in a sandbox'). It distinguishes itself from siblings: explain_failure is for diagnosing failures, history is for past actions, and run_checks is a broader term that might overlap, but this tool is explicitly about pytest verification, making its niche unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies the tool is for verifying changes before committing, but it does not explicitly state when to prefer this over run_checks or provide exclusion criteria. There is no mention of alternatives or conditions that would make this tool inappropriate, leaving the routing decision to the agent's inference.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
v0.1.0explain_failurehistoryrun_checksverifyScored across 4 tools
Each tool has a clearly distinct purpose: verify runs unit tests, run_checks runs static checks, explain_failure provides failure details, and history shows recurrence patterns. No overlap or ambiguity exists between tools.
Most tools follow a verb-first convention (verify, explain_failure, run_checks), but 'history' is a noun used as a query command. The style is otherwise consistent with snake_case and lowercase, making it readable, but the mix prevents a perfect score.
With only 4 tools, the server is tightly scoped to verification and failure analysis. Each tool earns its place, covering the necessary actions without bloat or redundancy.
The tool surface fully covers the core workflow: running tests/lint checks, explaining failures, and checking historical recurrence. There are no obvious missing operations for this domain, and the lifecycle is complete for its stated purpose.
QA platform for agents: coverage signals, in-repo test plans, verified tests and release governance.
Change-aware CI validation and affected-test guidance for coding agents.
Pre-execution governance for AI agents. Deterministic PASS/FAIL/REVIEW verdicts, replayable proof.
Approved test intent, reviewed Playwright automation and run evidence, inside your editor.
