AI Workbench MCP
AI Workbench MCP is a tool server for supervising AI coding agents, capturing evidence, validating work, applying acceptance policies, and producing auditable PR-ready reports. It provides the following tools:
workbench_open_run: Create a new run folder and initialize evidence artifacts for a project task (with options for risk level, policy pack, validation profile, etc.).workbench_select_model: Select an appropriate model tier based on task type, risk, and complexity, recording the choice tomodel_selection.json.workbench_select_policy_pack: Get an advisory recommendation for which policy pack to apply based on task metadata (risk level, task type, changed files).workbench_record_execution: Capture AI agent/model response text and execution details into structured evidence artifacts, tracking files touched and run status.workbench_validate_run: Run deterministic validation over a run directory, generating avalidation_report.jsonas part of the acceptance authority.workbench_quality_gate: Evaluate a completed run against quality criteria, producing arevision_decision.jsonwith an outcome ofaccept,needs_review, orblock.workbench_analyze_runs: Analyze local run ledgers and generate report artifacts, with filtering by date, task type, and evidence scope for acceptance analytics.
Integrates with GitHub to provide a PR gate that writes acceptance decisions and comments based on validated runs, enabling auditable acceptance workflows for AI agent output.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@AI Workbench MCP gate my latest agent output"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
AI Workbench
AI Workbench supervises AI coding agents, captures evidence, validates work, applies acceptance policy, and produces auditable PR-ready reports.
The PyPI package remains ai-workbench-mcp for this public alpha because the
ai-workbench package name is already occupied. The product and CLI are
AI Workbench:
pip install ai-workbench-mcp
ai-workbench --helpCurrent source metadata targets unpublished ai-workbench-mcp==0.8.0a0.
This public alpha consolidates local supervision, evidence capture, validation,
acceptance policy, and PR reporting into one product surface.
Public Alpha Warning
The supervisor is the preferred automated evidence path, but daemon, Codex hook, and OpenCode adapter coverage are alpha mechanisms. AI Workbench checks evidence quality and acceptance readiness; it does not prove the work is absolutely correct. High-risk work still requires human review.
Related MCP server: Spec Kit Acceptance Gate MCP
Architecture
AI Workbench supervisor captures local evidence.
AI Workbench validation writes
validation_report.json.AI Workbench quality gate writes
revision_decision.json.AI Workbench PR/report surfaces render
accept,needs_review, orblock.
Agent output is a proposal. Workbench accepts evidence.
MCP is the connection protocol. AI Workbench MCP is the tool server. Acceptance is decided by the selected validation profile and quality gate. The agent performs. Workbench accepts. MCP connects them.
Quick Start
Register a project once and start the local supervisor:
pip install ai-workbench-mcp
ai-workbench supervisor setup --project-dir . --task-type code_change
ai-workbench supervisor startRun Codex, OpenCode, Goose, or another supported local workflow in the project. Then inspect the latest report:
ai-workbench supervisor status
ai-workbench reports show latest --project-dir .Render PR-ready artifacts from a finalized run:
ai-workbench pr-gate --run-dir runs/<run_id>The canonical local run ledger is:
runs/<run_id>/
task_metadata.json
final_prompt.md
model_selection.json
model_output.md
validation_report.json
revision_decision.json
run_log.jsonl
metadata.json
transcript.jsonl
commands.jsonl
workspace/
validation/
artifacts/validation_report.json and revision_decision.json are the final acceptance
authority. Supporting supervisor reports are local evidence, not a substitute
for those Workbench artifacts.
Codex Hooks
Install project-local Codex hooks:
ai-workbench setup codex --project-dir . --task-type code_changeRestart Codex or start a new session, open /hooks, review the project hook,
and trust it once. Until a hook event is observed, supervisor status reports
Codex coverage as configured but unverified.
Goose MCP
AI Workbench still exposes the same MCP tool lifecycle. Register the server with Goose or another MCP host using:
ai-workbench mcp serveThe seven MCP tools remain:
workbench_open_run
workbench_select_policy_pack
workbench_select_model
workbench_record_execution
workbench_validate_run
workbench_quality_gate
workbench_analyze_runsPR Gate
Workbench PR acceptance consumes real Workbench run evidence:
ai-workbench pr-gate \
--run-dir runs/<run_id> \
--out runs/pr_gate/pr_comment.md \
--json-out runs/pr_gate/pr_decision.jsonOutcomes are exactly:
acceptneeds_reviewblock
Missing, unreadable, or scaffold-only evidence blocks. A green CI run, uploaded artifact, sticky PR comment, or model self-claim is not acceptance evidence.
Bootstrap Assets
To add starter configs, prompts, recipes, docs, and the GitHub PR-gate workflow to a repository:
ai-workbench bootstrap --target .The bootstrap keeps runs/ ignored.
Package Demo
For a package-only synthetic demo:
ai-workbench demo --target ./workbench-first-runThis shows accept, needs_review, and block PR-gate outcomes with fixture
evidence. It is not a real target-repository acceptance run.
Development
python -m pip install -e ".[dev,publish]"
python -m pytest -q -p no:cacheprovider
python -m ruff check . --no-cache
python -m mypy --no-sqlite-cache --no-incremental
ai-workbench demo --target runs/package_demo_smoke
ai-workbench validate --project ai_workbench_mcp --profile scaffold --run-dir runs/scaffold-smokeDo not commit runs/. Committed sample evidence must be sanitized and live
under examples/.
Docs
Goose demo walkthrough - recording-ready 3-5 minute public demo runbook
Recipes:
Sample evidence:
License
Apache-2.0. See LICENSE. MIT-origin attribution for the consolidated Prove It code is retained in NOTICE.
Available Tools
7 toolsworkbench_analyze_runsB
Analyze local Workbench run ledgers and write report artifacts.
Inputs: runs_dir, task_type, since, out_dir, +2 more. Returns: object
workbench_open_runC
Create a Workbench run folder and initial evidence artifacts.
Inputs: project (required), task (required), run_dir, prompt, +10 more. Returns: object
workbench_quality_gateD
Run the Workbench quality gate for a run directory.
Inputs: project (required), run_dir (required), mode, risk, +3 more. Returns: object
workbench_record_executionC
Capture Goose/model response text into Workbench evidence artifacts.
Inputs: project (required), run_dir (required), response_text (required), files_touched, +5 more. Returns: object
workbench_select_modelC
Select a Workbench model tier and write model_selection.json.
Inputs: project (required), task_type (required), risk (required), out (required), +10 more. Returns: object
workbench_select_policy_packB
Recommend an advisory Workbench policy pack from task metadata.
Inputs: task_text, task_type, changed_files, prompt, +1 more. Returns: object
workbench_validate_runC
Run deterministic Workbench validation over a run directory.
Inputs: project (required), out_dir (required), profile, changed_files, +2 more. Returns: object
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
TDQS
Scored across 7 tools
Each tool has a clearly distinct purpose: analyzing runs, opening runs, quality gating, recording execution, selecting model, selecting policy pack, and validating runs. No overlap in functionality.
All tools share the 'workbench_' prefix and mostly follow a verb_noun pattern (e.g., analyze_runs, open_run). Minor deviation with 'quality_gate' which is noun_verb, but overall consistent.
With 7 tools, the set is well-scoped for an AI workbench run management domain. Not too few to lack utility, not too many to be unwieldy.
The tools cover core actions like opening, analyzing, recording, and validating runs, but lack lifecycle management tools such as listing, closing, or deleting runs. Some gaps exist for a complete workflow.
Maintenance
Related MCP Connectors
Pre-execution governance for AI agents. Deterministic PASS/FAIL/REVIEW verdicts, replayable proof.
Run AI evaluations, record agent traces and gate releases on evidence, with scoped org access.
Deterministic validation for AI-generated artifacts: JSON Schema, OpenAPI response, SQL syntax.
Verify what your AI coding agent built: test coverage, recorded browser tests, a done verdict.
Related MCP Servers
- AlicenseAqualityBmaintenanceEnables verification of AI coding agent self-reports against git diff truth and a deterministic gate, producing pass/regenerate/reject directives to ensure claimed work matches actual changes.6144 PyPIAGPL 3.0
- FlicenseNot gradedqualityDmaintenanceEnables spec-driven development acceptance gate with structured receipts, audit logs, and reviewer-ready evidence.-
- AlicenseNot gradedqualityCmaintenanceEnables evidence-first regression testing for AI agents by turning production traces into reviewed, replayable cases that gate releases. It supports reproducible, auditable agent evaluation with controlled tool execution, evidence-based judging, and versioned quality gates.MIT
- AlicenseNot gradedqualityCmaintenanceEnables AI agents to run evidence QC gates mid-workflow—checking claims, sources, prose, and vault hygiene deterministically before research ships.MIT