agent-courtroom
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@agent-courtroom Put my implementation plan on trial for hidden risks and unverified claims."
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
Courtroom (agent-courtroom) is an adversarial multi-agent judicial governance framework, CLI, stdio MCP server, and agent skill across three jurisdiction modes (architecture, software, and pipeline).
AI coding agents love to write a plan, write the code, write the test, and announce "Everything verified!" in a single optimistic loop. Courtroom replaces self-certification with an adversarial court process backed by deterministic Python verification (The Bailiff), hierarchical living case law (Parts 0–3), provenance-blinded judges, and hard USD budget caps.
Why Courtroom? (And Why a Single Persona Prompt Isn't Enough)
Single-persona coding skills nudge an agent's tone (for example, "be minimal" or "act like a senior dev"). Tone is not governance. Under pressure, a single persona still hallucinates citations, drifts outside the approved blast radius, treats draft rules as active law, or rationalizes its own output as correct.
Courtroom separates advocacy, fact-checking, and judgment:
The Prosecutor attacks failure modes, concurrency races, security hazards, migration risks, and unpredicted blast radius.
The Public Defender (YAGNI & Simplicity Counsel) attacks speculative abstractions, premature optimization, redundant dependencies, and scope creep beyond the Law.
The Bailiff (
evidence.py) never trusts LLM prose—it verifies verbatim HTML/text quotes, Python AST symbols & line ranges, and SHA-256 command exhibits deterministically in code before any judge rules.The Blinded Judge (
opus_client.py) evaluates submissions with provenance neutralized ("Submission A"vs."Submission B") so production output gets zero home-court advantage.
Agent Failure Mode | What Goes Wrong Without Governance | How Courtroom Enforces It in Code |
Self-certified plans | Agent skips edge cases or over-engineers abstractions unchecked |
|
Hallucinated quotes & symbols | Claims |
|
Silent blast-radius creep | Fixing one bug breaks untouched modules or edits protected policy files |
|
Rule drift & forgotten decisions | Old owner rulings get edited or draft proposals leak into prod behavior | Hierarchical Parts 0–3 Law ( |
Judge self-preference & spend spikes | Judges favor |
|
Related MCP server: ape
End-to-End Local Walkthrough (Captured From Live Run)
Everything below comes directly from running the 4-step lifecycle locally on examples/sample_workspace/.
The Scenario in examples/sample_workspace/:
Two regulatory notice documents sit in the corpus:
DOC-001.html: "Notice 2026-10: Permit #A-100 is GRANTED for Offshore Block 4."DOC-002.html: "Notice 2026-11: Application #B-202 is REVOKED due to expired bond."
Part 0 Owner Decision
OD-1states: "Production output must never publish a status contradicted by the verbatim source notice."Part 1 Precedent
PREC-001holds: "Explicit GRANTED dispositions override generic notice defaults."Part 2
rulebook.json(v0.2.0) hasR-NOTICE-DEFAULT(enabled: true),R-GRANT(enabled: true), andR-REVOKE-PROPOSAL(enabled: false— a proposed rule that is not active law yet).
Step 1: Dual-Advocate Plan Hearing (plan_court.py)
Before touching code or rules, we submit PLAN-001.json (which declares blast_radius.allowed_files: ["law/rulebook.json"], forbidden_files: ["law/owner_decisions.md"], and expected_deltas: {"DOC-001.status": "GRANTED"}) alongside verbatim exhibit E1 from docs/DOC-001.html.
Both Prosecutor (OBJ-1) and Public Defender (OBJ-2) challenge ITEM-1. We answer both objections in PLAN-001.rebuttal.json, The Bailiff verifies Exhibit E1, and the Plan Judge issues overall=approved.
Step 2: Scoped Implementation & Living Case Law (law/)
Once approved, only the scoped change is applied (R-GRANT enabled in law/rulebook.json, bumped to 0.2.0 with matching changelog entry).
Courtroom assembles Parts 0–3 Law (law.assemble_law) in strict precedence order:
Part 0 —
owner_decisions.md: Dated## YYYY-MM-DD | OD-Nblocks. Strictly append-only—modifying or deleting an earlier decision failsdoc_check.pyandvalidate_owner_decisions_append_only.Part 1 —
CASE_LAW.json: Binding court precedents decided under owner delegation ("decided by the court under OD-1").Part 2 —
rulebook.json: Onlyenabled: truerules enter active law.R-REVOKE-PROPOSAL(enabled: false) is kept strictly segregated under DISABLED PROPOSALS — NOT LAW.Part 3 —
conventions.md: Engineering & formatting conventions (e.g., "Format only the files you change").
Step 3: Zero-Cost Replay Gate & Blind Adversarial Trial (replay_gate.py & court.py)
Next we run run_pipeline.py to generate candidate output (ours-r1.json) and check it with replay_gate.py at $0.00 LLM cost:
DOC-001.statustransitionedPENDING -> GRANTEDas predicted.unpredicted_diffs=[]andblast_radius_violations=[](law/rulebook.jsonis insideallowed_files).
Then court.py runs an evaluation trial across all documents against an independent Party B oracle (total sample round cost: $0.0620):
On
DOC-001, Party A and Party B both agree onGRANTED(zero judge calls needed).On
DOC-002, Party A falls back toPENDING(becauseR-REVOKE-PROPOSALis still disabled) while Party B outputsREVOKED. Advocate A and Advocate B present openings & rebuttals, The Bailiff verifies exact normalized text offsets (offset 0andoffset 38), and the Blinded Trial Judge awardsDOC-002to Party B citing verified decisive quote"REVOKED due to expired bond.".
Step 4: Round Report, Closing Hearing & Doc Integrity (closing.py & doc_check.py)
Rather than ignoring DOC-002 or hot-patching disabled rules without a hearing, every remaining loss is grouped in BRIEF.json (each loss appears in exactly one group) and sent to closing.py:
report.pysummarizes:1 Agreement,1 Contested,ACCEPTED-but-wrong: 0,Rule-Tuning Score: 50.0%,Round Cost: $0.0620,Baseline Check: PASS.closing.pyreviews GroupG1(DOC-002) and returnsdecision: continuewith action: "Submit PLAN-002 to enable R-REVOKE-PROPOSAL in derive stage."doc_check.py --require-renderedconfirmsSKILL.mdstructure, zero unresolved{{PLACEHOLDER}}tokens, append-onlyowner_decisions.md, andrulebook.jsonversion/changelog sync (passed=True).
Under the Hood: 4 Details Worth Knowing
1. How The Bailiff Catches Hallucinations (src/courtroom/evidence.py)
HTML & Text Quotes (
verify_quote): Strips HTML tags, unescapes HTML entities, collapses whitespace, enforces a 4-character minimum normalized length, and verifies exact substring presence plus match offset.Python AST & Line Ranges (
verify_ast_citation): Resolves paths safely insiderepo_root(candidate.relative_to(root)prevents../traversal), validatesL10-L25bounds against actual file line counts, or parses Python ASTs to verify dotted symbols (ClassName.method_nameor top-level variables/functions) and their exactlineno–end_linenorange.Command Exhibits (
run_command_exhibit): Executes bounded commands (timeout=30default) and records exit code plus a SHA-256 digest (output_sha256).If the Trial Judge rules with an
UNVERIFIEDor missingdecisive_quote(and advocates haven't conceded),court.pyflags it as a void ruling and retries once with explicit Bailiff feedback.
2. How Judge Provenance Blinding Works (blind_payload in src/courtroom/opus_client.py)
Before sending prompts to plan_judge, trial_judge, or closing_judge, CourtClient runs blind_payload() to neutralize provenance bias:
"Party A (production)"->"Submission A""Party B (independent)"->"Submission B""ours (production)"/"production output"->"Submission A""ours_a"/"ours"->"submission_a"
Judges rule strictly on verified evidence and active Law, never on which pipeline produced the answer.
3. How Party B Reuse & Budget Caps Keep Costs Predictable (court.py & opus_client.py)
Disk Cache & Ledger: Every model call is keyed by SHA-256 of
(model, role, schema_name, system_prompt, user_prompt)and cached on disk; spend is recorded inledger.json. Cache hits cost$0.00.Hard Budget Cap (
--budget-usd): Before every uncached model call,CourtClientcheckscurrent_spend() + estimate_cost(...) > budget_usdand raisesBudgetExceededErrorbefore spending a cent over budget.Cross-Round Reuse (
--reuse-b&--previous-round): In Round 2+, pass--reuse-b round-1to reuse Party B outputs across unchanged docs, and--previous-round round-1to skip re-judging trials where Party A's output didn't change (unless you file an appeal via--appeals).
4. Three Jurisdiction Modes (architecture, software, pipeline)
Courtroom isn't tied to extraction pipelines—the same 4-step engine governs three modes:
Mode | What Party A Submits | What Party B Produces | Key Gate | Built-in Preset |
| System RFC, boundaries, data models | Independent architecture critique | Prosecutor vs. Public Defender + |
|
| Application code, APIs, tests | Blind spec-to-test assertions generated solely from |
|
|
| Corpus outputs ( | Independent oracle extraction across corpus samples | Zero-cost |
|
Plug-and-Play Setup
Prerequisites
Python
3.11+uv(recommended) orpip
git clone https://github.com/krishnanvignesh/courtroom.git
cd courtroom
uv sync --dev
uv run pytest -v1. Antigravity
Render the SKILL.md skill into your Antigravity skills directory (global or workspace) and register the MCP server:
uv run courtroom render-skill --preset software-default --out "$HOME/.gemini/antigravity/skills/courtroom/SKILL.md"Add courtroom to your Antigravity MCP config (mcp_config.json):
{
"mcpServers": {
"courtroom": {
"command": "uv",
"args": ["run", "--directory", "/path/to/courtroom", "courtroom", "mcp"]
}
}
}2. Claude Code
Render the skill into .claude/skills/courtroom/SKILL.md in your target project and register the MCP server:
uv run courtroom render-skill --preset software-default --out .claude/skills/courtroom/SKILL.md
claude mcp add courtroom -- uv run --directory /path/to/courtroom courtroom mcp3. Cursor
Render the skill as a Cursor rule (.cursor/rules/courtroom.mdc) and configure .cursor/mcp.json:
uv run courtroom render-skill --preset software-default --out .cursor/rules/courtroom.mdc.cursor/mcp.json:
{
"mcpServers": {
"courtroom": {
"command": "uv",
"args": ["run", "--directory", "/path/to/courtroom", "courtroom", "mcp"]
}
}
}4. Any MCP Client (Codex, Windsurf, Cline, Custom Agents)
courtroom mcp runs a zero-dependency stdio JSON-RPC 2.0 MCP server exposing 7 tools:
court_verify_evidence, court_assemble_law, court_challenge, court_judge, court_replay_gate, court_trial_round, and court_closing.
{
"mcpServers": {
"courtroom": {
"command": "uv",
"args": ["run", "--directory", "/path/to/courtroom", "courtroom", "mcp"]
}
}
}CLI & Standalone Scripts Quick Start
Initialize a Governed Workspace
# Scaffold law/, plans/, tracker.md, and a rendered SKILL.md
uv run courtroom init --dir path/to/project --preset software-default
# Or render SKILL.md with custom placeholders or preset (software-default | arch-default | boe-pipeline)
uv run courtroom render-skill --preset arch-default --out .ai-instructions/skills/arch-courtroom/SKILL.mdReproduce the Full Sample Walkthrough Locally
# Step 1: Dual-advocate Plan Hearing (supports .json, .md, and .yaml plans)
uv run python tools/courtroom/plan_court.py challenge --plan examples/sample_workspace/plans/PLAN-001.json --law-dir examples/sample_workspace/law
uv run python tools/courtroom/plan_court.py judge --plan examples/sample_workspace/plans/PLAN-001.json --rebuttal examples/sample_workspace/plans/PLAN-001.rebuttal.json --law-dir examples/sample_workspace/law
# Step 3a & 3b: Pipeline run + Zero-cost Replay & Blast-Radius Gate
uv run python tools/courtroom/run_pipeline.py --manifest examples/sample_workspace/manifest.json --law-dir examples/sample_workspace/law --out ours-r1.json
uv run python tools/courtroom/replay_gate.py --baseline examples/sample_workspace/baseline.json --candidate ours-r1.json --expect DOC-001.status=GRANTED --plan examples/sample_workspace/plans/PLAN-001.json --changed-files law/rulebook.json
# Step 3c: Evaluation Court Round + Inspection
uv run python tools/courtroom/court.py --ours ours-r1.json --manifest examples/sample_workspace/manifest.json --law-dir examples/sample_workspace/law --out round-1 --budget-usd 5.0
uv run python tools/courtroom/report.py --round round-1
uv run python tools/courtroom/list_losses.py --round round-1
uv run python tools/courtroom/show_trial.py --round round-1 --id DOC-002
# Step 4: Closing Hearing & Doc Integrity Check
uv run python tools/courtroom/closing.py --round round-1 --brief examples/sample_workspace/BRIEF.json --law-dir examples/sample_workspace/law --tracker examples/sample_workspace/tracker.md --budget-usd 2.0
uv run python tools/courtroom/doc_check.py --skill examples/sample_workspace/SKILL.md --law-dir examples/sample_workspace/law --require-renderedRepository Layout
Path | Purpose |
Core library: Bailiff ( | |
Standalone CLI wrappers matching the | |
Parameterized | |
End-to-end sample workspace used in the walkthrough above | |
Visual architecture cards and local CLI capture screenshots | |
Offline pytest test suite covering Bailiff verification, law invariants, budget caps, trials, replay gates, doc checks, and MCP JSON-RPC |
FAQ
Do I need live API keys just to try it or run CI?
No. Every CLI command and all tests default to --provider mock, so you can run plan hearings, replay gates, blind trial rounds, closing briefs, and CI completely offline at $0.00. Switch to --provider bedrock, anthropic, or openai (with optional extras .[bedrock], .[anthropic], .[openai]) for live model trials.
Can I write plans in Markdown instead of JSON?
Yes. plan_court.py uses markdown_compiler.py to compile .md or .yaml plans and rebuttals into validated structured JSON automatically.
How does Courtroom relate to single-persona skills like Ponytail?
Single-persona prompts optimize coding style (minimalism, cutting bloat). Courtroom enforces engineering governance—pit Prosecutor (risk/correctness) against Public Defender (YAGNI), verify every quote/AST reference in Python, gate file blast radius at $0.00, and preserve append-only case law across sessions. You can even pair minimal coding style conventions inside Part 3 conventions.md with Courtroom's adversarial gates.
Contributing, Security & License
CONTRIBUTING.md — Development setup, offline tests, and governance conventions
CHANGELOG.md — Release history (
v0.1.0)SECURITY.md — Path confinement, blinding, budget guardrails, and vulnerability reporting
LICENSE — MIT License
Available Tools
7 toolscourt_assemble_lawC
Assemble hierarchical Parts 0-3 Law (Owner Decisions, Case Law, ENABLED Rules, Conventions).
Inputs: law_dir (required).
court_challengeD
Step 1: Challenge a plan (.json/.md/.yaml) with Prosecutor and Public Defender.
Inputs: law_dir (required), plan_path (required), model, out_path, +4 more.
court_closingC
Step 4: Run closing hearing over grouped round losses.
Inputs: law_dir (required), round_dir (required), brief_path (required), model, +5 more.
court_judgeC
Step 1: Judge a plan against challenges and rebuttals, recording verdict and precedents.
Inputs: law_dir (required), plan_path (required), rebuttal_path (required), challenge_path (required), +6 more.
court_replay_gateD
Step 3a: Zero-cost replay gate and blast-radius boundary verification.
Inputs: baseline_path (required), candidate_path (required), plan_path, expectations, +1 more.
court_trial_roundC
Step 3c: Run evaluation trials across sample artifacts with Bailiff fact-checking.
Inputs: law_dir (required), out_dir (required), ours_path (required), manifest_path (required), +9 more.
court_verify_evidenceC
Bailiff deterministic verification for verbatim quotes, AST citations, or command exhibits.
Inputs: exhibit (required), repo_root, source_lookup.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
TDQS
Scored across 7 tools
Each tool maps to a distinct courtroom phase (assembly, challenge, judging, replay gate, trial rounds, closing, evidence verification), and the explicit step labels (Step 1, 3a, 3c, 4) help order them. Minor overlap exists between court_trial_round and court_closing, which both evaluate losses, and between court_challenge and court_judge as sequential Step 1 actions, but their roles are separable.
All seven tools share the court_ prefix, which makes the family instantly recognizable and scannable. The suffixes mix verb forms (challenge, judge, assemble_law, verify_evidence) with noun forms (replay_gate, trial_round, closing), a small deviation but still predictable within the courtroom metaphor.
Seven tools for a multi-stage trial pipeline is well-scoped; each tool corresponds to a stage in the process rather than a redundant variant. Nothing feels padded, and the set stays small enough to be navigable.
The surface covers assembly, challenge, judgment, gating, trials, closing, and evidence checks, but the step numbering (1, 3a, 3c, 4) implies missing operations such as a rebuttal/Step 2 and a Step 3b phase. There are also no read/query tools to retrieve prior verdicts, precedents, or cases, only tools that record or run them, leaving agents without inspection paths.
Maintenance
Related MCP Connectors
Pre-execution governance for AI agents. Deterministic PASS/FAIL/REVIEW verdicts, replayable proof.
AI-native programming language and toolchain for accountable, verifiable agent-driven development.
Policy-aware AI execution, verification, cost, and trajectory evidence for agents.
A paid remote MCP for OpenAI Codex agent coordination MCP, built to return verdicts, receipts, usage
Related MCP Servers
- AlicenseNot gradedqualityCmaintenanceEnables multi-agent code review and iterative improvement loop using different LLM backends, with deterministic scoring and audit trail.5MIT
- AlicenseNot gradedqualityAmaintenanceEnables evidence-gated, multi-session AI coding runs with plan-build-ship state management, coordinating Claude Code and Codex native agents.20 npmMIT
- AlicenseNot gradedqualityAmaintenanceEnables AI coding providers to run structured multi-agent workflows with specs, tracked tasks, handoffs, health gates, and audited repair mode.29 npmMIT
- AlicenseNot gradedqualityCmaintenanceEnables host coding agents to conduct evidence-first exploratory testing, adversarial review, reproduction, and resumable reporting through immutable job contracts, action budgets, checkpointed ledgers, integrity-checked artifacts, and optional origin-restricted browser automation.MIT