cloture
This server does not currently have a deployable release on Glama. Follow the repository's setup instructions to install and configure it for your MCP client, including any required credentials and local dependencies.
cloture
End the multi-agent debate the millisecond the math proves who won — and catch when your judge panel is 2 correlated votes wearing 9 trench coats.
Try it now with zero install: Live Browser Playground (
215 KBzero-import Rust WebAssembly) — paste your own LLM-as-a-judge CSV privately (zero backend upload), run live Monte Carlo stopping races, or stress-test Active-PPI++ against adversarially inverted judges.
cloture is a zero-dependency Rust core (cloture-core) + single 2.47 MiB static CLI/MCP binary (cloture) + zero-dependency Python SDK (cloture) + 215 KB zero-import WebAssembly JS/TS SDK (js/cloture.mjs) + C ABI (include/cloture.h) for sub-microsecond anytime-valid consensus stopping, correlated-panel auditing, sequential agent racing, active prediction-powered evaluation, poisoning-resistant fusion, and distributed <1 KB evidence monoids.
Zero external Rust crates (
Cargo.lockis 15 lines;stdonly).Zero Python runtime dependencies (
import cloturecold-starts in26.5 msvs253 msNumPy /860 msSciPy /2,131 msppi_py).Zero-import
215 KBWebAssembly (73 KBgzipped) running in browsers, Node.js, Deno, Bun, and Cloudflare Workers (js/cloture.mjs).Exact finite-sample martingale & conformal guarantees — valid at every data-dependent stopping time, with zero p-hacking and zero asymptotic small-sample approximations.
Built-in JSON-RPC 2.0 MCP server (
cloture mcp,76.7 µsmedian tool roundtrip) and one-command self-benchmark (cloture bench).Battle-tested across Linux, macOS, and Windows:
266+tests passing (55Rust core + Python/CLI/MCP integration +203independent adversarial statistical, FFI, and fuzzing breaker tests + Node.js WASM suite).
Why cloture?
Multi-agent systems and LLM evaluation pipelines bleed tokens, wall-clock latency, and statistical validity in five ways:
Fixed-
Nself-consistency / Best-of-Nwastes 40%–92% of calls on easy prompts while under-sampling hard ones; naive early stopping ("stop when 5 agree" orp < 0.05peeking) inflates false-certification rates to 28.5%–53.8%.LLM-as-a-judge panels suffer correlated error collapse: 9 frontier judges often deliver only ~2.2 effective independent votes, losing ~20 pp vs. the Condorcet independence prediction — and dropping redundant judges improves accuracy while cutting API cost.
Correlated rollouts masquerade as population proof: pooling 16 correlated rollouts per prompt produces 33%–40% false-positive deployment certificates at nominal
5%error.Fixed-horizon PPI (
ppi-python) and naive A/B racing break under continuous peeking (71%coverage under peeking;40%tie-null false positives for heuristic rankers) and have zero online defense against biased or adversarially inverted LLM judges.Poisoned agents and doomed multi-turn trajectories corrupt naive ensembles and burn tokens long after success probability collapses.
cloture fixes all five in one sub-microsecond native engine.
Related MCP server: evalgate
10-Second Quickstart
Install
# Python SDK (universal py3-none wheel, zero runtime deps)
pip install cloture
# Single static CLI & MCP binary (SHA-256 verified)
curl -fsSL https://raw.githubusercontent.com/pgg898200-ship-it/cloture/main/scripts/install.sh | sh
# Rust CLI / crate (zero external crates)
cargo install cloture-cli
# Or build from source (~18s, zero dependencies)
cargo build --release --workspace && ./target/release/cloture bench1. Stop parallel async LLM/agent rollouts the millisecond consensus is certified
import asyncio
import cloture
# Category-agnostic CITE: works over arbitrary strings, code patches, SQL, or JSON
result = await cloture.consensus(
spawn_fn=lambda i: call_agent_async(prompt, seed=i),
delta=0.05,
max_calls=32,
concurrency=8,
)
print(result.winner, f"certified in {result.n_samples}/32 calls (cancelled {result.n_cancelled} in-flight)")
# Grey-box ASC-D: pass LLM logprobs / softmax vectors for 40%–92% fewer trajectories
probs = cloture.extract_logprob_distribution(top_logprobs, ["A", "B", "C", "D"])
stopper = cloture.AscdStopper(k=4, delta=0.05, max_budget=64)
dec = stopper.update(probs)2. Audit a correlated multi-agent / LLM-as-a-judge panel
import cloture
# Direct 1-line audit of any Promptfoo / Braintrust / Inspect AI / pandas CSV export (zero pandas required!)
res = cloture.audit_csv("judges.csv", gold_col="gold")
print(f"{res.report.k_judges} judges -> {res.report.n_eff_kish:.2f} effective votes | prune: {res.prunable_judges()}")
# Or pass integer vote matrices directly (N x K)
report = cloture.audit_panel(votes, gold, n_perms=2000, n_condorcet_sims=2000, n_boot=1000)
print(f"Condorcet gap: {report.condorcet_gap*100:.1f} pp | permutation p={report.permutation_p_value:.5f}")3. Stream from stdin or attach as an MCP server
# Halt an open-ended vote stream the millisecond CITE certifies a winner
printf '42\n42\n42\n42\n42\n42\n42\n42\n42\n42\n' | ./target/release/cloture stop --mode cite --delta 0.05
# Grey-box ASC-D stopping on streamed probability vectors
printf '[0.9,0.05,0.05]\n[0.88,0.07,0.05]\n' | ./target/release/cloture stop --mode ascd --k 3 --delta 0.05
# Audit a judge panel JSON ({"votes": [[...]], "gold": [...]})
./target/release/cloture audit --file panel.json
# Run built-in Rust microbenchmarks on your CPU
./target/release/cloture bench
# Start zero-dependency JSON-RPC 2.0 MCP server over stdio (Claude Code / Cursor / Codex)
./target/release/cloture mcpVerified Head-to-Head Benchmarks
All numbers below were measured live on Intel Xeon @ 2.30 GHz (2 vCPU), Python 3.12.16, Rust 1.100.0, against real PyPI packages (confseq==0.0.11, ppi-python==0.2.3, expectation==0.6.1, online-cp==0.4.0, bayesbench==0.4.0.dev0, numpy==1.26.4, scipy==1.15.3). Full raw receipts live in benchmarks/results.json; reproduce with PYTHONPATH=python .venv/bin/python benchmarks/run_benchmarks.py.
1. Multi-Agent Consensus Stopping (delta = 0.05, 200 trials/regime)
Regime |
|
| Fixed-64 / Fixed-256 | Naive Peeking ( |
Easy ( |
|
|
|
|
Medium ( |
|
|
|
|
Hard ( |
|
|
|
|
Exact-Tie Null ( |
|
| — |
|
Open-Ended Zipf ( | — (requires known |
|
|
|
Open-Ended Tie Null ( | — |
| — |
|
CPU Latency ( |
|
| — | — |
2. Correlated-Panel Audit (K=9 judges, N=1,000 items, 2k perms + 2k Condorcet MC + 1k bootstrap)
Metric |
| Vectorized NumPy / SciPy Ref | Result |
Median Wall-Clock |
|
|
|
Effective Votes ( |
| Identical ( | 9 judges behave like 2.2 independent votes |
Condorcet Gap & Omnibus |
|
| Correlated errors destroy majority lift |
LOO Redundant-Judge Pruning | Drop 1 judge ( | Identical | Fewer, less-correlated judges win |
3. Realized vs. Population Correlated Rollouts (rho = 0.60, 30 prompts x 16 rollouts, delta = 0.05)
Method | Null False-Fire ( | Power ( |
Naive Pooled Z-test |
| — |
Naive Pooled Empirical-Bernstein CS |
| — |
Single-Unit Consensus Misfire |
| — |
|
|
|
4. Anytime-Valid Confidence Sequences & Racing vs. Real PyPI Competitors (alpha = 0.05)
Engine / Library | Anytime Miscoverage ( | Width | Throughput ( | Notes |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| Single-call final-only is faster; 1-step streaming is |
Naive CLT / Wald Peeking |
|
|
| Invalid under continuous monitoring |
Multi-Arm Racing (6 arms, 200 tasks, 1,200 Fixed-N evals):
cloture.AgentRace(paired=True):299.9evals (75.0%saved),100.0%accuracy,0.0%null FWER.cloture.AgentRace(paired=False)(LUCB):709.0evals (40.9%saved),100.0%accuracy,0.0%null FWER.vs.
bayesbench: On a 6-arm 200-task race,cloture.AgentRace(paired=True)finishes in2.75 ms(306evals) vs.bayesbench.BayesianRanker30,514 ms(1,200evals,11,112xfaster). On exact-tie0.50 vs 0.50pairwise nulls (30 trials),bayesbench.comparefalse-certifies40.0%(196.6 ms/trial) vs.cloture.PairedEProcess0.0%(0.205 ms/trial,959xfaster).
5. Active-PPI++ vs. ppi-python, Human-Only CS, & Adversarial LLM Judges
Scenario |
|
|
| Human-Only EB-CS (10%) | Naive LLM Judge |
Accurate Biased Judge ( |
|
|
|
|
|
Adversarially Inverted Judge ( |
| — |
|
|
|
6. SafeFuse Quarantine, AbortCascade, OnlineConformal vs. online-cp, & 712-Byte EvidenceSketch
SafeFuse(5agents,2poisoned att=200, 50 trials): quarantines both poisoned agents on100%of runs (median94steps),0.0%clean-null FWER (and<= 0.75%even under adversarial heavy-tailed20*Bern(0.05),Exp(1), and left-skewed{0, 1.05}nulls), reducing cumulative loss by-200.0vs. uniform averaging (and beating the oracle clean mean by-2.74). RustSafeFuse(K=5)::update:1.24 µs.AbortCascade(500cal /500test, 50 trials, target recall0.92): achieves92.6%mean test recall (vs. naive uncalibrated0.35threshold collapsing to66.1%) while saving38.2%of failed-episode compute (27.8%total compute). Calibration:2.79 msmedian.OnlineConformal(4-rate AgACI, abrupt shift att=1000, target90%):64.5 ns/stepRust /2.02 µsPython —2,930xfaster (Rust) /93.3xfaster (Python) thanonline-cpSimpleJumper(188.9 µs) and154xfaster thanonline-cpConformalRidgeRegressor(312.6 µs). Post-shift coverage recovers to94.4%vs.online-cpCRR68.0%and fixed unadapted33.5%.EvidenceSketch: fixed712-byte wire format (b"CLTR"+ FNV-1a 64-bit checksum),76.3 nsRust merge (852 nsPython ctypes),0.0exact diff across1,000-worker fleet merge (100,000obs) and exact Leave-One-Out subtraction.
Engines & Module Map
Engine | Rust ( | Python ( | CLI / MCP | Mathematical Foundation |
1. Consensus Stopping |
|
|
| Grey-box |
2. Correlated-Panel Audit |
|
|
| Kish & Jacobi Eigenvalue $n_{\text{eff}}$, stratified Condorcet null & permutation test, Krippendorff's $\alpha$, LOO pruning (May 2026) |
3. Anytime-Valid CS, Racing & Active-PPI++ |
|
|
| Predictable plug-in EB-CS & Hedged ONS Betting CS (JRSS-B 2024), LUCB + paired e-process racing, e-BH (2022), centered affine Active-PPI++ (2024–2026) |
4. Safe Fusion & Abort Cascades |
|
|
| Scale-Free BOA + unclamped Fan-inequality quarantine over |
5. Distributed Evidence Monoid |
|
| Rust / C / Python | Fixed |
Honest Scope Boundaries
Single-shot fixed-horizon PPI without peeking: If you collect a single static dataset, look once at
N, and never monitor sequentially, asymptotic normalppi-pythonproduces tighter intervals (0.054vs.0.146) because it does not pay the finite-sample anytime-validO(log log t)and bounded-range Fan-inequality price. Usecloture.ActivePPIwhenever you stream, peek, stop early, or need worst-case finite-sample protection against biased/adversarial judges.Single-call offline batch EB-CS: If you only want one final Empirical-Bernstein bound over an already-collected static array of 100k floats without intermediate checks,
expectationis faster in a single batch call (22.8 ns/stepvs.110 ns/step). For sequential/streaming monitoring (checking after every observation or mini-batch),clotureis219xfaster in Python andO(1)per step.Population vs. unit certificates:
AscdStopperandCiteStoppercertify the modal answer of a single prompt's rollout distribution. To certify deployment-level superiority across prompts under correlated rollouts, always aggregate unit means inPopulationCert.
Documentation & Agent Discovery
License
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
This server cannot be deployed
Maintenance
Related MCP Connectors
Trust gate for AI agents: multi-model adversarial consensus, signed and verifiable verdicts.
Evidence-bound stopping for agents: judgment ledger, binding futility verdicts, signed receipts.
Cryptographic 563-leaf LLM rate oracle, context-cliff auditor, and prompt cache optimizer.
Checkpoint-paired twin trials for agent fleet changes with blinded verdicts.
Related MCP Servers
- AlicenseBqualityCmaintenanceVerify a number before an agent asserts it — a Deflated Sharpe Ratio for backtest, plus eval-gap, subset-win, and judge-bias checks, with signed receipts anyone can verify offline.34MIT
- AlicenseAqualityCmaintenanceEval-integrity statistics for AI benchmark claims — multiple-testing correction, power/MDE for model gaps, judge-bias and leaderboard-rank checks. Catches a benchmark number that won't survive a second look.9MIT
- FlicenseNot gradedqualityDmaintenanceScores AI agent trajectories and detects silent failures, loops, and reliability issues with zero external API cost.-
- AlicenseAqualityBmaintenanceEnables coding agents to make cheap, fast probabilistic decisions on every turn, with tools for coding-loop checks, review, verification, screening untrusted input, and ranking candidates.668MIT