Skip to main content
Glama

๐Ÿš€ Quick start

Prerequisites: Python 3.11+.

git clone https://github.com/Abhishek249/gavel-mcp.git
cd gavel-mcp
make install          # creates .venv, copies .env.example โ†’ .env
make smoke            # lint + 22 tests + MCP prove + offline sandboxes โ€” no secrets

Try it:

# Clean NYC taxi data (expect PASS)
gavel sandbox taxi-clean --params '{"candidate_key":"trip-0000000"}'

# Taxi data with duplicate rows (expect FAIL)
gavel sandbox taxi-duplicate-rows --params '{"candidate_key":"trip-0000000"}'

Related MCP server: MCP Gatekeeper

๐Ÿ“š Releases (what shipped when)

Version

Focus

Try it

v0.4

Renamed from Proofline โ†’ Gavel (gavel CLI, gavel-mcp server)

make smoke

v0.1

Taxi benchmark + geospatial evidence MCP tools

gavel taxi

v0.2

Sandbox spec + golden/candidate + metric DSL

make sandbox-offline

v0.3

Live DevInt collector (Postgres + WO + Dagster)

make validate-mr

Full sandbox spec: docs/sandbox-platform.md ยท Architecture: docs/architecture.md

๐Ÿงฐ MCP server (local)

make mcp-config    # prints JSON snippet for Cursor

Six tools:

MCP tool

Purpose

list_sandbox_definitions

Discover sandboxes under sandboxes/

validate_sandbox_run

Golden vs candidate metrics (snapshot or inline rows)

collect_manual_report_candidate_row

Live DevInt evidence for one manual_report_id

validate_manual_report_live

Collect + validate in one call

validate_candidate_pipeline

Synthetic NYC taxi reference vs candidate

validate_geospatial_run

BYO orchestrator/worker/DB evidence envelope

Restart Cursor after editing MCP config. Walkthrough: docs/mcp-setup.md.

โœจ What's inside

  • Declarative sandboxes โ€” YAML spec: components, golden/candidate datasets, metric rubric

  • Deterministic metrics โ€” exact, jaccard, area_ratio, not_null, row_count, gte, lte

  • Silent-success detection โ€” catches green orchestration with missing/wrong data

  • Shipped examples โ€” NYC taxi clean data (PASS), duplicate rows (FAIL)

  • Structured evidence reports โ€” every check: name, expected, actual, status, context

  • MCP stdio boundary โ€” real client/server tests in CI (prove_it.py, benchmark)

Stack: Python, Pydantic, Shapely, PyYAML, MCP SDK.

๐Ÿ—๏ธ Architecture

sandbox.yaml  โ†’  golden.json + candidate.json (file | inline | live)
                      โ†“
                 metric engine (deterministic)
                      โ†“
                 ValidationReport  โ†’  MCP tool response

Example: NYC taxi duplicate-row detection

trip_id
    โ†’ load golden: 500 clean trips
    โ†’ load candidate: 600 trips (100 duplicates)
    โ†’ validate: row_count, key_uniqueness, fare_total โ†’ FAIL

๐Ÿ“‚ Project structure

gavel-mcp/
โ”œโ”€โ”€ assets/
โ”‚   โ”œโ”€โ”€ gavel-readme-banner.png # README banner
โ”‚   โ””โ”€โ”€ gavel-icon.png          # square icon (social preview)
โ”œโ”€โ”€ sandboxes/
โ”‚   โ”œโ”€โ”€ taxi-clean/             # Clean NYC taxi data (PASS)
โ”‚   โ””โ”€โ”€ taxi-duplicate-rows/    # Duplicate detection (FAIL)
โ”œโ”€โ”€ src/gavel/
โ”‚   โ”œโ”€โ”€ server.py               # MCP stdio server (6 tools)
โ”‚   โ”œโ”€โ”€ cli.py                  # gavel CLI
โ”‚   โ”œโ”€โ”€ sandbox/                # spec loader, metrics, runner
โ”‚   โ”œโ”€โ”€ adapters/               # Postgres, WO, Dagster HTTP clients
โ”‚   โ””โ”€โ”€ collectors/             # manual_report evidence composer
โ”œโ”€โ”€ scripts/
โ”‚   โ”œโ”€โ”€ prove_it.py             # MCP boundary proof
โ”‚   โ”œโ”€โ”€ demo_live.py            # interactive demo
โ”‚   โ””โ”€โ”€ validate_manual_report_devint.sh
โ”œโ”€โ”€ docs/
โ”‚   โ”œโ”€โ”€ mcp-setup.md
โ”‚   โ”œโ”€โ”€ architecture.md
โ”‚   โ””โ”€โ”€ sandbox-platform.md
โ”œโ”€โ”€ Makefile
โ”œโ”€โ”€ .env.example
โ””โ”€โ”€ tests/                      # 21 tests, network mocked

๐Ÿ”ง Commands

make install           # venv + pip install -e ".[dev,live]"
make test              # pytest
make smoke             # lint + test + prove + offline sandboxes
make prove             # MCP boundary script
make sandbox-offline   # taxi-clean validation (expect PASS)
make sandbox-fail      # taxi-duplicate-rows (expect FAIL)
make demo              # interactive MCP demo
make mcp-config        # Cursor MCP JSON snippet
make lint              # ruff

CLI equivalents:

gavel list-sandboxes
gavel smoke
gavel sandbox taxi-clean --params '{"candidate_key":"trip-0000000"}'
gavel sandbox taxi-duplicate-rows --params '{"candidate_key":"trip-0000000"}'

๐Ÿ› ๏ธ Troubleshooting

  • MCP tools missing in Cursor: run make mcp-config, use absolute paths, restart Cursor.

  • Sandbox not found: check sandboxes/ directory structure and YAML syntax.

๐Ÿ“Š Taxi benchmark (v0.1)

Controlled fault injection on synthetic NYC taxi data โ€” verified benchmark record:

Signal

Result

Defect detection recall

100%

False-positive rate

0%

MCP call latency p50

~3 ms

These numbers apply to the published taxi fault model, not arbitrary production pipelines.

Roadmap

  • Sandbox eval platform with NYC taxi examples

  • Live adapters for production data warehouses

  • Signed evidence bundles ยท OTel traces ยท PR gates


๐Ÿ“„ License

MIT โ€” see LICENSE.

Available Tools

1 tool
validate_candidate_pipelineC

Compare a candidate taxi pipeline with a trusted reference and return evidence.

Inputs: seed, faults, row_count.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelogโ€ฆ

    TDQS

    C2.3/5.0

    Scored across 1 tool

    Disambiguation1/5

    With only one tool, there is no possibility of ambiguity between tools. However, the tool set is so minimal that the single tool must serve all purposes, which can lead to confusion about what it actually does despite its description.

    Naming Consistency3/5

    The single tool name 'validate_candidate_pipeline' follows a verb_noun pattern and is descriptive. However, with only one tool, it's hard to assess consistency across a set; it scores a neutral 3.

    Tool Count1/5

    One tool is extremely thin for a server named 'proofline-mcp'. The purpose seems to be validating pipelines, but a single tool likely cannot handle the full scope of validation, comparison, and evidence retrieval without additional supporting operations.

    Completeness1/5

    The server provides only one operation, which likely requires multiple steps (e.g., retrieving reference, comparing, generating evidence). Without additional tools for setup, retrieval, or even getting evidence details, the surface is severely incomplete for a meaningful workflow.

    Maintenance

    ActivityStale
    ResponsivenessNo recent community issues

    Related MCP Connectors

    Related MCP Servers

    • A
      license
      A
      quality
      D
      maintenance
      Enables AI agents to programmatically inspect, test, and validate other MCP servers by exposing MCP Workbench capabilities as structured tools. It supports automated test spec generation, execution, and detailed failure analysis to ensure server reliability.
      4
      7 npm
      Apache 2.0
    • A
      license
      Not graded
      quality
      B
      maintenance
      Enables AI agents to cross-verify candidate claims against caller-supplied source texts, flagging hallucinations, numerical drift, entity mismatches, contradictions, and unverified assertions. It returns sentence-level verdicts with matched evidence snippets and machine-readable factual grounding confidence scores, exposed over MCP stdio, HTTP REST, and A2A discovery routes.
      1
      MIT