This MCP server exposes a single tool to validate a candidate NYC taxi pipeline against a trusted reference, with optional fault injection.
Compare a candidate taxi pipeline to a trusted reference and get structured evidence.
Inject faults: duplicate_rows, missing_partition, timezone_shift, wrong_zone_mapping.
Configure the run with seed (default 42) and row_count (default 1000).
The provided schema defines no other tools (README describes additional Gavel tools not present here).
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type @ followed by the MCP server name and your instructions, e.g., "@proofline-mcp Validate my candidate pipeline with timezone_shift and give me the evidence report."
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
Prerequisites: Python 3.11+.
git clone https://github.com/Abhishek249/gavel-mcp.git
cd gavel-mcp
make install # creates .venv, copies .env.example → .env
make smoke # lint + 22 tests + MCP prove + offline sandboxes — no secretsTry it:
# Clean NYC taxi data (expect PASS)
gavel sandbox taxi-clean --params '{"candidate_key":"trip-0000000"}'
# Taxi data with duplicate rows (expect FAIL)
gavel sandbox taxi-duplicate-rows --params '{"candidate_key":"trip-0000000"}'Related MCP server: MCP Gatekeeper
Version | Focus | Try it |
v0.4 | Renamed from Proofline → Gavel ( |
|
v0.1 | Taxi benchmark + geospatial evidence MCP tools |
|
v0.2 | Sandbox spec + golden/candidate + metric DSL |
|
v0.3 | Live DevInt collector (Postgres + WO + Dagster) |
|
Full sandbox spec: docs/sandbox-platform.md · Architecture: docs/architecture.md
make mcp-config # prints JSON snippet for CursorSix tools:
MCP tool | Purpose |
| Discover sandboxes under |
| Golden vs candidate metrics (snapshot or inline rows) |
| Live DevInt evidence for one |
| Collect + validate in one call |
| Synthetic NYC taxi reference vs candidate |
| BYO orchestrator/worker/DB evidence envelope |
Restart Cursor after editing MCP config. Walkthrough: docs/mcp-setup.md.
Declarative sandboxes — YAML spec: components, golden/candidate datasets, metric rubric
Deterministic metrics — exact, jaccard, area_ratio, not_null, row_count, gte, lte
Silent-success detection — catches green orchestration with missing/wrong data
Shipped examples — NYC taxi clean data (PASS), duplicate rows (FAIL)
Structured evidence reports — every check: name, expected, actual, status, context
MCP stdio boundary — real client/server tests in CI (prove_it.py, benchmark)
Stack: Python, Pydantic, Shapely, PyYAML, MCP SDK.
sandbox.yaml → golden.json + candidate.json (file | inline | live)
↓
metric engine (deterministic)
↓
ValidationReport → MCP tool responseExample: NYC taxi duplicate-row detection
trip_id
→ load golden: 500 clean trips
→ load candidate: 600 trips (100 duplicates)
→ validate: row_count, key_uniqueness, fare_total → FAILgavel-mcp/
├── assets/
│ ├── gavel-readme-banner.png # README banner
│ └── gavel-icon.png # square icon (social preview)
├── sandboxes/
│ ├── taxi-clean/ # Clean NYC taxi data (PASS)
│ └── taxi-duplicate-rows/ # Duplicate detection (FAIL)
├── src/gavel/
│ ├── server.py # MCP stdio server (6 tools)
│ ├── cli.py # gavel CLI
│ ├── sandbox/ # spec loader, metrics, runner
│ ├── adapters/ # Postgres, WO, Dagster HTTP clients
│ └── collectors/ # manual_report evidence composer
├── scripts/
│ ├── prove_it.py # MCP boundary proof
│ ├── demo_live.py # interactive demo
│ └── validate_manual_report_devint.sh
├── docs/
│ ├── mcp-setup.md
│ ├── architecture.md
│ └── sandbox-platform.md
├── Makefile
├── .env.example
└── tests/ # 21 tests, network mockedmake install # venv + pip install -e ".[dev,live]"
make test # pytest
make smoke # lint + test + prove + offline sandboxes
make prove # MCP boundary script
make sandbox-offline # taxi-clean validation (expect PASS)
make sandbox-fail # taxi-duplicate-rows (expect FAIL)
make demo # interactive MCP demo
make mcp-config # Cursor MCP JSON snippet
make lint # ruffCLI equivalents:
gavel list-sandboxes
gavel smoke
gavel sandbox taxi-clean --params '{"candidate_key":"trip-0000000"}'
gavel sandbox taxi-duplicate-rows --params '{"candidate_key":"trip-0000000"}'MCP tools missing in Cursor: run make mcp-config, use absolute paths, restart Cursor.
Sandbox not found: check sandboxes/ directory structure and YAML syntax.
Controlled fault injection on synthetic NYC taxi data — verified benchmark record:
Signal | Result |
Defect detection recall | 100% |
False-positive rate | 0% |
MCP call latency p50 | ~3 ms |
These numbers apply to the published taxi fault model, not arbitrary production pipelines.
Sandbox eval platform with NYC taxi examples
Live adapters for production data warehouses
Signed evidence bundles · OTel traces · PR gates
MIT — see LICENSE.
Compare a candidate taxi pipeline with a trusted reference and return evidence.
| Name | Required | Description | Default |
|---|---|---|---|
| seed | No | ||
| faults | No | ||
| row_count | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description alone must disclose side effects and operational behavior. It only says 'compare' and 'return evidence,' but does not clarify whether the pipeline is executed, whether the operation is read-only, or what the evidence consists of.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence with no filler. It is concise, but the brevity contributes to important missing details about behavior and parameters.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given three parameters, no annotations, no output schema, and no sibling tools, the description is too thin. It does not define what the returned 'evidence' looks like, what 'trusted reference' means, or what the parameters control.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description does not explain seed, faults, or row_count, nor how they affect the comparison. The enum values for faults hint at their meaning, but the description adds no parameter-level semantics.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific action: compare a candidate taxi pipeline with a trusted reference, then return evidence. The verb and object are clear, though 'trusted reference' and 'pipeline' remain somewhat domain-specific and undefined.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance about when to use this tool versus alternatives, no prerequisites, and no mention of when it should not be used. The only implicit cue is that it is for validating a candidate pipeline.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
v0.1.0validate_candidate_pipelineScored across 1 tool
With only one tool, there is no possibility of ambiguity between tools. However, the tool set is so minimal that the single tool must serve all purposes, which can lead to confusion about what it actually does despite its description.
The single tool name 'validate_candidate_pipeline' follows a verb_noun pattern and is descriptive. However, with only one tool, it's hard to assess consistency across a set; it scores a neutral 3.
One tool is extremely thin for a server named 'proofline-mcp'. The purpose seems to be validating pipelines, but a single tool likely cannot handle the full scope of validation, comparison, and evidence retrieval without additional supporting operations.
The server provides only one operation, which likely requires multiple steps (e.g., retrieving reference, comparing, generating evidence). Without additional tools for setup, retrieval, or even getting evidence details, the surface is severely incomplete for a meaningful workflow.
MCP-native AI evaluation: rubric audits, eval suites, and proof reports for AI/LLM output.
Deterministic validation for AI-generated artifacts: JSON Schema, OpenAPI response, SQL syntax.
Tamper-evident proof creation and verification for AI agents via MCP, A2A, and REST.
Evidence-readiness MCP server: validate, audit, and score briefs, memos, and evidence packs.
