readback
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@readback what was the average flight turnaround delay by airline last quarter?"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
readback
Every number my agent says is read back to the warehouse before you see it.
readback is an analyst agent over three of my own data builds, served as an MCP server at protocol revision 2026-07-28 and graded against answers that were already known. You ask a question about housing (doorstep), flights (turnaround) or a fictional bank (ledger); the agent finds the table, writes the SQL, and before the answer leaves, a gate made of plain code re-runs every number it is about to say. A number that does not read back is repaired once, then refused. The live site answers with floor (no model run beats the floor on the development split).
Status: 15 of the eighteen items in the definition of done are done. Not done yet: item 10, item 15, item 18. The checklist below says what each is waiting on.
The desk | |
The MCP endpoint | https://readback-mcp.vercel.app/mcp (verified) |
Connect a host | |
The scorecard |
What it does, in numbers
The warehouses. 163 published tables from doorstep, turnaround and ledger, fetched at their pinned tags and checked file by file against SHA-256 digests in the lock. Nothing is recomputed; the builds are the record.
The catalog. 1,105 measure columns, every one with a unit (1,105), 657 with a definition lifted from the build's own documents and cited to the line. The rest sit in a quarantine of 2,129 named rows rather than getting a definition I made up.
The protocol. 7 tools, each with an output schema; stateless; public hour long cache hints; discovery; trace context carried through the request metadata; conformance passed over in memory, stdio, streamable http.
The golden set. 150 questions whose answers were known before the agent existed, split 50 development and 100 test.
The floor, the planner with no model in it: 98.0% on the test split (interval 93.0% to 99.4%), every one of its 174 shipped numbers read back. It shares the measures table with the golden truth, so read its row as a ceiling for the families it was written for, not as a rival.
The red team. 15 of 15 injection cases passed on the floor. The first run passed fewer, and DECISIONS.md says what it caught.
Models graded so far: gemini, groq, openrouter, sambanova, ollama have not run yet. A provider's row appears when the live eval has recorded its cassettes; it never extrapolates from a partial run.
Related MCP server: mnemiq
How it works
doorstep, turnaround, ledger at v0.1.0 (fetched by digest, read only)
v catalog: units, cited definitions, synonyms, a lexical index, the quarantine
v DuckDB, three read only schemas, the SQL guard in front
MCP SERVER warehouses describe search_catalog run_sql published readback lookup
^ the agent is a client of its own server (in memory in tests, HTTP in production)
AGENT classify plan act gate repair refuse interrupt answer (LangGraph)
| router: gemini, groq, openrouter, sambanova, ollama, the floor; a budget each
v
EVALUATION golden set, graders in code, red team, cassettes TRACING one trace per turnThe agent calls the same seven tools any host gets, so a bug in a tool shows up in the eval and a bug in the protocol shows up in the separate client. The gate is code, because a model checking a model is a vote, and a vote is not a verification. The floor exists because a free tier can vanish in a week, and a product that only works with a key is a demo. The known truth is the manifests, because three builds spent a day each proving their numbers. ARCHITECTURE.md has the rest.
The skills, and where each lives
Skill | Where it lives |
MCP server design | packages/mcp_server, docs/protocol.md, results/conformance.json |
MCP client | packages/agent (the client), tests over the in memory transport |
Agent orchestration | packages/agent: the LangGraph graph, interrupts, the checkpointer |
Tool use and structured outputs | the Answer schema, the tool schemas, the output mode per provider |
Natural language to SQL, verified | packages/planner, run_sql, the readback tool |
Evaluation engineering | packages/evals, evals/golden, the scorecard |
Known truth and graders in code | catalog/manifest_index.parquet, packages/evals graders |
Red teaming and injection defense | evals/redteam, the gate, docs/security.md |
Observability | packages/tracing, docs/observability.md, the /traces page |
Cost control and model routing | providers/registry.yaml, the budget ledger, the cassettes |
Resilience without a key | the floor, the recorded session on the site |
Durable execution and human in the loop | the Postgres checkpointer, the interrupt on an ambiguous name |
Context engineering | prompts/registry.yaml, search then describe, tokens per question |
Semantic layer and data catalog | catalog/, the units, the synonym table, catalog/measures.yaml |
Security | the SQL guard, the read only connection, redaction, the identifier scan, the client budget |
APIs and persistence | packages/api, scripts/check_persistence.py |
Serverless deployment | Dockerfile.vercel, the cold start measurement |
Spec driven, agent assisted engineering | AGENTS.md, .claude/skills, docs/SPEC.md, .mcp.json, DECISIONS.md |
Interoperability standards | server.json, the connect page |
Frontend for agents | web/: the streaming desk, the chips, the trace viewer |
Statistics for evaluation | Wilson intervals, Benjamini-Hochberg, kappa |
Communication | report/briefing.md, the pushback paragraph on every page |
Reproducibility | the claim gate, the rederive, the cassettes |
The definition of done
Item | Status | Evidence |
1 | done | data/warehouses.lock.json pins 163 tables by digest; catalog/quarantine.json publishes every rule; data/PROVENANCE.md restates the terms |
2 | done | evals/golden holds 150 questions split 50 and 100; results/golden_first_ten.json has the hand checked ten |
3 | done | results/conformance.json: revision 2026-07-28, discovery, 7 tools with output schemas, every transport, trace context |
4 | done | tests/warehouse/test_guard.py, tests/mcp/test_tools.py (readback, lookup round trip), tests/api/test_client_budget.py |
5 | done | providers/registry.yaml; tests/providers; the floor graded on both splits in results/evals/runs/floor.json |
6 | done | tests/agent/test_graph.py; evals/registration.json holds the plan and prompt hashes |
7 | done | results/deploy/verification.json from AJAY, Windows, PowerShell 5.1.26100.9711, plain JSON-RPC |
8 | done | tests/api/test_http.py; scripts/check_persistence.py wrote results/persistence.json |
9 | done | results/traces/*.parquet; the /traces page renders the waterfall |
10 | not done | needs a model graded on the test split and the clarity judge, which need a key |
11 | done | the default is floor: no model run beats the floor on the development split |
12 | done | AGENTS.md, CLAUDE.md, .github/copilot-instructions.md, .mcp.json, three skills (results/skills.json), docs/SPEC.md, server.json, docs/connect.md |
13 | done | the desk on GitHub Pages, checked in a browser in results/live_check.json |
14 | done | traces, evals, catalog, connect, playbook and report pages, in results/live_check.json |
15 | not done | results/latency.json with the cold start; results/live_check.json |
16 | done | results/rederive.json: no drift |
17 | done | DECISIONS.md has 30 dated entries, 2 reversals, 6 protocol decisions; notebooks 01 to 03 |
18 | not done | coverage 85.4, mypy clean, ruff clean, commits 36, rederive no drift |
Run it
uv sync
uv run python scripts/fetch_warehouses.py
uv run readback-mcp # the server over stdio
uv run uvicorn readback_api.main:app # the API and /mcp over HTTP
uv run readback eval --provider floor # the golden set through the floorNo key is needed. AGENTS.md has the tests and the gates.
What the platform engineer would push back on
The known answers are this portfolio's own published numbers, so the eval measures whether the agent reads them correctly, not whether they are right; each build proved its own. The gate checks that a number agrees with the query it cites, not that the query was the right one, which the graders check separately. The live eval runs on free quotas, so a model's row covers the share of the test split its budget allowed, and says so. The floor only answers the families it was written for, and refuses the rest.
How this was built
I wrote the specification and made the calls; an AI pair (Claude) wrote much of the code with me, and the commits carry its co-author trailer. Every number above comes from a query or a test the repository runs, and the claim gate fails the build if this file says anything else.
Answers are drawn from the published tables of three portfolio builds by Ajay Mekala: doorstep (public housing data from FHFA, the Census Bureau, HUD, the CFPB and Freddie Mac), turnaround (public flight data from the Bureau of Transportation Statistics, the FAA and OurAirports, with weather from Open-Meteo under CC BY 4.0) and ledger (a fictional bank, Harborline, with public HMDA, CFPB, FDIC and macro series). No person appears in any table, and questions are redacted before a model sees them. Every number is read back to the warehouse before it is shown. This is a portfolio demonstration, not financial, travel or lending advice.
Available Tools
7 toolsdescribeARead-onlyIdempotent
Describe one table: its grain, every column with its type, unit and definition (cited to the build's definitions file where one exists), the manifest figures that read it, and its first five rows sorted by the grain. Call it for one table at a time, after search_catalog has named the table.
Inputs: table (required), warehouse (required). Returns: rows, grain, table, columns, +5 more
lookupARead-onlyIdempotent
Resolve a name to the way the warehouses spell it: a metro (doorstep), a carrier or airport (turnaround), an applicant group, an AML rule or a county (ledger), a year or a quarter. When a name matches more than one entity (Portland, Springfield, Columbus) the server asks the client to choose; a client that cannot be asked receives the candidates with needs_choice set, and must ask the person.
Inputs: kind (required), name (required). Returns: kind, query, resolved, statement, +3 more
publishedARead-onlyIdempotent
Read values from a warehouse's published manifest, the numbers its own documents print, by exact key or by a glob such as ch5.* (at most 50, sorted by key). Each value comes with its unit, its published rounding, its interval where one was published, and its display string.
Inputs: warehouse (required), key_or_pattern (required). Returns: values, matches, pattern, statement, +1 more
readbackARead-onlyIdempotent
Re-verify one number before you say it. Give the value, its unit, the display string you will show, and exactly one source: a result_id from run_sql in this request chain (with the column and row), a SQL statement to run, a manifest key, or a catalog key for a count of tables, rows, metrics or manifest values. The server re-runs or looks up the source, converts units only where the catalog allows (fraction and percent), and returns agree or disagree with the difference and the tolerance: half a unit of the published rounding for a manifest key, half a unit of your display rounding for a query. Every number in an answer must have an agreeing readback.
Inputs: claim (required). Returns: claim, reason, verdict, evidence, +7 more
run_sqlARead-onlyIdempotent
Run one read only DuckDB statement against one warehouse schema. Only a single SELECT or WITH runs; writes, ATTACH, COPY, INSTALL, LOAD, PRAGMA, SET, file reading functions and tables outside the named schema are refused with the clause named. Unqualified table names are qualified with the warehouse. The limit is capped at 500 rows and the statement stops after 10 seconds. Returns the statement as it ran, the columns with units where the catalog knows them, the rows, the row count before the limit, and a result_id to cite in readback.
Inputs: sql (required), warehouse (required), limit. Returns: sql, rows, limit, digest, +6 more
search_catalogARead-onlyIdempotent
Search the catalog of the three warehouses for tables, columns, metrics and published manifest values, ranked by a lexical index after the query is expanded through the synonym table (house price finds value_anchored). Returns the expanded query and up to k hits (at most 20), each with its unit and one definition sentence. Pass kind to search one kind only (definition finds the entries of each build's definitions document). Use it to find where a number lives before writing SQL.
Inputs: query (required), k, kind, warehouse. Returns: hits, query, expanded, statement
warehousesARead-onlyIdempotent
List the three warehouses (doorstep, turnaround, ledger) with the tag and commit each is pinned at, how many tables and rows each has, the period each covers, and the statement. Call this first when you do not know which warehouse a question belongs to.
Inputs: no named parameters. Returns: statement, warehouses
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
TDQS
Scored across 7 tools
Each tool has a distinct role in the workflow (orient via warehouses, find via search_catalog/lookup, inspect via describe, query via run_sql, cite via published, verify via readback), and descriptions explicitly state call order. The only soft overlaps are search_catalog also returning published manifest values and describe vs. search_catalog both surfacing definitions, but the ranked-search vs. exact-key distinction is articulated well enough to steer selection.
Conventions are mixed: verb_noun for search_catalog and run_sql, bare verbs for describe and lookup, and nouns/adjectives for warehouses, published and readback. Names are individually readable and domain-idiomatic, but there is no predictable pattern an agent can generalize from.
Seven tools is well within the sweet spot, and each one covers a genuine stage of the read-verify pipeline rather than duplicating another. Nothing looks like padding or an afterthought.
The lifecycle is covered end to end: discovery (warehouses, search_catalog, describe, lookup), retrieval (run_sql, published), and verification (readback). Minor gaps remain, such as no direct way to enumerate all tables or columns in a warehouse schema without first naming them through search, which forces extra round trips.
Maintenance
Related MCP Connectors
Verified answers from your data, with the query and rows behind them, or a decline, never a guess.
Your Databricks Lakehouse in natural language: run SQL on your SQL warehouses, track long-running qu
The all-in-one data stack for agents. Upload files, run SQL, evolve tables, and render charts.
Query, explore, and manage your databases through natural conversation.
Related MCP Servers
- AlicenseNot gradedqualityBmaintenanceEnables AI agents to query governed data warehouses through plain language, returning grounded answers with SQL, confidence grades, and signed receipts while enforcing access, testing, and audit controls.159 PyPI17Apache 2.0
- AlicenseAqualityBmaintenanceAnswers natural-language questions against Postgres, Oracle, Snowflake, Databricks, DuckDB and SQLite via db_read(question) and get_schema(), each separately grantable. The model only proposes SQL — shape, access, dialect and query plan are checked deterministically before any rows are read, and the server refuses rather than returning an unverified answer.381Apache 2.0
- AlicenseNot gradedqualityCmaintenanceEnables an LLM to inspect a relational database schema and answer financial questions by running validated, read-only SQL queries against a synthetic SQLite dataset.MIT
- AlicenseAqualityBmaintenanceEnables agents to query curated metrics through a registry-backed semantic layer, preventing ad-hoc SQL and enforcing consistent definitions. Every numeric claim is verified against the underlying query result before it is allowed to be sent.4MIT