verified-analyst
Provides read-only SQL querying over CSV files loaded into DuckDB, including schema description, SELECT-only query execution, result caps, and timeouts.
This server does not currently have a deployable release on Glama. Follow the repository's setup instructions to install and configure it for your MCP client, including any required credentials and local dependencies.
Ask Claude, Cursor or any AI assistant about a spreadsheet and you usually get a confident answer with figures in it. Some of those figures were never calculated: the model pattern-matched them. verified-analyst closes that gap with a deterministic checker: every number in an answer must trace back to a value a real SQL query returned, or the answer is sent back for re-analysis.
It ships four ways:
what it is | for | |
MCP server |
| making Claude Desktop / Claude Code / Cursor check their own numbers |
Agent + CLI | a tool-calling analyst that cannot finish until its prose is grounded |
|
Agent skill | a | Claude Code, Codex and other agents that load skills, with no MCP setup |
Leaderboard | the same oracle-scored eval run across many models | answering "how often does each model invent numbers?" |

Real output of examples/catch_a_lie.py. The first draft sounds plausible and even names the wrong leader; the checker rejects it because neither figure was ever returned by a query.
flowchart LR
Q[Question] --> A[Model]
A -- run_sql --> D[(DuckDB, read-only)]
D -- rows --> A
A -- draft answer --> V{check_answer:<br/>every number in a result?}
V -- "no: 999 was never returned" --> A
V -- yes --> O[Answer with every number traced]Use it from Claude or Cursor (MCP)
pip install "verified-analyst[mcp] @ git+https://github.com/QasimJaffry/verified-analyst"
va data --out ~/va-demo # optional: a synthetic subscription-app dataset to try it on(Not on PyPI yet, so the install comes straight from GitHub.)
Claude Code:
claude mcp add verified-analyst -- va mcp --data /absolute/path/to/your/csv-folderClaude Desktop / Cursor (mcpServers in the client's config file):
{ "mcpServers": { "verified-analyst": { "command": "va", "args": ["mcp", "--data", "/absolute/path/to/your/csv-folder"] } } }Every .csv in the folder becomes a table. The server tells the assistant to compute figures in SQL and to call check_answer on its draft; check_answer returns which numbers were never returned by a query in the session:
{ "grounded": false, "ungrounded": ["999"], "numbers_checked": ["1234", "999"],
"note": "These numbers were not returned by any query in this session. Compute them with run_sql ..." }The data folder is fixed when the server starts, queries are SELECT-only with file access disabled, and the checker calls no model.
Related MCP server: duckdb-analytics-mcp
Use it as an agent skill, or from a script
npx skills add QasimJaffry/verified-analyst # installs skills/verified-numbersThe skill tells the agent to compute each figure in SQL and to run one stateless command on its draft before answering:
va check --data ./data \
--sql "SELECT platform, COUNT(*) AS users FROM users GROUP BY platform" \
--answer "iOS leads with 4,812 users."
# prints JSON with "ungrounded": ["4,812"] and exits 1Exit code 0 means every number came from one of the given queries, 1 means some did not, 2 means a query was rejected. That makes it usable in CI or any script, with no model involved.
The leaderboard: how often does each model invent numbers?
export OPENROUTER_API_KEY=... # one key, many models
va data
va leaderboard --models <model-id>,<model-id>,<model-id> --runs 3
# model ids: any tool-calling model from openrouter.ai/models (ids change often, so copy current ones)
# -> results/leaderboard/LEADERBOARD.md, leaderboard.json, index.html (+ per-model runs.csv)For each model it reports:
invented a number: share of runs whose first answer stated a figure none of its own queries returned (raw model behaviour, before the guard steps in)
answered the unanswerable: on questions the data cannot answer, how often it produced a number anyway
correct + grounded: final answers that match the oracle and pass the checker, with the guard on
saved by guard: runs that invented a number, got bounced back, then passed
median cost per question (from OpenRouter's price list) and latency
No real-model results are published yet. The first run will be committed under
results/with its date, versions and data hashes. Nothing on this page is a model's score.
Until then, here is the pipeline run against deterministic reference solvers (scripted, not real models). This proves the metrics measure what they claim (va leaderboard --models reference:oracle,reference:self_correct,reference:overconfident,reference:tampered_prose,reference:hallucinate --runs 1):

solver | invented a number | answered the unanswerable | correct + grounded | saved by guard |
| 0% | 0% | 100% | 0 |
| 100% | 0% | 100% | 21 |
| 19% | 100% | 81% | 0 |
| 100% | 0% | 19% | 0 |
| 100% | 100% | 0% | 0 |
Use it standalone
pip install "verified-analyst @ git+https://github.com/QasimJaffry/verified-analyst" # add [anthropic] for the Anthropic provider
va data # synthetic dataset in ./data
va ask "Which channel converts best after the trial?" --show-sql --chart-out chart.json
va eval --provider openrouter --model <model-id> --runs 5 --out results/my-modelchart.json is a Vega-Lite spec built only from rows the last query returned: the model picks the chart type and columns, never the data points.
What is different from the many "chat with your CSV" tools
A numeric-claim verifier on the final prose (
verify.py). Not an LLM judge: rounding-, percent- and suffix-aware matching of each number against returned cells. Arithmetic in prose is rejected on purpose, so the figure must beSELECTed.Oracle scoring, no LLM judge (
scoring.py). Expected answers come from executing each task's gold SQL. A run passes only if the value is correct and the prose is grounded.Questions it should refuse. 5 of 26 tasks have no valid numeric answer (missing data, or too vague). Inventing a number there is a failure.
The harness is tested against its own mutants. Deliberately wrong solvers must fail; a faithful one must pass 100%. CI enforces it.
Related open-source work: several DuckDB/SQL MCP servers give assistants query access, andon-verify re-checks numbers in finished reports, and text-to-SQL agents abound. This project's narrower claim is the in-loop, prose-level number check plus a refusal-aware oracle eval and a cross-model leaderboard.
Harness validation (no model involved)
python scripts/harness_table.py
reference solver | answerable passed (of 21) | refusal/clarify passed (of 5) | what it proves |
| 21 | 5 | a faithful solver passes everything |
| 0 | 5 | right prose, wrong |
| 0 | 5 |
|
| 0 | 0 | never queries, invents a number: verifier catches it; also fails refusals |
| 21 | 0 | answers vague/impossible questions with a number: scorer catches it |
| 0 | 3 | blanket refusal cannot game the eval (the 2 vague tasks need |
| 21 | 5 | a model the guard rescues still gets credit, and is counted as "saved by guard" |
How the verifier decides
answer says | query returned | verdict |
|
| grounded |
|
| grounded (rounded to the digits shown) |
|
| grounded |
|
| grounded |
|
| rejected (wrong at the second decimal) |
| rows | rejected: 3000 was never returned, so |
| n/a | dates and clock times are not claims |
Limit worth stating plainly: the verifier proves the numbers came from a query, not that the query answered the question. A real cell quoted for the wrong metric passes the verifier and is caught only by the oracle eval. See DESIGN.md.
SQL safety (does not rely on the model behaving)
exactly one statement, and DuckDB's own parser must classify it as
SELECTexternal access is switched off after the data loads and the configuration is locked, so
read_csv('/etc/passwd'),COPY,ATTACH,SET ...all fail100-row result cap and a 10 s timeout that cancels runaway queries
no code-execution tool (see DESIGN.md for why)
Eval design
Dataset: synthetic subscription mobile app (users, subscriptions, payments), seeded and reproducible. It is synthetic; nothing describes a real app.
Tasks: 26 in
evals/tasks.jsonl: lookup 5, aggregate 6, join 6, time 4, unanswerable 3, ambiguous 2.Reporting: pass rate with a Wilson 95% interval, latency as p50/p95 (never the mean), median steps and tokens, unverified-number rate, per-category breakdown,
runs.csvwith every run. Metadata (date, versions, dataset and task hashes) is written next to the numbers.Scale caveat: 26 tasks is a small, single-schema set, so treat the leaderboard as a pilot. Wilson intervals are reported precisely because they are wide.
Layout
src/verified_analyst/
agent.py loop, tools, verification gate (records the first draft before the guard)
verify.py numeric-claim extraction + grounding
mcp_server.py MCP tools for Claude / Cursor (core is transport-independent)
db.py DuckDB session, SELECT-only guard, schema description
charts.py Vega-Lite spec from returned rows
providers/ anthropic, openai-compatible (stdlib HTTP)
datagen.py seeded synthetic dataset
evals/ tasks, oracle scoring, stats, runner, reference solvers, leaderboard
evals/tasks.jsonl
tests/ 129 tests, incl. a real MCP stdio round trip
skills/ verified-numbers agent skill (uses `va check`)Limitations
Verified-by-query is not verified-correct: a real cell quoted for the wrong metric passes the checker and is caught only by the oracle eval.
Small integers can be grounded by coincidence because rounding tolerance is real.
In MCP mode the checker only helps if the assistant calls it; the server's instructions ask it to, but clients decide.
One schema and 26 tasks: the leaderboard is a pilot. More schemas and tasks are the next step.
Provider code is tested at the wire-format level, not against live APIs in CI.
MIT licensed.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
This server cannot be deployed
Maintenance
Related MCP Connectors
Query, join, profile, clean and convert CSV/JSON/Parquet with server-side DuckDB over MCP.
Text-to-SQL MCP server: read-only queries on PostgreSQL, MySQL and SQL Server with your real schema
ParseHub.dev is a hosted document-to-records service with a remote MCP server. Agents can upload a CSV, PDF, or Word file, summarize and analyze CSV data (filter, sort, group_by, and more), read text from PDF and Word files, extract attendees, agenda items, and action items from meeting notes, ask a question about an uploaded document, and store files as database records to list and read later. Each account sees only its own files and records. **Endpoint:** `https://parsehub.dev/mcp` (Streamable HTTP, stateless) **Auth:** API key in the `X-API-Key` header (keys start with `mcp_`), or `Authorization: Bearer <API key or JWT>`. Every request without credentials returns 401. **Plans:** Free (100 API requests/month, 1 document upload), or 5,000 requests/month on the $9/month or $42/year plan. Every MCP request counts toward the allowance. **Tools:** upload_csv, summarize_csv, analyze_csv, upload_document, read_document, extract_meeting_notes, ask_document, database_status, list_database_documents, get_database_document, store_document, extract_stored_meeting_notes, export_database (administrators only). Not affiliated with ParseHub (parsehub.com), the web-scraping tool. ParseHub.dev does not scrape websites. By Poietes LLC. Docs: https://parsehub.dev/guide#connect-via-mcp
Query your warehouse or a CSV with Claude/ChatGPT over MCP, governed by table-level ACL + audit.
Related MCP Servers
- AlicenseAqualityNot gradedmaintenanceAn MCP server that enables AI assistants to load, query, and analyze local CSV files using tools for filtering, aggregation, and grouping. It provides capabilities to describe schemas, calculate statistics, and sample data directly from CSV files.6-
- AlicenseAqualityDmaintenanceA read-only DuckDB MCP server offering context-efficient analytics tools (list_datasets, describe_table, profile_column, explain, query) with a semantic layer for business rules, security guards, and disclosed truncation to help LLMs produce correct answers while minimizing token usage.5MIT
- AlicenseAqualityCmaintenanceA minimal MCP server that provides a persistent DuckDB SQL engine to AI assistants, enabling natural-language querying of CSVs, Parquet, and cloud data with 12 tools and optional read-only mode.12MIT
- AlicenseNot gradedqualityDmaintenanceMCP server that turns coding agents into data analysts: attach CSV/Parquet/JSON/XLSX or databases, profile data, run read-only SQL, and render Vega-Lite charts as SVG files.1MIT