Skip to main content
Glama
dataparity

dataparity

Dataparity

Cross-database data diff and reconciliation for AI agents, with a portable row hash.

Compare two tables — in the same database or in two different ones — and get back what changed, which columns changed, and which slice of your data the changes concentrate in. Runs as a CLI, an MCP server, and a library. MIT licensed.

npm CI License: MIT MCP Glama Node

What it does · Why the hash is portable · Install · CLI · MCP · Library · How it works · Status


What it does

Give it two tables and a key, and it tells you what changed:

HIGH  3 difference(s) found across 5 row(s) (60.00% of the left side).

left   pg :: public.orders_prod  (5 rows)
right  mysql :: analytics.orders_staging  (5 rows)
strategy  checksum   keys  id

added 1   removed 1   modified 1   unchanged 3

where the changes are concentrated:
  country = DE  4 change(s)  80.0% of changes  6.7x over-represented

sample changes:
  removed  key=2
  modified key=3  amount
  added    key=6

signals:
  - 1 row(s) exist only on the left
  - 1 row(s) exist only on the right
  - 1 row(s) differ in at least one column

Three things make it different from a SELECT EXCEPT you write yourself:

  1. It works across two different databases. Postgres ↔ MySQL, Postgres ↔ DuckDB, Parquet ↔ Postgres. Both sides compute the same canonical row hash inside their own engine, so only hashes cross the wire — never row data.

  2. It says where the changes are, not just that there are changes. A diff that reports "432 rows differ" is nearly useless. A diff that reports "98.6% of the modified rows are German orders" tells you the VAT rule landed and nothing else did.

  3. It is built for agents. An MCP server for source discovery, schema inspection, diffing, and persistent run history, with a stable JSON result contract and a risk verdict (none / low / medium / high / critical) that a coding agent can act on.

Related MCP server: PgVouch

Why the hash is portable

Comparing rows across engines means agreeing on what a value is. 0.1 in Postgres and 0.1 in MySQL are both the double closest to one tenth, and 0.1 + 0.2 is not 0.3. A boolean is true in Postgres and 1 in MySQL. A timestamp has microsecond precision in one engine and second precision in another. A NULL is not an empty string.

Dataparity solves this with two rules, applied identically in every engine:

  • Every value is tagged. NULL renders as N; every other value renders as V followed by its normalized text. NULL and '' can never be confused, and a real value can never be mistaken for NULL.

  • Every value is length-prefixed before it is joined into a row hash. A text value containing the field separator cannot shift a column boundary and forge a match.

Floats are scaled to a fixed number of decimals in exact decimal arithmetic before they are rendered, so 0.1 + 0.2 and 0.3 agree. Timestamps are normalized to UTC and truncated to seconds. Booleans render as true/false everywhere.

The result is an MD5 row hash that is byte-identical across engines. This is not a claim in a README — it is asserted by test/integration/canonical.test.ts, which writes the same awkward values into Postgres, MySQL and DuckDB and requires the hashes to match:

docker run -d --name pgdiff -e POSTGRES_PASSWORD=test -e POSTGRES_DB=difftest -p 55432:5432 postgres:16-alpine
docker run -d --name mydiff -e MYSQL_ROOT_PASSWORD=test -e MYSQL_DATABASE=difftest -p 53306:3306 mysql:8
npm run test:integration

Those tests need the databases. When one is unreachable they report themselves as skipped rather than passed, so a green run never means "nothing was tested" — and CI fails the job if anything was skipped.

Install

npm install -g dataparity

Or run it without installing:

npx dataparity --help

Requires Node.js 22.16 or newer.

CLI

Configure your sources once, in ~/.dataparity/config.json (or point DATAPARITY_CONFIG at another path):

{
  "sources": [
    { "name": "prod",    "type": "postgres", "connectionString": "postgres://..." },
    { "name": "staging", "type": "postgres", "connectionString": "postgres://..." },
    { "name": "mysql",   "type": "mysql", "host": "127.0.0.1", "port": 3306,
                         "user": "readonly", "password": "...", "database": "analytics" },
    { "name": "local",   "type": "sqlite", "path": "data/app.db" },
    { "name": "files",   "type": "duckdb", "files": ["data/orders.parquet"] }
  ]
}

Then diff:

# Two tables in the same database
dataparity diff \
  --left prod --right staging \
  --left-table public.orders_prod --right-table public.orders_staging \
  --key id

# Two tables in two different databases
dataparity diff \
  --left prod --right mysql \
  --left-table public.orders_prod --right-table orders_staging \
  --key id --json

# Explore first
dataparity tables --source prod
dataparity describe --source prod --table public.orders_prod

Exit codes follow the GNU diff convention: 0 identical, 1 differences found, 2 error. That makes it drop straight into a CI check.

Option

Meaning

--left, --right

Source names from the config file

--left-table, --right-table

Tables to compare

--key

Comma-separated key columns used to match rows

--columns

Columns to compare (default: every shared column)

--where

A WHERE expression applied to both sides

--max-changes

Maximum row changes to report (default 100)

--max-attributions

Maximum attribution segments to report (default 5)

--sample

Compare only a sample; the result is marked as sampled

--sample-limit

Maximum rows to sample per side (default 100000)

--bucket-count

Explicit bucket count for cross-database diffs

--json

Machine-readable output

Credentials can come from the environment instead of the file, so they never have to be written to disk:

export DATAPARITY_PROD_CONNECTION_STRING='postgres://...'
export DATAPARITY_MYSQL_PASSWORD='...'

MCP

claude mcp add --scope user dataparity -- npx -y dataparity mcp

Or in .cursor/mcp.json:

{
  "mcpServers": {
    "dataparity": { "command": "npx", "args": ["-y", "dataparity", "mcp"] }
  }
}

Tools:

Tool

What it does

dataparity_list_sources

The configured sources and their engine types

dataparity_list_tables

Table names in a source

dataparity_describe_table

One table's columns, types and nullability

dataparity_diff_tables

Compare two tables and report what changed

dataparity_list_diff_runs

List recent saved diff runs

dataparity_get_diff_run

Retrieve a saved full diff report by run id

dataparity_delete_diff_run

Delete a saved diff report

dataparity_create_baseline

Capture a named table baseline

dataparity_list_baselines

List saved baselines

dataparity_compare_baseline

Check a table for drift from a saved baseline

dataparity_delete_baseline

Delete a saved baseline

Use dataparity_list_tables to find candidate tables, then dataparity_describe_table to inspect column details before choosing diff keys. Tools take source names, never connection strings — credentials never pass through the model. Every tool is read-only with respect to configured databases; history and baselines are stored in Dataparity's local SQLite store. Successful diffs save the complete report, including any row-level keys and values returned by the diff. Baselines store schema and bucket-level canonical hashes, not table row values. Delete tools remove saved entries.

The store defaults to ~/.dataparity/history.sqlite; set DATAPARITY_HISTORY_PATH to choose another path. In Docker or hosted environments, mount a persistent volume at that path if history and baselines must survive container replacement. Without a persistent volume, the store may be lost when the container is recreated.

Library

import { diffTables, PostgresSource, MysqlSource } from 'dataparity';

const prod = await PostgresSource.connect({ connectionString: process.env.PROD_URL! });
const staging = await MysqlSource.connect({
  host: '127.0.0.1', port: 3306, user: 'ro', password: '...', database: 'analytics',
});

const report = await diffTables(prod, 'public.orders', staging, 'orders_staging', {
  keyColumns: ['id'],
});

if (report.risk.level === 'critical') {
  console.error(report.risk.signals.join('\n'));
  process.exit(1);
}

How it works

Same database → a single in-database join. The three-way classification (added, removed, modified) is one anti-join plus one outer join, entirely inside the database. Row data never leaves it, and the result is exact.

Different databases → canonical hashes, compared in buckets. Each side computes a per-bucket count and two independent checksums inside its own engine. Only buckets whose aggregates disagree are expanded down to the row level, so a billion-row table that differs in a few rows transfers a few hashes, not a billion rows.

Attribution. For each candidate column, the distribution of that column among the changed rows is compared to its distribution across the whole table. A value that is much more common among the changed rows than expected points at the affected slice. The score is descriptive ("over-represented"), never causal.

Risk. The verdict weighs the size of the change against the schema difference. A removed or retyped column alongside added or removed rows is critical; a small percentage of changed rows with no schema change is low.

Supported engines

Engine

Same-db

Cross-db

Notes

PostgreSQL

yes

yes

MySQL

yes

yes

SQLite

yes

yes

Via node:sqlite, read-only by default

DuckDB

yes

yes

Reads Parquet, CSV and JSON directly

Parquet / CSV / JSON

—

yes

Through DuckDB

A boolean and a 0/1 integer are treated as the same logical type, so a boolean column in Postgres compares correctly against the integer column SQLite has no choice but to use.

Status

Early. The core is solid and tested against live Postgres, MySQL and DuckDB, but the surface is small and the edges are sharp.

What works today

  • Same-database diffs with exact column-level detail

  • Cross-database diffs between Postgres, MySQL, SQLite and DuckDB

  • Schema comparison, change attribution, risk verdicts

  • CLI, MCP server and a typed library API

  • Read-only by construction; credentials never reach the model

  • Cross-database diffs localise which columns changed, by comparing a per-column hash computed inside each engine — so the column detail survives the hash comparison without any value crossing the wire

Known limits

  • Cross-database diffs report which columns changed, but not the before/after values: only hashes cross the wire, so the value itself never leaves either engine. A same-database diff carries the values too.

  • JSON columns are compared as text, so key order differences read as changes. Compare them structurally instead.

  • Floats and decimals are compared to six decimal places. Beyond that, two engines may disagree on the last bits.

  • The where clause is a raw SQL fragment. It is passed to your engine, not to the model, but treat it as trusted input.

  • No monitors, no lineage graph, no dbt integration, no UI.

Contributing

docs/design.md is the source of truth for the scope and the key decisions behind them. Read it before changing behaviour — it records the reasoning, not just the outcome.

The workflow is:

npm ci
npm test                   # unit tests
npm run test:integration   # needs live Postgres + MySQL; skips if absent
npm run test:audit         # every engine x every option x every error path
npm run lint
npm run typecheck
npm run build

The integration suite needs Postgres 16 on :55432 and MySQL 8 on :53306:

docker run -d --name pgdiff -e POSTGRES_PASSWORD=test -e POSTGRES_DB=difftest -p 55432:5432 postgres:16-alpine
docker run -d --name mydiff -e MYSQL_ROOT_PASSWORD=test -e MYSQL_DATABASE=difftest -p 53306:3306 mysql:8

License

MIT. See LICENSE.

Available Tools

11 tools
dataparity_compare_baselineA

Compare a configured table with a saved baseline for the same source and table. Returns schema differences and changed hash buckets; use dataparity_diff_tables for row-level details against a live table.

Inputs: name (required), table (required), source (required).

dataparity_create_baselineA

Capture a named baseline for a configured table. The local store keeps schema and bucket-level canonical hashes, not table row values. A later dataparity_compare_baseline checks for data or schema drift.

Inputs: name (required), table (required), source (required), key_columns (required).

dataparity_delete_baselineB

Permanently delete a saved table baseline by name.

Inputs: name (required).

dataparity_delete_diff_runA

Permanently delete a saved diff report from local history by run id.

Inputs: run_id (required).

dataparity_describe_tableA

Return detailed metadata for one table, including each column name, type, and nullability. Use this after dataparity_list_tables to choose key and comparison columns.

Inputs: table (required), source (required).

dataparity_diff_tablesA

Compare two tables and report what changed. Works across different databases: both sides compute the same canonical row hash, so only hashes are compared, never raw data. Returns added, removed and modified rows, schema differences, a risk verdict, and the segments of data where the changes concentrate.

Inputs: left (required), right (required), left_table (required), key_columns (required), +7 more.

dataparity_get_diff_runA

Retrieve a previously saved full diff report by the run id returned from dataparity_diff_tables or dataparity_list_diff_runs.

Inputs: run_id (required).

dataparity_list_baselinesB

List saved table baselines, optionally filtered by configured source.

Inputs: source.

dataparity_list_diff_runsA

List recent saved diff runs, newest first. Successful dataparity_diff_tables calls are saved locally; use dataparity_get_diff_run to retrieve a full report.

Inputs: limit.

dataparity_list_sourcesA

List the configured data sources that can be diffed. Returns each source name, engine type and label.

Inputs: no named parameters.

dataparity_list_tablesA

List table names in a configured source. Use this to find candidate tables, then call dataparity_describe_table when you need column names, types, or nullability before choosing diff keys.

Inputs: source (required), schema.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    TDQS

    A3.8/5.0

    Scored across 11 tools

    Disambiguation4/5

    Each tool has a distinct resource+action pairing, and the descriptions actively differentiate the closest pair (compare_baseline vs diff_tables by explaining baseline-drift vs live row-level diff). Minor potential confusion exists between the baseline-read tools and the diff-run-read tools, but the naming and descriptions resolve it.

    Naming Consistency5/5

    All 11 tools use a uniform `dataparity_` prefix followed by a snake_case verb_noun pattern (list_tables, describe_table, create_baseline, delete_diff_run). No mixed conventions or vague one-word verbs.

    Tool Count5/5

    11 tools sit squarely in the well-scoped 3-15 range. Every tool maps to a clear operation in the baseline/diff lifecycle with no redundant entries.

    Completeness4/5

    Solid coverage: table discovery, baseline create/list/compare/delete, diff execution plus run list/get/delete, and source listing. Minor gaps exist — no direct get/describe for a single baseline's metadata and no rename/update operation — but agents can work around these via list_baselines and compare_baseline.

    Maintenance

    ActivityMaintained
    ResponsivenessNo recent community issues

    Related MCP Connectors

    Related MCP Servers

    • A
      license
      Not graded
      quality
      D
      maintenance
      Enables AI agents to compare database tables using read-only tools, with credentials managed locally and never exposed to the model.
      MIT
    • A
      license
      Not graded
      quality
      B
      maintenance
      Exposes read-only tools that let AI assistants detect schema drift, analyze lock impact, verify data with checksums, and plan safe PostgreSQL migrations that are validated by deterministic checks and shadow runs.
      37 npm
      Apache 2.0
    • A
      license
      A
      quality
      B
      maintenance
      Gives an AI agent a database's tables, columns, value lists, null shares and joins from a dbprint snapshot committed to your repo, with no database connection or credentials. Tools list and search tables and columns, render prompt-ready context for a table, resolve a filter value to its stored spelling, and report what changed since the last snapshot.
      4
      7
      195 PyPI
      4
      Apache 2.0
    • A
      license
      A
      quality
      B
      maintenance
      Enables AI agents to inspect tables, estimate bytes/cost, rehearse SQL on 1%→3%→10% samples, predict full-scale results, and only run a query inside a budget across DuckDB, BigQuery, Snowflake and Postgres. Also supports impact/lineage analysis, schema drift detection, key-based data diffs, SQL translation, incremental-model advice and cheap data-quality checks.
      16
      209 PyPI
      2
      MIT