This server lets AI agents design and generate synthetic, relational datasets with exact declared outcomes (aggregates, rates, referential integrity), preview/validate schemas, and seed real databases.
List built-in domains – shows the 18 supported business domains (SaaS, fintech, healthcare, etc.) with trigger keywords for natural-language generation.
Preview story interpretation – parses a plain-English dataset description and returns detected domain, confidence, locale, scale, and table previews without generating rows.
Inspect schema – returns the full underlying schema (tables, columns, types, distributions, relationships) for a story, useful for authoring YAML or reviewing structure.
Generate dataset from a sentence – turns a story into multi-table CSV files on disk, returning paths, row counts, and row samples; supports row counts, seeds, and output location.
Generate dataset from a schema – the primary tool: accepts a custom dict schema with table/column specs, constraints, correlations, outcome curves, rate curves, state machines, time-series autocorrelation, missingness, exact incidence, rollups, and more; guarantees FK integrity, exact declared aggregates, and seed-reproducible output, and returns an integrity proof plus samples.
Seed a real database – reads a Postgres/SQLite schema, generates matching data, and inserts it respecting FK order; plans by default (non-destructive), with explicit apply/truncate/append modes for safety.
Validate a misata.yaml – runs structural JSON Schema validation plus semantic checks (probability sums, FK references, no cycles, curve column existence) before generation.
You declare the outcome. Misata generates the data that provably matches it.
Realistic, relational rows that hit exact revenue curves, fraud rates, referential integrity, and statistical structure. From a sentence, YAML, or your database. No real data, no ML model.
Prefer no code? Try Misata Studio, the no-code synthetic data generator: design a schema on a canvas or describe your dataset in plain English, then generate it in your browser. Same engine, same integrity proof.
Most synthetic-data tools learn from a real dataset and imitate it. Misata works the other way: you declare the outcome you want: "monthly revenue rises from $50k to $200k," "fraud is 3% in Q1 rising to 8% by Q4," "every customer's total_spent equals the sum of their orders", and Misata generates individual rows whose aggregates hit those targets exactly, with full referential integrity, from no source data at all.
This is outcome-conformant generation. The mechanism is formalised in an arXiv preprint (2606.08736): a closed-form method that satisfies declared aggregates to $0.00 error, where off-the-shelf imitation synthesisers trained on the same data miss by 74–86%. Every run can also emit an Oracle report, a proof bundle covering referential integrity, constraints, temporal consistency, and reproducibility.
It generates from a plain-English description, a YAML schema, or an existing database schema. No machine-learning model is required. No real data is needed.
Built for:
Known-answer testing: declare the KPI, generate the data, then assert your dbt, Spark, or SQL transform returns exactly that number. A pipeline test with a ground truth, before any real data exists
Database seeding: fill dev and staging environments with production-like data
Integration tests: relational fixtures with FK integrity across every table
Demos and prototypes: realistic numbers, names, and distributions, no PII
BI and dashboard development: data shaped like your real domain before launch
Statistical method validation: longitudinal, grouped, and multi-site datasets that pass mixed-effects models, ICC tests, and autocorrelation checks
Misata works in two modes, and the difference is the whole point:
Declare (the default, no data required). You state the schema and the outcomes you want, exact revenue curves, fraud rates, rollups, constraints, and Misata generates rows from scratch that conform to them. Use this when you do not have real data, or when you need a known answer to test a pipeline, dashboard, or demo against.
Mimic (when you already have data). Point misata.mimic() at a real CSV and get a synthetic twin that matches its distributions and correlations but contains none of the original rows, with fidelity_report and privacy_report to measure the result. Use this for privacy-safe copies of data you already hold.
Most synthetic-data tools only do the second, learning from a real dataset and imitating it. Misata leads with the first: you declare the answer, then generate the data around it.
Related MCP server: MCP Data Visualization Server
Misata's exact-aggregate engine is backed by an arXiv preprint:
Declarative Outcome-Conformant Synthesis: Exact, Closed-Form Specification Satisfaction and a Conformance Benchmark
Muhammed Rasin, arXiv:2606.08736 (2026)
https://arxiv.org/abs/2606.08736v1
The paper formalises the core claim: when you declare "SaaS MRR from $50k in January to $200k in December", Misata generates individual transactions whose monthly totals match the declared curve to exactly $0.00 error, not approximately, but provably, via a closed-form Gamma conditional-sum mechanism (Lukacs' characterisation). Off-the-shelf imitation synthesisers trained on the very same data miss the declared monthly aggregate by 74–86%; Misata reaches exactly 0.
The paper also introduces SpecBench: the first benchmark measuring conformance to analytical outcomes for cold-start relational synthesis. Misata is the reference implementation.
@article{rasin2026declarative,
title = {Declarative Outcome-Conformant Synthesis: Exact, Closed-Form
Specification Satisfaction and a Conformance Benchmark},
author = {Rasin, Muhammed},
year = {2026},
url = {https://arxiv.org/abs/2606.08736v1}
}pip install misataOptional extras:
pip install "misata[llm]" # multi-provider LLM schema generation
pip install "misata[documents]" # PDF output via weasyprint
pip install "misata[advanced]" # SDV/CTGAN statistical synthesis
pip install "misata[mcp]" # MCP server, expose Misata to Claude, Cursor, and other AI agents
pip install "misata[evalpack]" # evalpacks: verified eval databases for data agents (DuckDB)Misata ships an Agent Skill, so Claude Code and anything else that reads
SKILL.md knows which entry point fits which request and what is worth
declaring:
/plugin marketplace add rasinmuhammed/misata
/plugin install misata@misataThe skill drives the CLI, so pip install misata is still required. There is
also an MCP server (pip install "misata[mcp]") and a Claude Desktop extension
in mcpb/.
Misata ships a built-in Model Context Protocol server with a clear division of labour: the AI agent designs the schema, Misata guarantees the math. Agents are good at knowing that a veterinary clinic needs a species column; Misata is good at making 50 000 rows where every foreign key resolves, every roll-up reconciles to the cent, and the same seed reproduces byte-identical output. The primary tool, generate_from_schema, accepts the agent's schema dict and returns the data plus an integrity proof: per-relationship orphan counts the agent can show you.
1. Install:
pip install "misata[mcp]"2. Add to Claude Desktop (~/Library/Application Support/Claude/claude_desktop_config.json):
{
"mcpServers": {
"misata": {
"command": "misata-mcp"
}
}
}Restart Claude Desktop. Then just ask:
"Generate a fintech dataset with 1 000 customers, payments, and a 2% fraud rate."
"Design a clinical-trials database (sites, patients, visits, adverse events) and generate 100k rows."
"I need SaaS data: MRR from $50k in January, doubled by December, with a Q3 slump."
The agent designs whatever tables the request needs (any domain; it isn't limited to Misata's built-ins), calls Misata, writes CSVs to disk, and reports back with previews and the verified integrity summary. See the MCP guide for Cursor/Windsurf/Zed setup and all six available tools.
mcp-name: io.github.rasinmuhammed/misata
misata generate \
--story "Brazilian fintech with R$ payments, CPF verification, and 3% fraud" \
--rows 1000 \
--output-dir ./demo_data
# Writes CSVs plus:
# ./demo_data/oracle_report.jsonimport misata
# One sentence → multi-table DataFrame dict
tables = misata.generate("A SaaS company with 5k users, monthly subscriptions, and 20% churn")
print(tables["users"].head())
print(tables["subscriptions"].head())# Or from the CLI
misata generate --story "A SaaS company with 5k users and 20% churn" --rows 5000The Oracle report is Misata's proof layer. It separates hard guarantees from advisory realism checks so generated data can be trusted in CI, demos, notebooks, and research comparisons.
Guaranteed checks:
referential integrity across configured relationships
requested row-count fulfillment
schema validation and configured constraints
deterministic reproducibility when a seed is set
Advisory checks:
quality score and plausibility warnings
privacy heuristics
schema-vs-output fidelity score
locale/domain fit for countries, cities, phone prefixes, and national IDs
data-card metadata
import misata
schema = misata.parse("Brazilian fintech with CPF verification", rows=1000)
tables = misata.generate_from_schema(schema)
oracle = misata.build_oracle_report(tables, schema, seed=schema.seed)
print(oracle["passed"])
print(oracle["advisory"]["locale_domain_fit"]["locale"])Point misata.mimic() at a real dataset and get a synthetic twin that matches every column's distributions but contains none of the original rows. No schema authoring, no config.
import pandas as pd
import misata
real = pd.read_csv("titanic.csv")
twin = misata.mimic(real, rows=2000, seed=42, table_name="passengers")["passengers"]The profiler handles the columns that break other tools:
Alphanumeric code columns (Ticket "A/5 21171", Cabin "C85", SKUs, reference numbers) are detected by their character-class shape and reproduced structurally, same shapes in the right proportions, entirely new values, zero verbatim leak from the source. They no longer fall through to prose text generation.
Floats keep their cents. A Fare of 7.25 generates as 7.25-shaped values. The profiler infers decimal places from the data; semantic quantization (charm pricing) never fires on mimicked columns.
Distributions are fit from the data. Skewed-positive columns get lognormal; constant columns get a uniform stub; everything else gets normal. Categorical columns with fewer than 50 values carry their real frequencies.
# Verify: no verbatim rows can leak through
shared = [c for c in real.columns if c in twin.columns]
overlap = pd.merge(real[shared].astype(str), twin[shared].astype(str), how="inner")
assert len(overlap) == 0tables = misata.generate("A fintech startup with 10k customers, fraud rate 3%, and IBAN accounts")Misata reads the story, infers domain (fintech), scale (10 000 rows), and column semantics (fraud flag, IBAN format), no schema authoring needed.
A sentence is read by a recogniser that handles a fixed set of phrasings. Anything it cannot turn into a declaration is named in a warning rather than dropped, so you always know what took effect. When a prompt needs to be exact, write it as a spec instead.
Prose that declares tables, row counts, columns and rules is not a story, and guessing at it is the wrong tool. Misata detects that shape and parses it directly, so "exactly 4000" means exactly 4000. No model, no inference, no rewriting.
Table 1: accounts
Rows: exactly 600
Columns:
account_id
company_name
plan
seats
signed_up_on
plan must only be:
Starter
Professional
Enterprise
seats must be 1 to 120
signed_up_on must be 2023-01-01 to 2023-12-31
Table 2: invoices
Rows: exactly 3200
Columns:
invoice_id
account_id
amount
issued_on
account_id must match values from accounts table
amount must be 120 to 8500
issued_on must be 2024-01-01 to 2024-12-31
Revenue curve on invoices.amount by issued_on:
Jan 180000
Feb 195000
Mar 210000tables = misata.generate_from_schema(misata.parse(open("spec.txt").read()))What the spec guarantees:
You write | You get |
| 3200 rows, not about 3200 |
| a foreign key with zero orphans |
| those values and no others |
| every row inside the bound |
| dates inside that window |
| each month lands on its figure to the cent |
Rules can sit inside a table's block or in one section at the end; either way they attach by column name. A column ending _on, _at, _date or _for is generated as a date. Anything the parser cannot translate is listed back to you, never guessed at.
misata init # scaffolds misata.yaml in the current directory
misata generate # reads misata.yaml automatically# misata.yaml
name: my-app
seed: 42
tables:
users:
rows: 1000
columns:
user_id: { type: int, unique: true }
email: { type: text, text_type: email }
plan: { type: categorical, choices: [free, pro, enterprise] }
orders:
rows: 5000
columns:
order_id: { type: int, unique: true }
user_id: { type: foreign_key }
amount: { type: float, min: 5.0, max: 500.0 }
relationships:
- "users.user_id → orders.user_id"
constraints:
- name: amount_above_cost
table: orders
type: inequality
column_a: amount
operator: ">"
column_b: costschema = misata.load_yaml_schema("misata.yaml")
tables = misata.generate_from_schema(schema)from misata import schema_from_db, generate_from_schema, seed_database
# Introspect the live schema: no manual column definitions
schema = schema_from_db("postgresql://user:pass@localhost/myapp")
tables = generate_from_schema(schema)
# Seed it back: insert order respects FK dependencies automatically
report = seed_database(tables, "postgresql://user:pass@localhost/myapp_dev")
# SeedReport: seeded 6 tables, 47,300 rows in 1.2s# One-command workflow
misata init --db postgresql://user:pass@localhost/myapp # writes misata.yaml
misata generate --db-url postgresql://user:pass@localhost/myapp_dev --db-createSQLAlchemy models are supported too:
from misata import seed_from_sqlalchemy_models
from myapp.models import Base
report = seed_from_sqlalchemy_models(Base, db_url="sqlite:///test.db", row_count=500, create_tables=True)cd my-dbt-project && misata dbt-seedNo story, no config. Misata reads the properties YAML your project already
has and generates seed CSVs that satisfy it: relationships tests become
foreign keys with guaranteed integrity, accepted_values become the exact
category pools, unique and not_null become hard constraints, and
data_type plus column-name semantics decide the rest. Then:
dbt build # seed + run + test — the tests you already wrote, passing on day zeroBoth the legacy inline test syntax and the dbt 1.9+ arguments: nesting are
understood. Tests Misata can't translate (dbt_utils.*, custom generics) are
listed in the output rather than silently guessed at.
cd my-app && misata prisma-seedReads the schema.prisma your app already maintains: @relation becomes foreign
keys with zero orphans, enums become the exact value pools, @id and @unique
are honoured, @@id/@@unique become composite uniqueness, and optional fields
may be null. CSVs land in seed-data/ ready for your seed script.
schema = misata.from_dict_schema({
"customers": {
"id": {"type": "integer", "primary_key": True},
"email": {"type": "email"},
"plan": {"type": "string", "enum": ["free", "pro", "enterprise"]},
},
"orders": {
"id": {"type": "integer", "primary_key": True},
"customer_id": {"type": "integer", "foreign_key": {"table": "customers", "column": "id"}},
"amount": {"type": "float", "min": 1.0, "max": 999.0},
"order_date": {"type": "date"},
},
}, row_count=5_000)
tables = misata.generate_from_schema(schema)Declared outcome curves: add __outcome_curves__ as a top-level key alongside the table definitions. Generated rows sum to every declared target exactly, to the cent:
import pandas as pd
schema = misata.from_dict_schema({
"__outcome_curves__": [{
"table": "orders",
"column": "amount",
"time_column": "order_date",
"time_unit": "month",
"value_mode": "absolute",
"start_date": "2024-01-01",
"avg_transaction_value": 120.0,
"curve_points": [
{"month": 1, "target_value": 50_000.0},
{"month": 6, "target_value": 110_000.0},
{"month": 12, "target_value": 200_000.0},
],
}],
"orders": {
"__rows__": 5000,
"order_id": {"type": "integer", "primary_key": True},
"amount": {"type": "float", "min": 5, "max": 500},
"order_date": {"type": "date"},
},
}, seed=42)
tables = misata.generate_from_schema(schema)
monthly = (
tables["orders"]
.assign(m=pd.to_datetime(tables["orders"]["order_date"]).dt.month)
.groupby("m")["amount"].sum()
)
assert abs(monthly[1] - 50_000) < 0.01 # exact
assert abs(monthly[12] - 200_000) < 0.01 # exactExact group shares: declare how a measure divides across a categorical column ("Electronics is 40% of revenue, Home 25%") with __group_shares__. Paired with an outcome curve on the same table and measure, the shares hold to the cent inside every declared period, and the period totals still hold; without a curve, the shares hold over the table total:
schema = misata.from_dict_schema({
"__group_shares__": [{
"table": "orders",
"measure": "amount",
"group_column": "category",
"shares": {"Electronics": 0.4, "Home": 0.25, "Toys": 0.2, "Grocery": 0.15},
}],
# ... same orders table and __outcome_curves__ as above,
# plus a "category" enum column
}, seed=42)A period with fewer rows than positive-share groups is skipped with a warning rather than silently mangled; see LIMITATIONS.md. story_audit verifies the shares in the output, and evalpacks turn each period-group pair into a verified filtered-aggregation question.
Constraints and correlations: enforce business rules and inter-column relationships directly in the dict schema:
schema = misata.from_dict_schema({
"patients": {
"__rows__": 1000,
"__constraints__": [
# visit must be on or after enrollment: enforced at generation, not post-processing
{"type": "inequality", "column_a": "visit_date",
"operator": ">=", "column_b": "enroll_date", "action": "cap"},
],
"__correlations__": [
# heavier patients tend to have higher blood pressure (r = 0.41)
{"col_a": "bmi", "col_b": "systolic_bp", "r": 0.41},
],
"patient_id": {"type": "integer", "primary_key": True},
"enroll_date": {"type": "date"},
"visit_date": {"type": "date"},
"bmi": {"type": "float", "min": 16, "max": 55},
"systolic_bp": {"type": "float", "min": 90, "max": 200},
},
})__rate_curves__ works the same way for per-period rate targets on boolean or categorical columns (fraud rates, churn flags, plan distributions).
from misata import LLMSchemaGenerator
gen = LLMSchemaGenerator(provider="groq", model="llama-3.3-70b-versatile") # free tier, fast & reliable
# gen = LLMSchemaGenerator(provider="anthropic") # Claude
# gen = LLMSchemaGenerator(provider="ollama", model="llama3") # fully local, no API key
schema = gen.generate_from_story(
"A fraud detection dataset, 2% positive rate, FICO scores, transaction velocity features"
)
tables = misata.generate_from_schema(schema)Requires pip install "misata[llm]" plus one of GROQ_API_KEY, OPENAI_API_KEY, ANTHROPIC_API_KEY, GOOGLE_API_KEY.
Groq model tip:
llama-3.3-70b-versatileis the reliable free-tier default. Larger models (e.g.openai/gpt-oss-120b) can return413 Request too largeon Groq's free tier, so use them only on a paid tier. Whatever the model returns, generation never crashes on an imperfect schema: missing relationships, malformed probabilities, and out-of-rangetime_units are repaired automatically.
tables = misata.generate("A fintech company with 1000 customers", seed=1)
# Add 1 000 more rows: IDs auto-offset, FK integrity maintained across both batches
tables = misata.generate_more(tables, schema, n=1000, seed=2)
print(len(tables["customers"])) # 2000Synthetic data rarely fails on the big numbers; it fails on the small tells a reviewer spots in five seconds. Misata kills each tell with a specific, deterministic mechanism. No LLM is involved; everything is seeded and reproducible.
The tell | The mechanism |
| Joint identity sampling: |
| Temporal profiles: scheduled events snap to 15-minute grids in business hours with weekends damped; signups follow waking-hour rhythms; only machine events (logs, clicks) keep sub-second precision; birth dates are dates. |
Every category equally likely | Zipf–Mandelbrot marginals: unweighted categoricals follow the rank-frequency power law real statuses, countries, and categories follow, with the dominant value varying per column. Declared probabilities always win. |
| Geographic facts: distances between named cities are computed (haversine × road circuity) from 289 embedded city coordinates, and travel times follow from distances. Facts, not distributions: so the Oracle can verify them. |
A five-star review that reads "disappointing", or lorem ipsum | Grammar microtext: review text is generated from the row's rating by a seeded grammar (1★ reads angry, 5★ reads delighted), a verifiable invariant. Free-text notes come from a business-note grammar. Lorem ipsum cannot reach output. |
A 19-minute appointment, a price of $43.27 | Numeric quantization: scheduled durations snap to the slot grids calendars actually offer (15/30/45/60), retail prices end in .99/.95/.00, ages are integers. Measured quantities are left alone. |
An order shipped before it was placed, by a customer who had not signed up yet | Lifecycle and causality ordering: a row's timestamps sort along the real e-commerce/SaaS/logistics lifecycle, and a child row is shifted so it never predates its FK parent, across multi-level chains, preserving the row's own gaps. |
| Address-chain coherence: city, state, postal format, and phone calling code all agree with the row's country (and a known city carries its exact state), across 14 countries and 8 postal formats. |
| Statistical priors knowledge base: recognised column names draw their real-world shape automatically: J-shaped ratings, Zipf order quantities (60% ones), lognormal salaries, .99 price endings, ~3% rare-event flags. Explicit declarations always win. |
An | Cross-table value coherence: a line item's |
tables = misata.generate("A hospital with 300 patients, doctors and appointments", seed=7)
# patients: Tae-yang Ahn (Male) · Valentina Esposito (Female) · pooja.kapoor@icloud.com
# appointments: 2023-03-08 14:00:00 · 2022-07-21 09:15:00: 15-min grid, business hours, 2% weekendsEvery coherence class above is also a detector. story_audit checks a generated
dataset against the full invariant catalog: FK orphans, cross-table temporal
causality, roll-up agreement, status gating, count and percent bounds, rare-flag
base rates, age against birth date, and more. Nothing incoherent ships silently.
tables = misata.generate_from_schema(schema, verify=True) # warns on any finding
report = misata.story_audit(tables, schema) # or audit explicitly
print(report.summary()) # "Coherence: clean" or a scored list of findingsEvery evalpack manifest embeds this verdict alongside its DuckDB answer certificate, so a pack asserts both that its answers are right and that the data telling the story is internally coherent.
Within a version, generation is deterministic. The same schema, seed, and misata version produce byte-identical tables. Evalpack manifests record the version, seed, and a SHA-256 of the spec for exactly this reason.
Across versions, RNG streams may change when generation improves (they did in 0.8.1.29 and 0.8.2). Declared outcomes still hold: aggregates, rates, identities, and integrity survive any upgrade; the individual rows may differ. Pin the version when you need bit-identical regeneration.
The public API is the documented top-level surface (misata.generate,
generate_from_schema, story_audit, coherence_audit, build_evalpack,
the schema classes, and the builders). Underscore-prefixed modules and
functions may change without notice.
Where the library is expected to fail is documented honestly, boundary by boundary, in LIMITATIONS.md. Every entry there started as a reproduced defect or a deliberate design refusal.
The 18 built-in domains are templates. For everything else, Misata refuses to fake understanding, and refuses to give up. A compositional synthesizer derives structure from your sentence: plural noun phrases become tables, "80 beekeepers" binds a row count, and a small archetype lattice (person / asset / place / event / document) provides honest structural columns and foreign-key wiring.
tables = misata.generate(
"A beekeeping cooperative with 12 apiaries, 80 beekeepers, hives, inspections and honey harvests"
)
# beekeepers: beekeeper_id, first_name, last_name, email, joined_at, status
# inspections: inspection_id, beekeeper_id, apiary_id, hive_id, inspection_date, status
# → full FK integrity, profiled timestamps, Zipfian statuses: from one sentence, no LLMWhat it will not do is invent domain semantics: unknown entities get structural columns (reference codes, statuses, dates) and the detection report says exactly that, pointing to the two upgrade paths, a schema dict, or an LLM. The same gate also prevents confabulation: a story that only weakly matches a built-in template (one incidental keyword) is composed from its own entities instead of being forced into the wrong template.
A capsule is one shareable JSON file of domain vocabularies (the species, treatments, and model names a domain calls things) with provenance for every list. Intelligence is spent once, at creation; generation stays deterministic, offline, and free.
# Mine a capsule from example data you already have: no LLM, no key
misata capsule create --domain veterinary --from-csv ./samples/ -o vet.capsule.json
misata capsule show vet.capsule.json# Vocabularies override built-in pools for matching columns
tables = misata.generate("a veterinary clinic with patients and visits",
capsule="vet.capsule.json")Capsules can also be written by an LLM once and reviewed before use (capsule_from_llm, BYO key; Groq's free tier works), or written by hand: it's JSON. Because a capsule is a file, it's a community artifact. Share it via git, a gist, or HF datasets.
Misata automatically detects the country context from your story and generates statistically accurate data for that locale, the right names, salary distributions, national ID formats, currencies, postcodes, and company naming conventions.
# Locale is detected automatically: no extra flag needed
tables = misata.generate("German SaaS company in Berlin with 2k enterprise customers")
# → names from de_DE Faker pool, salary ~ lognormal(μ=10.71, σ=0.5) ≈ €45k median,
# postcodes are 5-digit, company names end in GmbH/AG/UG
tables = misata.generate("Brazilian fintech with R$ payments and CPF verification, 50k users")
# → pt_BR names, salary median ~BRL 33.6k, national IDs match CPF format ###.###.###-##
tables = misata.generate("Indian startup in Bangalore with ₹ salary bands and Aadhaar KYC")
# → hi_IN names, salary median ~₹350k/yr, national IDs match Aadhaar 12-digit formatForce or override a locale explicitly:
schema = misata.parse("An ecommerce store with 10k orders")
tables = misata.generate_from_schema(schema) # defaults to en_US
# CLI
misata generate --story "Ecommerce store" --locale ja_JPLocale | Country | Currency | Salary median | National ID |
| United States | USD / $ | $62 000 | SSN |
| United Kingdom | GBP / £ | £34 000 | NIN |
| Germany | EUR / € | €45 000 | Steuer-IdNr |
| France | EUR / € | €38 000 | NIR |
| Brazil | BRL / R$ | R$33 600 | CPF |
| Spain | EUR / € | €27 000 | NIE |
| India | INR / ₹ | ₹350 000 | Aadhaar |
| Japan | JPY / ¥ | ¥4 400 000 | My Number |
| China | CNY / ¥ | ¥90 000 | Resident ID |
| Saudi Arabia | SAR | SAR 96 000 | National ID |
| South Korea | KRW / ₩ | ₩42 000 000 | RRN |
| Netherlands | EUR / € | €42 000 | BSN |
| Italy | EUR / € | €29 000 | Codice Fiscale |
| Poland | PLN | PLN 72 000 | PESEL |
| Turkey | TRY | TRY 720 000 | TC Kimlik |
Each pack carries real salary distributions (median and lognormal priors), age distributions, top-ranked cities, phone-number prefixes, postcode patterns, company suffixes, and VAT rates, sourced from OECD, World Bank, ILO, and national statistics offices (2023–24 data).
# Inspect a locale pack directly
pack = misata.get_locale_pack("de_DE")
print(pack.salary_median) # 45000
print(pack.currency_symbol) # €
print(pack.top_cities[:3]) # ['Berlin', 'Hamburg', 'Munich']
print(pack.company_suffixes) # ['GmbH', 'AG', 'UG', 'KG', 'e.K.']
# Auto-detect from a story
locale = misata.detect_locale("South Korean company in Seoul with KRW salaries")
# → "ko_KR"Enforce business rules that survive every row of generation:
from misata.constraints import (
InequalityConstraint, # price > cost on every row
ColumnRangeConstraint, # min_price <= price <= max_price
RatioConstraint, # 70% free / 30% pro
UniqueConstraint, # no duplicate (user_id, date) pairs
SumConstraint, # total_hours per employee per day <= 8
NotNullConstraint, # no nulls in required columns
)
c = InequalityConstraint("price", ">", "cost")
df = c.apply(df)Constraints can also be declared in misata.yaml, they run at generation time, not as a post-processing step.
Make parent summary columns reconcile with child rows, so the data survives a GROUP BY ... JOIN. A customers.total_spent column generated independently of that customer's actual orders is a giveaway that data is fake; a roll-up computes it from the real child rows.
schema = misata.from_dict_schema({
"name": "shop",
"tables": {
"customers": {
"rows": 500,
"columns": {
"customer_id": {"type": "int", "unique": True},
# total_spent = sum(orders.amount) per customer
"total_spent": {"type": "float", "rollup": {
"from_table": "orders", "fk": "customer_id",
"agg": "sum", "column": "amount"}},
# completed_spend = sum(amount) where status == "completed"
"completed_spend": {"type": "float", "rollup": {
"from_table": "orders", "fk": "customer_id", "agg": "sum",
"column": "amount", "where": {"status": "completed"}}},
},
},
"orders": {
"rows": 3000,
"columns": {
"order_id": {"type": "int", "unique": True},
"customer_id": {"type": "foreign_key", "references": "customers.customer_id"},
"amount": {"type": "float", "distribution": "lognormal", "mu": 4, "sigma": 0.5, "min": 1},
"status": {"type": "categorical", "choices": ["completed", "cancelled", "pending"]},
},
},
},
})
tables = misata.generate_from_schema(schema)
# tables["customers"]["total_spent"] reconciles exactly with the orders table.Aggregations: sum, count, mean, max, min. When a parent column name explicitly names a child table (num_orders, total_orders), the roll-up is inferred automatically with no declaration. Roll-ups survive the misata.yaml round-trip and run at generation time.
Most synthetic data tools generate rows independently. That works for database seeding and pipeline tests. It breaks the moment the data needs to pass a statistical method: an autocorrelation test on repeated measurements, a mixed-effects model checking whether groups differ, or an audit that catches values outside plausible bounds.
Misata 0.8.1.0 adds a suite of features that close this gap. All are declared in the same plain dict schema and are reachable from MCP agents, Studio, and direct Python callers.
A realistic A/B test dataset does not draw all users from one conversion distribution. The control group looks different from the treatment group. Use profiles to declare this precisely on any column:
schema = misata.from_dict_schema({
"users": {
"__rows__": 5000,
"user_id": {"type": "integer", "primary_key": True},
"cohort": {
"type": "string",
"enum": ["control", "variant_a", "variant_b"],
"probabilities": [0.50, 0.25, 0.25],
},
"session_duration": {
"type": "float",
"distribution": "lognormal",
"mean": 180.0, "std": 90.0, # fallback for unmatched rows
"profiles": [
{"when": "cohort == 'control'", "distribution": "lognormal", "mean": 180.0, "std": 90.0},
{"when": "cohort == 'variant_a'", "distribution": "lognormal", "mean": 240.0, "std": 100.0},
{"when": "cohort == 'variant_b'", "distribution": "lognormal", "mean": 310.0, "std": 120.0},
],
},
}
})The when expression is evaluated as a pandas query against already-generated columns in the same batch. Rows that match no profile get the column's top-level distribution. Profiles can reference any column generated before the current one in declaration order.
Real-world datasets have non-random missing values. Misata models both mechanisms:
Missing At Random (MAR): The probability of a value being missing depends on an observed column. High-spending users are more likely to skip the optional income field.
"annual_income": {
"type": "float",
"nullable": True,
"missing_if": {
"predictor": "total_spend",
"relationship": "higher_increases_probability",
"base_rate": 0.05,
"max_rate": 0.40,
"mechanism": "MAR",
},
}Missing Not At Random (MNAR): The probability of a value being missing depends on the value itself. Very low satisfaction scores are the ones most likely to go unreported.
"satisfaction_score": {
"type": "float",
"distribution": "normal", "mean": 7.5, "std": 1.8,
"nullable": True,
"missing_if": {
"predictor": "satisfaction_score", # references its own column
"mechanism": "MNAR",
"relationship": "lower_increases_probability",
"base_rate": 0.02,
"max_rate": 0.50,
},
}Conditional nulls (null_when): Null a column whenever a boolean expression is true.
"cancellation_reason": {
"type": "string",
"enum": ["price", "competitor", "unused", "other"],
"nullable": True,
"null_when": "churned == False",
}A boolean column with probability: 0.03 gives approximately 3% True values across many runs. If you need the dataset to contain exactly 3% (auditable against its own spec) use exact_incidence:
"is_fraud": {
"type": "boolean",
"exact_incidence": {
"mode": "exact",
"rate": 0.03, # exactly floor(n * 0.03) rows are True
},
}Per-segment exact rates work the same way:
"converted": {
"type": "boolean",
"exact_incidence": {
"mode": "exact",
"group_by": "cohort",
"rates": {"control": 0.12, "variant_a": 0.18, "variant_b": 0.24},
},
}The difference between "approximately 3% fraud" and "exactly 3% fraud" is the difference between a dataset that passes an audit and one that does not.
Without autocorrelation, a longitudinal dataset (user sessions, IoT readings, financial time series) is statistically identical to a cross-sectional one. Every time-series test (Ljung-Box, Durbin-Watson, autocorrelation plot) will immediately detect that rows are independent and the data is synthetic.
The time_series spec re-writes a column to have real within-entity autocorrelation:
"daily_revenue": {
"type": "float",
"distribution": "lognormal", "mean": 8500.0, "std": 3000.0,
"time_series": {
"entity_id": "store_id", # one process per store
"order_by": "day_number",
"model": "AR1", # AR1 | LINEAR_TREND | RANDOM_WALK | MEAN_REVERSION
"phi": 0.72, # autocorrelation coefficient (0 = independent, 1 = random walk)
"noise_std": 800.0,
"trend": {
"slope_mean": 45.0, # average daily growth per store
"slope_std": 12.0, # per-store growth variability
},
},
}Four models are available:
Model | Use case |
| Measurements that persist between periods: revenue, active users, inventory |
| KPIs with a declared direction: growth, decay, weight loss, skill improvement |
| Asset prices, exchange rates, any mean-free Brownian process |
| Bounded metrics that pull back toward average: NPS, inventory fill rate |
When a child table's column should be anchored to its parent entity's value, use a formula in distribution.mean:
"stores": {
"__rows__": 50,
"store_id": {"type": "integer", "primary_key": True},
"baseline_daily_revenue": {"type": "float", "distribution": "lognormal", "mean": 8500.0, "std": 3000.0},
},
"daily_sales": {
"__rows__": 18250, # 50 stores × 365 days
"record_id": {"type": "integer", "primary_key": True},
"store_id": {"type": "integer", "foreign_key": {"table": "stores", "column": "store_id"}},
"revenue": {
"type": "float",
"distribution": "normal",
"mean": {"formula": "@stores.baseline_daily_revenue"}, # anchored to each store's baseline
"std": 800.0, # day-to-day noise
},
}The engine resolves the FK for every row and draws from that entity's personalised distribution. Between-store variation comes from the spread of baseline_daily_revenue; within-store day-to-day noise is std: 800. Generating all rows from one shared distribution (as every column-independent generator does) collapses between-entity and within-entity variance into a single number and fails every random-effects test.
When rows are grouped under parent entities (stores, regions, branches), observations within the same group tend to look more alike than observations across groups. This within-group homogeneity (the intraclass correlation coefficient (ICC)) is a defining feature of grouped data. Without it, all groups look statistically identical.
__cluster_effect__ is declared on the parent table and applies per-entity random intercepts to columns in the child table:
"regions": {
"__rows__": 8,
"__cluster_effect__": {
"affects_table": "stores",
"affects_columns": {
"avg_order_value": {
"icc": 0.22, # target intraclass correlation
"sd_total": 45.0, # sd_between = sqrt(0.22) * 45 ≈ 21
},
"conversion_rate": {
"sd_between": 0.04, # supply sd_between directly
},
},
},
"region_id": {"type": "integer", "primary_key": True},
"name": {"type": "string", "enum": ["North", "South", "East", "West", "Central", "NW", "NE", "SE"]},
}One random intercept is drawn per parent entity from N(0, sd_between) and added to every child row in that group. The marginal distribution across all rows is preserved. Typical ICC values: 0.05–0.20 for store-level retail metrics, 0.10–0.30 for educational outcomes across schools, 0.15–0.40 for branch-level banking metrics.
For tables with many correlated columns, the matrix syntax is cleaner than a list of pairs:
"__correlations__": {
"matrix": {
"columns": ["session_duration", "pages_viewed", "revenue", "satisfaction"],
"values": {
"session_duration": [1.00, 0.71, 0.55, 0.32],
"pages_viewed": [0.71, 1.00, 0.48, 0.28],
"revenue": [0.55, 0.48, 1.00, 0.41],
"satisfaction": [0.32, 0.28, 0.41, 1.00],
}
}
}The matrix is expanded into pairwise pairs and enforced via Iman-Conover rank reordering, which hits declared Pearson r values while preserving each column's marginal distribution exactly. Pairwise list syntax still works unchanged.
Any column that represents an entity's position in a process (customer lifecycle stage, order fulfilment state, subscription status) should follow a Markov chain, not a flat probability. __state_machine__ generates the correct terminal distribution:
"orders": {
"__state_machine__": {
"state_column": "status",
"initial_state": "placed",
"transitions": {
"placed": {"confirmed": 0.95, "cancelled": 0.05},
"confirmed": {"shipped": 0.92, "cancelled": 0.08},
"shipped": {"delivered": 0.97, "returned": 0.03},
},
},
...
}States with no outgoing transitions are terminal. The engine traverses the chain per row until a terminal state is reached. Declared transition probabilities are preserved in expectation. Works alongside exact incidence, profiles, correlations, and time series in the same table.
After generation, validate against declared domain bounds before the data reaches a model or a dashboard:
tables = misata.generate_from_schema(schema)
report = misata.validate_domain(tables, domain="financial")
print(report.summary())
# Domain validation (financial): 0 errors, 0 warnings.
assert report.passedBuilt-in ranges for financial / fintech: price ≥ 0, discount 0–1, rate –1 to 100, salary ≥ 0. Column matching is by substring on the lowercased column name, "unit_price" matches the price rule.
Add custom ranges via the custom_ranges dict for any column type. Declare "__domain__": "financial" in the dict schema to attach the domain to the SchemaConfig for downstream tooling.
# Columnar / analytical
misata.to_parquet(tables, "data/")
misata.to_arrow(tables, "data/") # Apache Arrow IPC; requires pip install pyarrow
misata.to_duckdb(tables, "data/dataset.duckdb")
# Row-oriented
misata.to_jsonl(tables, "data/")
misata.to_sql(tables, "data/", dialect="postgresql") # CREATE TABLE + INSERT statements
# dialects: ansi, postgresql, mysqlGenerate additional rows that append cleanly to an existing dataset without ID collisions:
# Day 1: generate the base dataset
schema = misata.from_dict_schema({...}, seed=1)
base = misata.generate_from_schema(schema)
for name, df in base.items():
df.to_csv(f"./data/{name}.csv", index=False)
# Day 2: generate only new rows, PKs offset above existing max
new_rows = misata.generate_diff(
schema,
existing_dir="./data/",
new_rows={"customers": 200, "orders": 1500},
output_dir="./data/delta/", # optional: write delta CSVs
)generate_diff reads existing CSVs to find the maximum PK per table and generates new rows with PKs offset above that maximum. Use for streaming pipelines, day-over-day test fixtures, and any workflow where you need to extend a dataset without regenerating it from scratch.
Generate realistic, referentially-correct test data straight into Delta Lake: no
production data required. The misata.spark module bridges Misata's pandas output to
Spark/Delta on Databricks (Free Edition or full), AWS Glue, EMR, or any PySpark 3.3+ cluster.
import misata
from misata import spark as mspark
schema = misata.from_dict_schema({
"customers": {"__rows__": 500, "id": {"type": "integer", "primary_key": True},
"email": {"type": "email"}, "country": {"type": "string", "text_type": "country"}},
"orders": {"__rows__": 2000, "id": {"type": "integer", "primary_key": True},
"customer_id": {"type": "integer",
"foreign_key": {"table": "customers", "column": "id"}},
"total": {"type": "float", "distribution": "lognormal", "mu": 4.5, "sigma": 0.9}},
})
# One call: generate all tables (FK integrity guaranteed) and write to Delta
result = mspark.generate_to_delta(schema, spark, catalog="dev", database="bronze", mode="overwrite")
print(result.summary())
# ✅ customers (500 rows) → dev.bronze.customers
# ✅ orders (2,000 rows) → dev.bronze.ordersWhat it does that dbldatagen can't: multiple related tables in one call, guaranteed
referential integrity, realistic distributions, and outcome conformance, declare an exact
aggregate or rate (e.g. "fraud is 1.8% in Jan ramping to 4.1% by Jun") and the data conforms,
giving downstream pipeline tests a known ground truth to assert against.
Function | Purpose |
| One-liner: generate + write all tables to Delta |
| Convert Misata DataFrames to Spark with an explicit, type-correct schema |
| Write to Delta with partitioning, liquid clustering, table properties, or |
| Check FK integrity of Delta tables via Spark SQL anti-joins |
| Import an existing Unity Catalog schema (structure only) → generate matching data, FKs auto-inferred |
| Append incremental rows with non-colliding PKs |
| Stream-write 100M+ row datasets without buffering |
On Databricks serverless / Free Edition, install plain misata (PySpark is already on the
cluster, installing misata[spark] would stop a serverless session). On other environments:
pip install misata[spark].
End-to-end tutorial: a complete fraud-detection medallion pipeline (Bronze → Silver → Gold)
tested entirely on synthetic data, with a CI-grade ground-truth assertion,
examples/databricks/. Full API reference: docs/spark.md.
Render one document per row from any table, useful for demo datasets that need to look real end-to-end:
# Built-in templates: invoice, patient_report, transaction_receipt, user_profile
paths = misata.generate_documents(
tables, "invoice", table="orders", output_dir="/tmp/invoices", format="html"
)
# format="pdf" requires: pip install "misata[documents]"
# Custom Jinja2 template
tmpl = "<h1>Order #{{ order_id }}</h1><p>Amount: ${{ amount }}</p>"
paths = misata.generate_documents(tables, tmpl, table="orders", output_dir="/tmp/custom")bundle = misata.analyze_generation(tables, schema) # runs privacy, fidelity, data_card
print(bundle.fidelity.overall_score) # 0–100 statistical fidelity score vs. schema intent
print(bundle.fidelity.grade) # letter grade for the same score
print(bundle.privacy.overall_risk_score) # heuristic PII / re-identification risk
print(bundle.data_card.tables) # per-table row counts and metadataPublished text-to-SQL benchmarks are built by annotating question/answer pairs on top of an existing database, and that annotation step is where pervasive answer-key errors creep in. An evalpack inverts the order: the ground truth is the declared spec itself (outcome curves, rate curves, FK relationships), Misata generates a database that satisfies it, and every question shipped in the pack is then verified by executing its gold SQL against the written CSV files with DuckDB, an engine that shares no code with the generator. Questions whose observed answer does not exactly match the declared answer are dropped and recorded in the manifest. A wrong answer key is impossible by construction and double-checked by independent execution.
pip install "misata[evalpack]"
misata evalpack --config misata.yaml -o ./my_pack --seed 42from misata.evalpack import build_evalpack
result = build_evalpack(schema, "my_pack")
assert result.all_verifiedEach pack ships the tables as CSVs, questions.jsonl, a per-question verification certificate, a manifest with the spec hash and seed, and a standalone verify.py that anyone can re-run with nothing but duckdb installed. Use it to benchmark SQL agents, RAG-over-database systems, or any tool that claims to answer questions about data, against a database whose right answers are known before a single row exists.
18 built-in domain schemas, each generates a fully relational, multi-table dataset with realistic distributions, FK integrity, and domain-appropriate column semantics.
Domain | Trigger keywords | Tables generated |
SaaS | saas, subscription, mrr, churn | users, subscriptions, invoices |
Ecommerce | ecommerce, orders, store, retail | customers, products, orders, order_items |
Fintech | fintech, payments, banking, fraud | customers, accounts, transactions |
Healthcare | healthcare, patients, doctors, clinic | doctors, patients, appointments |
Marketplace | marketplace, sellers, buyers, listings | sellers, buyers, listings, orders |
Logistics | logistics, shipping, drivers, routes | drivers, vehicles, routes, shipments |
HR | hr, employees, payroll, workforce | departments, employees, payroll |
Social | social media, instagram, feed, followers | users, posts, follows, reactions |
Real Estate | real estate, housing, mortgage | agents, properties, transactions |
Pharma | pharma, clinical, trials | researchers, projects, trials, timesheets |
Food Delivery | food delivery, restaurant, takeout | restaurants, customers, couriers, orders, order_items |
EdTech | edtech, courses, students, enrollments | instructors, courses, students, enrollments, quiz_attempts |
Gaming | gaming, players, leaderboard, esports | players, matches, sessions, achievements |
CRM | crm, salesforce, deals, pipeline | companies, contacts, deals, activities |
Crypto / Web3 | crypto, blockchain, ethereum, defi | wallets, tokens, transactions, token_prices |
Insurance | insurance, policy, claims, premium | customers, policies, claims, payments |
Travel | travel, hotel, flights, bookings | users, hotels, flights, bookings, reviews |
Streaming | streaming, netflix, subscribers, watch history | subscribers, content, watch_history, ratings |
No keyword match → the compositional synthesizer builds a structural multi-table schema from your sentence's own entities (see Unknown domains above); stories with no entities at all fall back to a generic single table with smart column inference.
story / YAML / dict / DB introspection / MCP tool call
↓
StoryParser · compositional synthesizer · locale detection · load_yaml_schema · schema_from_db
↓
DetectionReport (domain, confidence, near_misses, table_preview, warnings)
↓
SchemaConfig ← validate_schema() catches issues before any rows are generated
↓
DataSimulator
├─ topological sort (FK dependency order)
├─ domain priors → locale priors (salary, age, monetary)
├─ constraint engine (inequality, range, ratio, sum, unique)
├─ outcome curves (monthly targets from narrative control points)
├─ stratified profiles (per-subgroup distributions, pandas eval)
├─ AR1 / time-series autocorrelation (per entity, 4 models)
├─ state machine (Markov terminal states)
├─ ICC cluster effects (per-parent-entity random intercepts)
├─ Iman-Conover correlation engine (pairwise + full matrix)
├─ MAR / MNAR missingness (predictor-scaled and value-dependent)
├─ exact incidence (floor(n × rate), per-group rates)
├─ realism core (joint identities, temporal profiles, Zipf marginals,
│ geo facts, grammar microtext, numeric quantization)
└─ RealisticTextGenerator (capsules + Faker locale + vocabulary assets)
↓
{table_name: DataFrame}
↓
validate_domain · seed_database · to_parquet · to_arrow
to_duckdb · to_sql · to_jsonl · generate_documents · MCP CSV outputDomain priors: monetary columns get log-normal distributions. Categoricals use Zipf sampling. Blood types, country distributions, and salary bands reflect real-world statistics.
Locale priors: salary and age distributions are overridden with country-specific lognormal/normal parameters sourced from national statistics. "Brazilian fintech" in your story means salaries are sampled from the BRL distribution, not the USD one.
Outcome curves: natural-language narrative is parsed into exact monthly control points. Named events, quarters, and multipliers all work:
# All of these produce precise, shaped outcome curves:
misata.generate("SaaS mrr from $50k in Jan to $200k in Dec, with a Q3 slump")
misata.generate("Ecommerce orders, Black Friday spike, Christmas peak")
misata.generate("SaaS startup, MRR 10x growth over the year")
misata.generate("Fintech payments, strong Q4, dip in Q1")Realism rules: cost is always less than price. delivered_at is always after shipped_at. hire_date is after date_of_birth + 18 years and never in the future. tenure_years is derived on the same row from hire_date. Email addresses derive from first and last name columns, names agree with declared genders, route distances agree with their cities, and review text agrees with its star rating.
Comparison reflects each tool's documented, out-of-the-box behavior as of late 2025; all of these are capable libraries built for different goals, and a "No" means "not a built-in feature," not "impossible."
Faker | Synth | syda | SDV | Misata | |
No config, one line to multi-table data | No | No | No | No | Yes |
Story auto-detects locale + country stats | No | No | No | No | Yes |
18 built-in domain schemas (SaaS → streaming) | No | No | No | No | Yes |
Narrative curves (Q4 push, Black Friday, 10×) | No | No | No | No | Yes |
Unknown domains composed from the sentence itself | No | No | No | No | Yes |
Coherent identities (name ↔ gender ↔ email agree) | No | No | No | No | Yes |
Review text provably matches its star rating | No | No | No | No | Yes |
Real city distances on route tables | No | No | No | No | Yes |
Shareable domain vocabulary capsules | No | No | No | No | Yes |
Mimic mode: clone distributions from a CSV | No | No | No | Yes | Yes |
Pairwise + full-matrix correlation (Iman-Conover) | No | No | No | Yes | Yes |
Geospatial columns (lat, lng, postal_code) | No | No | No | No | Yes |
Anomaly injection (per-column outlier rate) | No | No | No | No | Yes |
MCP server: usable from Claude / Cursor | No | No | No | No | Yes |
YAML schema committed to git | No | Yes | Yes | No | Yes |
JSON Schema validation + editor auto-complete | No | No | No | No | Yes |
DB introspection → generate → re-seed | No | Yes | No | Limited | Yes |
Direct DB seeding (Postgres / MySQL / SQLite) | No | No | No | No | Yes |
SQLAlchemy model seeding | No | No | No | No | Yes |
Referential integrity across all FK tables | No | Yes | Yes | Yes | Yes |
Inequality / range constraints ( | No | Limited | No | Yes | Yes |
Aggregate target curves (monthly MRR shape) | No | No | No | No | Yes |
Stratified distributions per subgroup (profiles) | No | No | No | No | Yes |
MAR and MNAR informative missingness | No | No | No | No | Yes |
Exact incidence control (floor(n × rate) True values) | No | No | No | No | Yes |
AR(1) / time-series autocorrelation per entity | No | No | No | No | Yes |
Hierarchical ICC cluster effects (multi-site) | No | No | No | No | Yes |
@parent formula in distribution mean/std | No | No | No | No | Yes |
Markov state machine terminal states | No | No | No | No | Yes |
Domain-aware validation (clinical/financial ranges) | No | No | No | No | Yes |
SQL INSERT export (ansi / postgresql / mysql) | No | No | No | No | Yes |
Apache Arrow IPC export | No | No | No | No | Yes |
Reproducible incremental rows (generate_diff) | No | No | No | No | Yes |
Domain-realistic distributions | No | No | No | Limited | Yes |
Multi-provider LLM (Groq / OpenAI / Claude / Gemini / Ollama) | No | No | Yes | No | Yes |
Fully offline, no LLM required | Yes | Yes | No | Yes | Yes |
Document generation (HTML / PDF per row) | No | No | No | No | Yes |
Quality + privacy reports | No | No | No | Limited | Yes |
Pure Python, no external services | Yes | No | No | Yes | Yes |
Faker generates individual fake values, not relational, no schema, no statistical accuracy.
Synth excels at schema-as-code git workflows; limited distribution control.
syda uses an LLM for every row, semantically rich but expensive, slow, and requires an API key.
SDV learns from real data, a different problem (you need real data first).
Gretel is a cloud service that needs an API key and sends data off-premise; Misata runs locally.
Misata generates from intent, offline by default, seeds databases directly, and now brings country-accurate statistics to every column automatically.
Full head-to-head comparisons: Misata vs Faker, Misata vs SDV, Misata vs Gretel.
Measured on Apple M-series (single core, no GPU):
Workload | Rows | Time | Throughput |
Single table, lognormal | 1 000 000 | 0.06 s | ~16M rows/s |
Star schema (5 tables, 4 FKs) | 1 055 030 | 1.54 s | ~687k rows/s |
git clone https://github.com/rasinmuhammed/misata
cd misata
pip install -e ".[dev]"
pytest tests/1,088 tests, 0 failures. Issues and PRs welcome, github.com/rasinmuhammed/misata/issues
Score a folder of CSVs for the contradictions a human reader would catch.
This is Misata's coherence audit run on data that already exists — data an agent generated in an earlier step, data a user built by hand, or the output of some other tool. It checks, among other things:
timestamps that run backwards (shipped before ordered, resolved before opened),
derived columns that do not reconcile with their inputs
(total != quantity * unit_price),
geographic fields that disagree (city / state / postcode / country),
near-constant columns (98% one value — a distribution tell),
filler text and out-of-scale numerics.
A score of 100 is clean. Below ~85 usually means the schema is missing
realism structure, not that individual rows need patching: add
__correlations__, profiles, time_series, a __state_machine__,
or an __outcome_curves__ declaration and regenerate.
Args:
dataset_dir: Directory containing one CSV per table (e.g. the
output_dir returned by generate_from_schema).
top_findings: Max findings to include in the response (default 20).
Returns:
{"score": 0-100, "clean": bool, "counts": {...}, "findings": [...]}.
| Name | Required | Description | Default |
|---|---|---|---|
| dataset_dir | Yes | ||
| top_findings | No |
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations (readOnly, openWorld, idempotent, non-destructive) already cover the safety profileestr; the description adds real behavioral context on top: what kinds of contradictions it detects, that scores below ~85 indicate schema issues rather than data issues, and the recommended remediation path. It also specifies the top_findings cap and the return shape. No contradiction with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is long but each section earns its place: the opening sentence states the operation, the bulleted list gives the agent concrete detection categories, the score-interpretation paragraph is actionable, and the parameter/return details are compact. The multi-line structure with bullets and monospace examples is readable.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description explains what this audit is for, when it is meant to be run (on already-generated data), how to interpret the 0-100 score, and what to do next (add realism structures). It avoids duplicating the output schema but explains the score semantics and the folder format. Minor gap: it does not explicitly say that a clean score means no action is needed, but the score interpretation and remediation advice cover the essential behavior.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is effectively 0% for both parameters (one has only a type and a default, the other only a type), so the description carries the burden. It adequately explains 'dataset_dir' (one CSV per table, the output_dir of generate_from_schema) and 'top_findings' (max findings, default 20). That covers both parameters even though the schema itself has descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Score a folder of CSVs' with a specific verb and resource. It explicitly enumerates the types of contradictions detected (timestamps, reconciliation, geography, near-constant columns, filler text), which gives the agent a precise picture of what this tool does and distinguishes it from generation/schema tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says when to use this tool: on data that already exists, whether agent-generated or hand-built. It also tells the agent what the findings mean and how to respond — add structure declarations instead of patching data — which acts as an implicit exclusion for fix-data workflows. Sibling context (generate_from_schema) reinforces the routing.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Generate a synthetic dataset from a story and write it to disk as CSV files.
Returns the output directory, file paths, row counts per table, and a small sample of rows for each table so the agent can show the user what was produced without loading every row into context.
Args: story: Plain-English description of the dataset. rows: Default row count for the primary table. seed: Optional random seed (same seed → byte-identical output). output_dir: Where to write CSVs. Defaults to a fresh temp dir. sample_rows: Number of rows from each table to include in the response (max 50).
| Name | Required | Description | Default |
|---|---|---|---|
| rows | No | ||
| seed | No | ||
| story | Yes | ||
| output_dir | No | ||
| sample_rows | No |
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds substantial behavioral context beyond the annotations. It states that the tool writes CSV files to disk, returns output directory and file paths, and includes row counts and sample rows. It also discloses determinism: 'same seed → byte-identical output.' These details are not present in the annotations (readOnlyHint false, destructiveHint false) and valuable for the agent to anticipate side effects. No contradiction with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and well-structured. The opening sentence states purpose, followed by a clear list of return values, then a bulleted Args section. Every sentence earns its place, and the use of a list for parameters improves readability. There is no redundant information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has 5 parameters, an output schema, and complex behavior (file writing, determinism). The description explicitly mentions return values (output directory, file paths, row counts, sample rows), addresses parameter defaults, and explains the sample_rows cap. It is complete for an agent to use effectively without further documentation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has no property descriptions (coverage 0%), so the description carries the full burden. It includes an Args section that explains each of the 5 parameters: story, rows, seed, output_dir, and sample_rows, adding meaning like 'Defaults to a fresh temp dir' and 'max 50'. This fully compensates for the schema's lack of descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's action: 'Generate a synthetic dataset from a story and write it to disk as CSV files.' It specifies the resource (dataset from a story) and distinguishes it from siblings like generate_from_schema, which focuses on schema-driven generation. The verb 'generate' and the context 'from a story' are specific and unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context for when to use the tool: when you have a plain-English story to convert into a dataset. It also explains that it returns sample rows and file paths, which is helpful for the agent to show results. However, it does not explicitly mention alternatives (e.g., 'use generate_from_schema if you have a schema') or state when NOT to use it, leaving a slight gap in decision-making guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Generate a dataset from a schema you design. This is the primary Misata tool.
DIVISION OF LABOUR You (the agent) design what the data should look like — the tables, columns, business rules, and declared targets. Misata handles the hard guarantees: every FK resolves, every rollup reconciles to the cent, every declared outcome curve is hit exactly, every seed produces byte-identical output. The response includes a per-relationship integrity proof.
SCHEMA FORMAT {table_name: {column_name: spec, ...}, ...}
TABLE-LEVEL KEYS (inside a table dict) "rows": 5000 Per-table row count; overrides the global rows arg. Always set this — one global count rarely fits all tables. "constraints" List of row-level business rules (see CONSTRAINTS below). "correlations" List of pairwise Pearson targets (see CORRELATIONS below). "state_machine" Markov terminal-state assignment (see STATE MACHINE below).
COLUMN TYPES integer, float, decimal Numeric. string Short categorical or free text. text Long free text (descriptions, notes). email, phone, url, uuid Semantic strings; always valid format. date, datetime Temporal; realistic granularity applied automatically. boolean True/False with declared probability.
COLUMN SPEC KEYS (inside a column dict) primary_key: true Auto-incremented PK; column excluded from CSV output. foreign_key: {table, column} Child FK; referential integrity guaranteed + verified. min / max Numeric or date bounds. decimals: 2 Decimal places for float output. unique: true All values in the column are distinct. nullable: true Allow nulls (default true). enum: [...] Categorical choices. Add probabilities: [...] for weights; omit for realistic Zipf-shaped rank frequencies. probabilities: [...] Weights for enum choices; must sum to 1.0.
DISTRIBUTIONS (float / integer columns) distribution: normal Also: lognormal, uniform, exponential, beta, poisson, power_law, gamma. mean / std Normal params. Can be a scalar OR a per-row parent-entity lookup: mean: {formula: "@patients.hba1c_baseline"} The FK is resolved per-row so each child row's distribution is anchored to its parent's value. Use this for longitudinal data where within-entity variation should be modelled separately from between-entity variation. mu / sigma Lognormal params (mu and sigma are of log(x)). min / max Hard clamps applied after sampling. Use lognormal for money, file sizes, session durations — anything right-skewed and strictly positive. Use normal for measurements.
DERIVED COLUMNS formula: "quantity * unit_price" Row-level arithmetic; pandas eval syntax. formula: "hours * @employees.rate" Cross-table via FK: @parent_table.column. rollup: {from_table, fk, agg, column} Parent column that EXACTLY reconciles with child rows under JOIN. agg: sum/count/mean/ max/min. Add where: {col: val} to filter. RULE: use rollup (not formula) for any parent column that summarises child rows. Rollups are closed-form exact; formulas cannot cross the FK boundary correctly.
CODE-STYLE STRINGS pattern: "SKU-\d{5}" Single pattern expanded per row. pattern: ["A/\d{5}", "\d{6}"] List: one shape drawn per row. pattern_weights: [0.7, 0.3] Weights for pattern list (optional). Supported tokens: \d (digit), [A-Z] (uppercase letter), [a-z] (lowercase), literal chars, {n} repeat count. Example: "[A-Z]{2}-\d{4}" → "AB-3721".
TEXT SEMANTICS text_type: person_name Always beats column-name inference. Options: person_name, email, company, city, country, postal_code, phone, url, description, username, product_name, review_text, address, job_title. Dates: appointment times snap to 15-min business-hours grids; signups follow waking-hour rhythms; machine events keep sub-second precision. Names, genders, and emails are generated jointly and always agree.
STRATIFIED DISTRIBUTIONS (profiles) Use when different subgroups need different distributions for the same column. profiles: [ {when: "arm == 'placebo'", distribution: normal, mean: -0.35, std: 0.50}, {when: "arm == 'high_dose'", distribution: normal, mean: -1.25, std: 0.55}, ] Rows that match no profile get the column's top-level distribution. The when expression is a pandas eval string; reference any already-generated column in the same table. Always list profiles after the columns they reference.
INFORMATIVE MISSINGNESS (MAR) null_when: "dropout == False" Null this column when expression is true. missing_if: Missing-At-Random tied to a predictor column. predictor: hba1c_baseline relationship: higher_increases_probability # or lower_increases_probability base_rate: 0.05 # null probability at predictor median max_rate: 0.40 # null probability at predictor extreme Use null_when for status-conditional nulls (dropout_visit is null when not dropped out). Use missing_if when missingness is correlated with an observed variable.
EXACT INCIDENCE CONTROL exact_incidence: Hit the declared count exactly (not approximately). mode: exact rate: 0.22 # exactly floor(n * 0.22) rows become True group_by: arm # optional: apply per group rates: {placebo: 0.15, high_dose: 0.55} # per-group exact rates Use exact_incidence instead of probability on boolean columns when the user states a precise rate that must hold in the data, not just on average.
WITHIN-ENTITY TIME SERIES (longitudinal autocorrelation) time_series: Re-writes a column to have AR1 autocorrelation entity_id: patient_id within each entity group. order_by: visit_number model: AR1 # AR1 | linear_trend | random_walk | mean_reversion phi: 0.72 # autocorrelation coefficient (AR1 only) noise_std: 0.30 anchor_column: hba1c_baseline # starting value (column in the same table) trend: slope_mean: -0.08 # mean drift per step slope_std: 0.02 # per-entity slope variability Required for any longitudinal dataset (clinical visits, IoT sensors, user sessions). Without it every row is independent and the data fails any time-series test.
CONSTRAINTS (table-level constraints list) {"type": "inequality", "column_a": "visit_date", "operator": ">=", "column_b": "enroll_date", "action": "cap"} Enforces column_a OP column_b. action: "cap" (snap column_a to column_b) or "drop" (remove violating rows). Works on dates and numerics. {"type": "col_range", "low_column": "min_price", "column": "price", "high_column": "max_price", "action": "cap"} Keeps low_column <= column <= high_column. {"type": "max_per_group", "group_by": "user_id", "max_count": 3} Limits rows per group value. {"type": "unique_combination", "columns": ["user_id", "product_id"]} No duplicate (col_a, col_b) pairs. Use constraints for any business rule that must hold on every row: visit_date >= enrollment_date, price > cost, resolution_day > onset_day.
CORRELATIONS (table-level correlations list) [{"col_a": "bmi", "col_b": "systolic_bp", "r": 0.41}] Enforced via Iman-Conover (rank reordering): preserves each column's marginal distribution while hitting the declared Pearson r exactly. Declare correlations for any pair of measurements that co-vary in the real domain (bmi/bp, income/spending, tenure/salary). Also supports full matrix syntax: correlations: matrix: columns: [hba1c, glucose, bmi] values: hba1c: [1.00, 0.65, 0.28] glucose: [0.65, 1.00, 0.22] bmi: [0.28, 0.22, 1.00]
ICC CLUSTER EFFECTS (parent table cluster_effect) cluster_effect: affects_table: visits affects_columns: hba1c: icc: 0.18 # intraclass correlation coefficient sd_total: 1.5 # total standard deviation; sd_between = sqrt(icc)*sd_total systolic_bp: sd_between: 8.0 # supply sd_between directly if preferred Applies per-parent-entity random intercepts to the named child columns. Required for multi-site or multi-centre designs — without it all sites look identical and any ICC statistical test will detect the synthetic origin. icc: 0.10-0.30 is typical for clinical measurements across sites.
STATE MACHINE (table-level state_machine) state_machine: state_column: patient_status initial_state: enrolled transitions: enrolled: {on_treatment: 0.97, screen_failure: 0.03} on_treatment: {completed: 0.77, dropout: 0.23} Assigns one terminal state to every row by following the Markov chain. States with no outgoing transitions are terminal. Use for any process with defined states: clinical trial statuses, customer lifecycle, order fulfilment stages, support ticket resolution.
SCHEMA-LEVEL DIRECTIVES (top-level keys, siblings of the tables)
outcome_curves Declare aggregate targets the engine hits EXACTLY. [{"table": "orders", "column": "amount", "time_column": "order_date", "time_unit": "month", "value_mode": "absolute", "start_date": "2024-01-01", "avg_transaction_value": 120.0, "curve_points": [ {"month": 1, "target_value": 50000.0}, {"month": 6, "target_value": 110000.0}, {"month": 12, "target_value": 200000.0} ]}] ALWAYS use this when the user states what a number should sum to per period: "revenue grows from $50k to $200k", "Q4 spike", "10x growth". avg_transaction_value drives row count per period; set it to roughly the median row value for that column.
rate_curves Per-period rate targets for boolean/categorical columns. [{"table": "transactions", "column": "is_fraud", "time_column": "transaction_date", "rate_points": [ {"period": "2024-01", "rate": 0.02}, {"period": "2024-Q4", "rate": 0.05} ]}] Use when fraud rate, churn rate, or conversion rate changes over time.
group_shares Exact shares of a measure across a categorical column. [{"table": "orders", "measure": "amount", "group_column": "plan", "shares": {"Starter": 0.2, "Pro": 0.5, "Enterprise": 0.3}}] The measure sums to those proportions per group, exactly. Paired with an outcome_curves on the same table+measure, the split holds inside every declared period. Use for any "A is 40% of revenue, B is 35%..." statement.
waterfalls Movements that reconcile to declared running balances. [{"table": "mrr_movements", "starting_value": 100000, "points": [{"period": "2026-01", "ending_value": 106000}, ...], "inflow_shares": {"new": 0.7, "expansion": 0.3}, "outflow_shares": {"churn": 1.0}}] Use for MRR bridges, cash-flow statements, any "opening + inflows - outflows = closing" ledger that has to tie out.
stock_flows Per-unit inventory identity: closing = opening + received
shipped, enforced for every SKU across every period. Use for warehouse / inventory data where stock levels must be internally consistent.
lifecycles A state machine with legal transitions (stricter than a table-level state_machine: illegal jumps are impossible, not just unlikely). [{"name": "order_flow", "table": "orders", "state_column": "status", "start_column": "placed_at", "initial": "placed", "states": [{"name": "placed"}, {"name": "paid"}, {"name": "shipped"}, {"name": "delivered"}, {"name": "refunded", "terminal": true}], "transitions": [["placed","paid"],["paid","shipped"], ["shipped","delivered"],["delivered","refunded"]]}]
missingness Why a value is missing, conditionally (schema-level MNAR). [{"table": "contacts", "column": "notes", "rate": 0.75, "else_rate": 0.05, "when_column": "is_active", "when_op": "==", "when_value": false}] The null rate is exact per branch. Use when missingness itself carries signal a cleaning step should be tested against.
DIRTY DATA ON PURPOSE (exact counts, so a test has a known number to find) duplicates [{"table": "contacts", "count": 60}] typos [{"table": "contacts", "column": "city", "count": 120}] outliers [{"table": "orders", "column": "amount", "count": 40}] Each injects exactly that many defects, leaving primary/unique/foreign keys intact. Use when the user is building or testing a data-quality or cleaning pipeline.
domain Domain hint for post-generation validation.
"domain": "clinical_trial" # or "clinical", "financial", "fintech"
When set, generate_from_schema runs domain validation automatically and
returns it as domain_validation in the response — no second call
needed. Built-in ranges: HbA1c 4-14 %, BMI 10-80, systolic BP 60-260,
age 0-130, glucose 2-40, cholesterol 1-20, hemoglobin 3-25 for clinical;
price >= 0, discount 0-1, rate -1 to 100 for financial.
There are ~24 schema-level declarations in total. The ones above cover the common cases; for the rest (retention cohorts, DAG edges, closure tables, graph motifs, time grids, bitemporal history) fetch https://misata.studio/docs/reference/declarations.md and check the exact key before inventing one.
DESIGN RULES — follow these to get the best result in one pass
Always set rows per table. A fintech schema with customers=2000, accounts=4000, transactions=50000 is far better than 1000 everywhere.
Every child table needs a FK column pointing to its parent PK. Without it orphan rows are generated and the integrity proof will fail.
Use lognormal for money, file sizes, response times (right-skewed, strictly positive). Use normal for measurements (height, score, temp).
Declare correlations for any pair that co-varies in the real domain. Generated data with an identity correlation matrix is the clearest synthetic-data tell there is.
Use exact_incidence instead of probability when the user states a precise rate. "3% fraud" with probability: 0.03 gives ~3% on average; exact_incidence gives exactly 3%.
Use rollup (not formula) for any parent column that must reconcile with child rows. customers.total_spent generated independently of orders will never match; a rollup makes it exact.
Use outcome_curves any time the user mentions a revenue shape, a growth trajectory, a seasonal pattern, or a specific period total. It is the single feature most likely to be forgotten and most visible when absent.
Use profiles when two groups need different distributions. A clinical trial where all arms share one HbA1c distribution is statistically wrong and the difference will be caught by any summary table.
For longitudinal data (visits, sessions, sensor readings), add time_series to the key measurement columns. Independent rows fail every autocorrelation test and are visually obvious when plotted.
Add state_machine to any entity that moves through a process. An order table with no status progression is not realistic order data.
Args: schema: Dict of table defs plus optional schema-level directives. rows: Default row count for tables without rows. seed: Random seed (same seed → byte-identical output on any machine). output_dir: Where to write CSVs. Omit to use a fresh temp dir. sample_rows: Rows per table to include in the JSON response (max 50).
| Name | Required | Description | Default |
|---|---|---|---|
| rows | No | ||
| seed | No | ||
| schema | Yes | ||
| output_dir | No | ||
| sample_rows | No |
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds meaningful behavioral context beyond annotations: same seed produces byte-identical output, CSV output goes to output_dir or a temp dir, domain validation may run automatically, and the response includes an integrity proof. It does not contradict the annotations and substantially helps an agent anticipate side effects and outputs.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very long, but the length is largely justified by the complexity of the schema DSL it defines. It is organized with clear section headers, front-loads the primary purpose, and includes practical design rules, making the length readable rather than a hazard.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description is effectively a spec: it covers the schema structure, column types, constraints, correlations, state machines, time-series handling, top-level directives, and it even tells the agent where to fetch documentation for unsupported declarations. Given the tool's complexity and the otherwise empty schema descriptions, this is complete enough to call correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description carries the entire burden of parameter meaning. It documents every argument in the Args section and uses a very detailed, nested DSL to explain the schema parameter—covers row counts, constraints, distributions, rollups, outcome curves, duplicates, and more.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a clear verb-and-resource statement: 'Generate a dataset from a schema you design' and reinforces it as 'the primary Misata tool.' However, it does not explicitly disambiguate from the sibling generate_dataset tool, so it is clear but not fully sibling-differentiated.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is abundant design guidance and feature-selection advice within the description, such as when to prefer rollup over formula and when to use exact_incidence. However, it does not explicitly say when to use this tool instead of generate_dataset or the validation siblings, so usage vs. alternatives is implied rather than explicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Return the full schema (tables, columns, relationships) for a story without generating data.
Heavier than preview_story — includes every column with its type and
distribution params. Use when the user wants to see the structure they'll
get, or to author a misata.yaml file from a natural-language seed.
Args: story: Plain-English description of the dataset. rows: Default row count for the primary table.
| Name | Required | Description | Default |
|---|---|---|---|
| rows | No | ||
| story | Yes |
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already mark this as read-only and non-destructive; the description builds on this by noting it does not generate data and by explaining the type of output (every column with type and distribution params). It stops short of detailing performance implications beyond 'heavier' or clarifying idempotent behavior, but adds meaningful context beyond annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is tightly written, starting with a clear one-sentence purpose, followed by a comparative detail, use cases, and a compact Args list. Every sentence earns its place with no redundancy or fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a read-only inspection tool with two parameters and a documented output schema, the description covers the what, when, and parameters, and even notes the lack of data generation. It could go further by describing error scenarios or output format specifics, but these are already captured by the output schema and annotations, making it adequately complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description must compensate, and it does via an Args section that explains `story` as 'Plain-English description of the dataset' and `rows` as 'Default row count for the primary table.' This adds clear meaning beyond the bare types, though it could elaborate on how changing `rows` affects its behavior. The high coverage shift justifies a score above baseline.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Return the full schema (tables, columns, relationships) for a story without generating data,' using a specific verb and resource scope. It goes beyond a generic statement by explicitly contrasting with `preview_story` ('Heavier than') and providing concrete use cases, making its purpose unmistakable and distinctive.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly notes this tool is 'Heavier than `preview_story`' and provides two direct use cases: showing the structure the user will get and authoring a `misata.yaml` file from a seed. This gives an agent clear when-to-use guidance and signals an alternative, even if it doesn't mention explicit exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List the 18 built-in business domains Misata can generate from natural language.
Each domain has trigger keywords and a sample story you can pass to
preview_story or generate_dataset. Use this when the user asks
"what kinds of data can you generate?" or to suggest a story format.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, idempotentHint, and non-destructive behavior. The description adds useful context beyond annotations by revealing that the result contains 18 domains, each with trigger keywords and a sample story, which the agent can pass to preview_story or generate_dataset.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact and front-loaded with the core action in the first sentence. The subsequent sentences add useful behavioral and usage context without unnecessary fluff, though it could have been slightly tighter by merging the first two sentences.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description is sufficient for a parameterless list tool: it names the exact output scope, explains what each returned domain contains, gives intended usage triggers, and references downstream consumer tools. The output schema and annotations cover the remaining structural and safety aspects.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool takes zero parameters, and the schema coverage is effectively 100% because there is no input to document. There is nothing for the description to add about parameter syntax or meaning, so the baseline score for a zero-parameter tool applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description starts with a specific verb+resource: 'List the 18 built-in business domains Misata can generate from natural language.' This clearly distinguishes the tool from the generative siblings by indicating it is a read-only catalog of available domains.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It explicitly states when to use this tool: 'Use this when the user asks "what kinds of data can you generate?" or to suggest a story format.' It also mentions downstream tools like preview_story or generate_dataset, but it does not explicitly address when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Inspect what Misata would generate from a story — without generating any rows.
Returns the detected domain, confidence, near-misses, locale, scale, and a preview of the tables that would be produced. Use this to confirm interpretation before committing to a (potentially large) generation.
Args: story: Plain-English description of the dataset. rows: Default row count for the primary table (affects preview only).
| Name | Required | Description | Default |
|---|---|---|---|
| rows | No | ||
| story | Yes |
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and destructiveHint=false. The description adds context: it clarifies no rows are generated, that rows affects preview only, and lists returned fields (domain, confidence, near-misses, locale, scale, tables). No contradiction with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured: a one-sentence purpose, a list of returned items, a usage note, and an Args section. It is concise with no redundant text and front-loads the core purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (2 params, output schema exists), the description covers purpose, usage, parameters, and return values. It provides sufficient context for an agent to understand when and how to invoke it, without needing to rely on the output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must clarify parameters. It provides Args: 'story: Plain-English description of the dataset' and 'rows: Default row count for the primary table (affects preview only).' This fully compensates for missing schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Inspect what Misata would generate from a story — without generating any rows.' This uses a specific verb (inspect) and resource (story interpretation) and differentiates from sibling generation tools like generate_dataset and seed_database.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly says when to use: 'Use this to confirm interpretation before committing to a (potentially large) generation.' It implies the alternative is to generate, but does not name the exact sibling tools. Still, the guidance is clear and actionable.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Fill a live Postgres or SQLite database with realistic, connected data, read from the database's own schema.
Reads the tables, columns, and foreign keys directly from the target database, generates data that respects them, inserts parents before children, then queries the database back to confirm every foreign key resolves. No schema file and no ORM are needed: a connection string is enough.
SAFETY — this is the only Misata tool that writes to a user's database:
It plans by default. With apply=False (the default) nothing is
written; you get the table list, insert order, existing row counts,
and what would be inserted. Show that plan to the user.
Only call again with apply=True after the user has seen the plan
and agreed. Never pass apply=True on a first call.
If any target table already has rows, the write is refused unless the
user chooses truncate=True (wipe and reseed) or append=True
(keep existing rows, seed only empty tables, and draw foreign keys
from the rows already there). Never guess between these.
truncate=True DESTROYS existing data. Only use it on a throwaway
development database and only when the user explicitly asks.
Args:
db_url: Connection string, e.g. postgresql://localhost/myapp_dev
or sqlite:///dev.db.
rows: Base row count; reference and transaction tables scale from it.
apply: False (default) plans only. True performs the write.
truncate: Wipe target tables (children first) before seeding.
append: Keep populated tables and seed only the empty ones.
tables: Optional allow-list of table names to seed.
skip_tables: Tables to leave untouched (migrations, auth, etc.).
seed: Random seed; the same seed reproduces the same data.
Returns:
A plan (applied: false) or a result with per-table row counts and
a per-relationship integrity proof (integrity.verified).
| Name | Required | Description | Default |
|---|---|---|---|
| rows | No | ||
| seed | No | ||
| apply | No | ||
| append | No | ||
| db_url | Yes | ||
| tables | No | ||
| truncate | No | ||
| skip_tables | No |
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description goes beyond the annotations by detailing the destructive nature (truncate=True destroys data), the write behavior, and the safety flow (plan first, then apply). It transparently discloses side effects and the conditions under which writes are allowed, adding crucial context that annotations alone do not provide.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Although lengthy, the description is well-structured with clear sections (summary, safety, args) and every sentence adds value. It avoids redundancy and uses formatting (bold, bullet points) to improve readability, making it efficient despite its length.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the 8 parameters and output schema, the description thoroughly covers all aspects: parameter usage, safety considerations, and the return value (plan vs result with integrity verification). It is complete and self-contained, leaving no critical gaps for an agent to misuse the tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 0% description coverage, but the description's Args section explains every parameter in detail, including examples, defaults, and how they interact (e.g., rows scaling, tables/skip_tables filtering, apply vs truncate/append). This fully compensates for the missing schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's function: filling a live Postgres or SQLite database with realistic, connected data based on the database's own schema. It distinguishes itself from sibling tools like generate_from_schema by emphasizing that no schema file or ORM is needed, and it is the only tool that writes to a database.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly explains when to use this tool (to populate a database) and differentiates it from alternatives by noting it is the only one that writes. It also provides clear instructions on the safe default (apply=False) and when to use truncate/append, making it obvious when to call it versus other tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Flag values that are physiologically or financially impossible for a domain.
Where audit_dataset checks internal consistency, this checks values
against what the outside world allows. Built-in ranges include, for
clinical / clinical_trial: HbA1c 4-14%, BMI 10-80, systolic BP
60-260, age 0-130, glucose 2-40, cholesterol 1-20, hemoglobin 3-25; for
financial / fintech: price >= 0, discount 0-1, rate -1 to 100.
Use after generating a dataset in a regulated or measurement-heavy domain, or when the user asks "is this data plausible for a real clinic / bank?".
Args:
dataset_dir: Directory containing one CSV per table.
domain: One of clinical_trial, clinical, financial,
fintech.
Returns:
{"passed": bool, "errors": [...], "warnings": [...]}. passed
is True when there are no ERROR-level findings.
| Name | Required | Description | Default |
|---|---|---|---|
| domain | Yes | ||
| dataset_dir | Yes |
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, openWorldHint=true, idempotentHint=true, and destructiveHint=false, covering safety. The description adds significant behavioral context beyond that: the exact output structure (passed, errors, warnings), the meaning of 'passed', and the built-in domain ranges. It also clarifies that it does not check internal consistency, preventing misinterpretation. No contradictions with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured: a one-sentence summary, a clarifying contrast, domain-specific ranges, usage guidance, and a clean Args/Returns section. It is front-loaded with the core purpose and every sentence serves a distinct function. No fluff or repetition.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having an output schema, the description still details the return format and the logic for 'passed'. It covers parameter semantics, domain values, file structure, and usage timing. For a tool with only two parameters and a well-defined output schema, this is complete and leaves no ambiguity for the agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description carries the full burden. It explains dataset_dir as 'Directory containing one CSV per table' and domain as one of four enumerated values. This fully compensates for the schema's lack of descriptions, giving the agent everything needed to populate both parameters correctly.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a precise verb and resource: 'Flag values that are physiologically or financially impossible for a domain.' It immediately contrasts with the sibling audit_dataset, distinguishing its purpose clearly. The specific built-in ranges reinforce the scope, making the tool's function unmistakable.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It explicitly states when to use: 'after generating a dataset in a regulated or measurement-heavy domain' and when the user asks about plausibility. It also names the alternative audit_dataset and differentiates by checking 'internal consistency' vs 'outside world' values. This gives the agent clear routing criteria.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Validate a misata.yaml document at two levels.
Runs both checks in sequence:
Structural — the published JSON Schema (correct field types, required fields, enum values). Catches typos and shape errors.
Semantic — misata.validate_schema (probabilities sum to 1.0,
every foreign_key has a matching Relationship, no cycles, outcome
curves reference real columns, etc.). These are the rules that
would crash generation; the error messages include suggested fixes.
Use this when an agent has authored or edited a misata.yaml on the
user's behalf and wants to confirm it parses and will actually
generate before invoking generate_dataset.
Args: yaml_text: The full contents of a misata.yaml file as a string.
Returns:
{"valid": true} if both checks pass; otherwise
{"valid": false, "errors": [...], "stage": "structural"|"semantic"}
with the layer that failed first.
| Name | Required | Description | Default |
|---|---|---|---|
| yaml_text | Yes |
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description significantly enriches the annotations (readOnlyHint, idempotentHint) by detailing the two-step execution (structural then semantic), giving examples of semantic rules, and specifying the exact return format including the 'stage' of failure. It doesn't mention any rate limits or auth needs, hence not a 5, but it provides strong context beyond the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely well-structured and concise for the complexity it handles. It uses markdown headers for the two validation levels, a 'Use this when' section for guidance, and clearly formatted Args and Returns blocks. Every sentence provides valuable information without unnecessary verbosity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema and detailed annotations, the description perfectly complements them. It explains the two types of validation, the order of execution, and what happens on failure. This is complete for an agent to understand the tool's behavior, when to use it, and what to expect in return.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Although schema coverage is 0% (no 'description' field in the schema for the parameter), the description explicitly documents the 'yaml_text' parameter: 'The full contents of a misata.yaml file as a string.' This fully compensates for the schema's lack of description, so while it increases transparency, the schema handles the variable type.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool validates a 'misata.yaml' document at two distinct levels (structural and semantic). It uses a specific verb+resource pattern and the detailed explanation of the two checks distinguishes it from sibling tools like 'validate' on other resources or 'inspect_schema'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly states when to use the tool: 'Use this when an agent has authored or edited a misata.yaml on the user's behalf...'. It also clearly defines the alternative by mentioning 'before invoking generate_dataset', which is a sibling tool, and contrasts parsing with semantic validation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
v0.1.9audit_datasetvalidate_domainv0.1.0generate_datasetgenerate_from_schemainspect_schemalist_domainspreview_storyseed_databasevalidate_yamlScored across 9 tools
Most tools are clearly distinct: generate_from_schema and generate_dataset both create data but via different input modes (schema vs story), which could cause some confusion. The remaining tools (validate_yaml, audit_dataset, validate_domain, inspect_schema, preview_story, list_domains, seed_database) each have a distinct purpose.
Tool names mostly follow a verb_noun pattern: generate_from_schema, validate_yaml, audit_dataset, validate_domain, generate_dataset, inspect_schema, list_domains, preview_story, seed_database. The pattern is consistent (verb + object), though generate_from_schema and generate_dataset are slightly redundant in naming style.
9 tools is well-scoped for a synthetic data generation platform. Each tool covers a distinct phase: schema definition, validation, generation, inspection, auditing, domain checking, and database seeding.
The tool surface covers the full lifecycle: design (generate_from_schema, preview_story, inspect_schema), validate (validate_yaml), generate (generate_dataset, seed_database), and verify (audit_dataset, validate_domain). Minor gaps include no explicit tool for editing/deleting generated datasets, but the core workflows are well covered.
Generate realistic, FK-consistent synthetic test data for your databases from your AI assistant.
Generate realistic relational test data — 156 field types, 22 locales, JSON/CSV/SQL, free previews.
Ask data questions in natural language. Get SQL, insights, and charts from your databases.
Privacy-safe synthetic financial data for LatAm fintech, AI agents, testing and ML.
