Agentic Data Cleaner -- notebook + Muse connector

Finds and fixes problems in tabular data (CSV, TSV, Excel, JSON, Parquet, ...): duplicates, placeholder nulls (N/A, ?), numbers/dates stored as text, spelling variants, encoding damage, outliers, personal data. Safe fixes apply automatically; anything that deletes or invents data waits for a human yes. Every run produces the cleaned file, an audit report and a standalone replay script.

One engine, three ways to use it:

datacleaner/engine.py  ──►  agentic_data_cleaner.ipynb   (generated by tools/build_notebook.py -- can't drift)
        │
        └──────────────►  server/  ──►  MCP endpoint  /mcp   (what Muse connects to)
                                   └──►  REST API      /v1/...  (OpenAPI at /docs)

1. Use the notebook

pip install -r requirements.txt jupyter
jupyter lab agentic_data_cleaner.ipynb     # Run All -- cleans input/uncleaned_data.csv into output/

Or as a library: from datacleaner import AgenticDataCleaner; AgenticDataCleaner("data.csv").run(interactive=False). A model is optional (LLM_* in .env); without one, deterministic rules plan everything.

Related MCP server: data-quality-loop

2. Run the Muse connector locally

pip install -r requirements.txt
cp .env.example .env            # then edit: API_KEYS, PUBLIC_BASE_URL, ALLOWED_HOSTS
set -a; . ./.env; set +a
uvicorn server.app:app --port 8000
curl localhost:8000/healthz     # {"ok":true}

MCP endpoint: http://localhost:8000/mcp (Streamable HTTP, Authorization: Bearer <key>). REST docs: /docs.

Tools Muse sees

Input can be file_base64, csv_text, a public file_url (https only, SSRF-guarded) or an upload_id.

3. Deploy

Pick one -- both configs are in this folder:

# Fly.io (fly.toml):  fly launch --no-deploy  ->  fly secrets set API_KEYS=...  ->  fly deploy
# Render   (render.yaml):  push to GitHub -> New + -> Blueprint -> read the API key from the dashboard

Or anywhere with Docker (untested -- no Docker here): docker build -t data-cleaner . && docker run -p 8000:8000 --env-file .env data-cleaner

Then set the env vars to your real domain (PUBLIC_BASE_URL, ALLOWED_HOSTS) and always set API_KEYS (empty means open). Generate a key with: python -c "import secrets; print(secrets.token_urlsafe(32))".

Before you submit, run the reviewer test against your live URL:

python tools/preflight.py https://your-domain.example YOUR_API_KEY

It checks everything Muse's reviewers will touch (public pages, auth, MCP handshake, a real clean + download) and exits 0 only when all pass. Scaling: datasets live in process memory, so run one instance/worker; to go multi-instance, move server/store.py and the meters in server/quota.py to Redis/S3.

Security model (what is actually enforced, and tested)

• Bearer-key auth (constant-time compare); each client's datasets are isolated -- "not yours" and "doesn't exist" look identical.

• Unguessable dataset/download/upload tokens; datasets, artifacts and upload slots expire (default 1 h) and can be deleted on demand.

• Only allow-listed formats are parsed (csv/tsv/txt/json/jsonl/xlsx/parquet). No pickle, HDF5, SQLite, HTML/XML, legacy .xls; no server file paths; row/column/size caps; zip-bomb checks on .xlsx; Parquet metadata checked before reading.

• URL fetching: https/443 only, no credentials, every address must be public (DNS is resolved once and the connection is pinned to that IP), redirects re-validated, size and time capped.

• The actions are a fixed registry -- no code execution, no user regexes. A model (if you enable one) cannot lower risk, invent actions or touch missing columns; and LLM_ENABLED is off by default with cell values redacted when on.

• Errors return generic messages (stack traces stay in server logs); logs hold ids and counts, never data.

• DNS-rebinding protection on /mcp (ALLOWED_HOSTS), nosniff, no-store, strict CSP on the public pages.

Tests

pip install -r requirements-dev.txt
pytest                # 89 tests: engine, planner guard rails, server, SSRF, tiers, live MCP client, notebook end-to-end

What is not done / not verified (please read)

• Not deployed and not submitted -- I can't do either from here. You need a host, a domain, real legal text and to fill the form (muse/submission.md).

• Muse's exact requirements are unverified. I could not confirm how Muse authenticates to a connector (bearer key vs OAuth), how payment entitlement reaches your server, or the review criteria. Check Meta's current developer docs. Auth is one function (server/app.py::_client_id); entitlement is one function (server/quota.py::tier_for).

• The Dockerfile is untested (no Docker in the build environment); the app itself was tested via uvicorn + a real MCP client.

• Scheduled cleaning is not built -- it is the feature most likely to justify a subscription (see docs/monetization.md).

• Long calls: a 200k-row file took ~8 s in the sandbox. Confirm Muse's tool timeout before advertising that size.

• Legal pages are drafts, not legal advice.

See CHANGELOG.md for every bug fixed in the original notebook, and docs/monetization.md for the earning model.

Built by Ishan Makkar -- https://github.com/iishanmakkar/dc-mcp