Feature

What it does

Boundaries

Ingest

CSV/TSV (incl. semicolon-delimited, comma-decimal Spanish formats), Excel, Parquet, JSON/NDJSON, SQLite, a folder glob, a URL, or pasted text.

A single ingest call runs in a background thread with a 5-minute cap; very large files should be Parquet or pre-filtered CSV rather than a 50-sheet Excel workbook. Ingesting under a name that is already taken is refused (and the existing dataset left untouched): use refresh to reload it or pick another name.

Transform

21 step kinds (filter, select, drop, rename, cast, fill_null, drop_duplicates, derive, split_column, text ops, replace, bin, date_parts, group, pivot, unpivot, join, union, sort, sample, window, raw SQL) — every one previewable before it's applied.

The sql step and the SQL query tool both accept exactly one read-only statement; no writes, no multi-statement scripts.

Undo / redo / recipe

Every version is a real table; a side panel shows the full recipe, exports it as a runnable SQL script, and can replay it against re-ingested source data.

Branching after an undo discards the redone-away versions (like any editor's undo stack) — there's one timeline, not a tree.

Quality rules

9 rule kinds (not_null, unique, accepted_values, range, regex, row_count, freshness, referential integrity, custom SQL), run on demand, with a pass/fail history. A failed unique rule says how many values repeat, how many rows are extra copies, and how many rows would remain after de-duplication.

Rules check the dataset's current version; they don't run automatically on ingest unless you re-run them yourself.

Charts & dashboards

11 chart kinds, aggregated in SQL (never a full table pulled client-side), rendered interactively (Vega-Lite) and as PNG (for export/agent use); dashboards combine saved charts with SQL-expression KPI tiles.

Charts read the dataset's current version at render time — a chart doesn't freeze past data, it reflects the latest cleaning.

Dashboards as code

A dashboard as a plain YAML file over a reusable semantic layer (named metrics/dimensions per dataset) — filters, tabs, rows of metric/chart/table/text widgets, for loops over top-N dimension values, if conditions, and derived metrics (e.g. a metric shifted a month back). Validated with a hint on every issue, rendered with each widget's own SQL and error (one bad widget never breaks the rest), kept in a 20-version history with diffs, and exportable into the item-based dashboards above. See docs/DASHBOARDS_AS_CODE.md.

Everything compiles to read-only SQL over a dataset's own view, same trust level as a chart filter.

Models

Supervised learning (auto task detection, cross-validation, feature importance, confusion matrix), k-means clustering (elbow + silhouette), PCA, isolation-forest anomaly detection, Holt-Winters forecasting with interval and a documented seasonal-naive fallback. A "Write results to" control (new dataset by default, with a one-line explanation of the alternative) chooses where predictions/labels/flags land: new_dataset (default) puts them in their own new dataset (<source>__model_<id> etc.), the source dataset untouched; new_version writes onto the source itself instead, and even then only ever adds a column; none just shows the result. Every copied source column — in either destination — keeps its exact DuckDB type (a DATE stays a DATE, a DECIMAL stays a DECIMAL); only the new prediction/label columns are computed in pandas.

Training samples down at 200k rows; automatic feature selection excludes near-unique identifier-like columns (reported back) to avoid a one-hot blow-up — pass an explicit feature list to override.

Delete a dataset

Removes a dataset, all its versions, and cascades to its quality rules, charts, and models; a dashboard that used one of those charts keeps working (it shows the item as unavailable) rather than breaking. Available from the Datasets screen (with a confirmation dialog listing what depends on it), DELETE /api/datasets/{name} (?force=true to go past dependents), and the data_ingest MCP tool's action="delete".

Blocked by default when charts/dashboards/models depend on the dataset — pass force/force=true to delete anyway.

Analysis log

Every operation (ingest, transform, quality run, chart, model, export...) gets an id like N-000123, with input, a one-line summary, timing, and whether it came from you or the assistant.

The log is append-only and capped per-field in size; it's an audit trail, not a full data backup.

Ask your data

A plain-language question becomes one SQL query (shown, not hidden) run through a shared local language model via Hoard Link.

Needs a resolved LLM backend (Faustus, a local llama.cpp server, Ollama, or an OpenAI-compatible endpoint) — with none configured it says so clearly instead of guessing.

Lab

Deeper data science on top of the workbench, with its own page (seven tabs — Explore, Models, Diagnose, Optimize, Compare & drift, Pipeline, Report): full EDA (correlation, null patterns + imputation preview, IQR/z-score outliers, encoding suggestions with one-click apply-as-step) and a 0-100 quality score (only ever shows 100 when every component is actually perfect); a pluggable model registry (linear/ridge/logistic, random forest, gradient boosting, extra trees, k-NN, MLP, Gaussian process, plus XGBoost/LightGBM if installed) with versioned, persisted models; model diagnostics (residuals, predicted-vs-actual, error by group, bias over time, over/under-fitting, calibration, and for classifiers a plotted ROC curve — one-vs-rest per class when there are more than two); hyperparameter tuning (Optuna TPE or a randomized fallback); explanations (SHAP or permutation importance + partial dependence); Bayesian optimization of a model's inputs (GP surrogate + EI/UCB) and multi-objective Pareto fronts; drift detection and curve/series comparison between datasets; a PDF report; and a visual pipeline editor (add/reorder/edit/preview steps as a node chain, applied through the same step engine as the Datasets page). See docs/LAB.md. Datasets and Models each link straight into it ("Open in Lab" / "Open in Lab registry").

XGBoost/LightGBM/Optuna/SHAP are optional (requirements-lab.txt) — every feature degrades to a documented fallback without them.

Variable

Default

Meaning

NIGHTINGALE_PORT / PORT

5189

Preferred port; PORT_STRICT=1 pins it, otherwise the first free port from there.

NIGHTINGALE_DATA_DIR

<repo>/data (data-demo with --demo)

DuckDB file, metadata SQLite, mcp-token, exports, chart images.

NIGHTINGALE_ALLOWED_HOSTS

Extra host names accepted behind a tunnel (see below).