Analyze frequency, cardinality, and distribution of categorical (text) columns. Args: dataset_id (str): ID of the dataset. top_n (int): How many of the most frequent values are listed per column. Default is 10. Returns: dict: For each categorical column: unique count, most frequent value and its count, missing count, and the percentages of the top_n values.

Calculate correlations between numerical features. Args: dataset_id (str): ID of the dataset. Returns: dict: Dataset ID and correlation values between numerical columns (None for constant columns).

Analyze the distribution, spread, and shape of numerical columns. Args: dataset_id (str): ID of the dataset. Returns: dict: Descriptive statistics, skewness, and kurtosis (None where it cannot be computed).

Analyze the correlation between two numerical columns. Args: dataset_id (str): ID of the dataset. col_name1 (str): First numerical column. col_name2 (str): Second numerical column. Returns: dict: Dataset ID, column names, and correlation value (None if it cannot be computed).

Automatically choose and save appropriate charts for a dataset:

• histograms of numerical columns

• count charts of low-cardinality categorical columns

• correlation heatmap (2+ numerical columns)

• target column charts (scatter plots of the most correlated features for a numerical target, class balance for a categorical target) Args: dataset_id (str): ID of the dataset. target_column (str, optional): Target column to focus the charts on. Returns: dict: Dataset ID, the charts created (each with a reason), and skipped columns.

Generate a categorical bar chart. Args: dataset_id (str): ID of the dataset. column_name (str): Categorical column to visualize. Returns: dict: Dataset ID, column name, and saved image filename.

Cross-validate several candidate models on the TRAINING rows of a split and rank them by their average score. More reliable than compare_models: every model is scored on several folds (mean and standard deviation), timings are included, and the test rows are never touched. Nothing is stored in Models; train the winner with its own tool (or compare_models) afterwards. The split must have only numerical, complete features. Args: split_id (str): ID of a split made by train_test_split. models (list[str], optional): Training tool names to try (same names and defaults as compare_models). task (str): "auto" (inferred from the target), "regression" or "classification". Default is "auto". cv (int): Number of folds, 2 to 10. Classification is stratified and uses at most as many folds as the smallest class has rows. Default is 5. sort_by (str, optional): Metric that ranks the models (mean over folds). Regression: r2 (default), mae, mse, rmse. Classification: f1 (default), accuracy, balanced_accuracy, precision, recall, roc_auc. include_slow (bool): Also run SVM on large data (over 10,000 training rows). Default is False. Returns: dict: leaderboard with mean and std of every metric, mean fit time, which models are statistically comparable to the best (within one standard deviation), failed/skipped models and notes.

Turn a numerical column into bins (e.g. age -> age groups). Bin edges are learned from the training rows only; the first and last bins are open-ended so unseen test values still fall in a bin. Args: dataset_id (str): ID of a dataset or of a split. column (str): Numerical column to bin. bins (int): Number of bins for "quantile" and "uniform", 2 to 50. Default is 5. strategy (str): "quantile" (equal-sized groups), "uniform" (equal-width bins) or "custom". Default is "quantile". edges (list[float], optional): Required for "custom". The inner boundaries, e.g. [18, 65] makes three bins: <=18, 18-65 and >65. Bins include their upper boundary. labels (list[str], optional): Names for the bins (one per bin). If omitted, bins are numbered 0, 1, 2, ... new_column (str, optional): Name of the new column. Default is "_bin". drop_original (bool): Drop the original column. Default is False. Returns: dict: The inner edges used, the number of bins and how many training rows fall in each bin.

Generate and save box plots for one or all numerical columns. Args: dataset_id (str): ID of the dataset. column_name (str, optional): Numerical column to visualize. If omitted, generates box plots for all numerical columns. Returns: dict: Dataset ID, selected columns, and saved image filename.

Winsorize extreme values: values beyond the IQR bounds are set to the bound instead of being removed. Columns where IQR is 0 (flags, one-hot columns) are skipped. Capped columns become float. Args: dataset_id (str): ID of the dataset. columns (list[str], optional): Numerical columns to cap. Default is all numerical columns (consider excluding the target column). Returns: dict: how many values were capped in each column, the bounds used and the skipped columns.

Per-class precision, recall, F1 and support, plus accuracy and macro / weighted averages. Use EITHER a trained classification model (model_id) OR your own lists of true and predicted labels. Args: model_id (str, optional): ID of a trained classification model. The report is recomputed from its stored data. dataset (str): "test", "train" or "both". Only used with model_id. Default is "test". y_true (list, optional): True labels (use together with y_pred). y_pred (list, optional): Predicted labels, same length as y_true. Returns: dict: accuracy, per_class metrics, averages, a printable text table and (for a model) ROC-AUC.

Return Pandas data types for each column. Args: dataset_id (str): ID of the dataset. Returns: dict: column_types (column name -> dtype).

Train several candidate models on the same split, score them all on the same train and test data, and return a ranked leaderboard. The trained models are stored (model_id) so you can plot or inspect them. The split must have only numerical, complete features (run encode_categorical / impute_missing first). Each model uses the default settings of its own training tool; tune the winner afterwards. Args: split_id (str): ID of a split made by train_test_split. models (list[str], optional): Training tool names to try, e.g. ["ridge_regression","random_forest_regressor"]. Default: linear, ridge, lasso, decision tree, random forest, gradient boosting and xgboost (regression); logistic, knn, decision tree, random forest, svm, gradient boosting and xgboost (classification). elastic_net is available but not tried by default. Models whose package is missing are skipped and listed. task (str): "auto" (inferred from the target), "regression" or "classification". Default is "auto". sort_by (str, optional): Metric that ranks the models, measured on the test data. Regression: r2 (default), mae, mse, rmse, mape_percent. Classification: f1 (default), accuracy, balanced_accuracy, precision, recall, roc_auc, f1_macro (multiclass only). include_slow (bool): Also run SVM on large data (over 10,000 training rows). Default is False. keep (str): "all" keeps every trained model; "best" deletes all but the winner (less clutter). Default is "all". Returns: dict: leaderboard (rank, model_id, algorithm, test score on sort_by, full train/test metrics, generalisation gap, training seconds), best_model_id, failed/skipped models, and notes (ties, overfitting, imbalance).

Confusion matrix as numbers (rows = true class, columns = predicted class), plus the most confused class pair and, for two classes, TN / FP / FN / TP. For a picture use confusion_matrix_plot. Use EITHER a trained classification model (model_id) OR your own lists of true and predicted labels. Args: model_id (str, optional): ID of a trained classification model. dataset (str): "test" or "train". Only used with model_id. Default is "test". normalize (str, optional): None for counts, "true" (each row sums to 1: recall per class), "pred" (each column sums to 1: precision per class) or "all" (whole matrix sums to 1). y_true (list, optional): True labels (use together with y_pred). y_pred (list, optional): Predicted labels, same length as y_true. Returns: dict: labels, matrix, most_confused and (binary) the TN/FP/FN/TP breakdown.

Generate and save a confusion matrix of a classification model on its test data. Args: model_id (str): ID of the trained classification model. normalize (bool): Show each row as a fraction of its true class. Default is False. Returns: dict: Model ID, classes, confusion matrix values, and saved image filename.

Convert a column to a specified data type. Args: dataset_id (str): ID of the dataset. col_name (str): Column to convert. convert_type (str): One of int, float, string, category, datetime. "int" needs a column without missing values (fill or drop them first). Returns: dict: the column and its new dtype.

Save a copy of a dataset under a new id, e.g. before experimenting with cleaning steps. The original is left untouched and the copy can be changed independently. Args: dataset_id (str): ID of the dataset to copy. Returns: dict: dataset_id, rows, columns and column_names of the new copy.

Generate and save a count plot showing how often each value of a column occurs. Works for categorical columns and for numerical columns with few distinct values. Args: dataset_id (str): ID of the dataset. column_name (str): Column to count. top_n (int): Maximum number of values to show. Default is 20. Returns: dict: Dataset ID, column name, counts shown, and saved image filename.

Create a new column from a row-wise arithmetic expression over existing columns. Examples: "selling_price / km_driven", "log(km_driven + 1)", "(year - 2000) * 2", "engine cc * power". Use backticks around column names that contain spaces. Available functions: log, log10, log1p, sqrt, exp, abs, sin, cos, tanh, floor, ceil. Method calls (x.mean()) and assignments are rejected because they would compute statistics per split and leak test data. Recommended: pass a split_id so the target column cannot be used in the expression by accident. Args: dataset_id (str): ID of a dataset or of a split. name (str): Name of the new column. expression (str): The expression to evaluate for every row. overwrite (bool): Replace the column if the name already exists. Default is False. Returns: dict: The new column, its dtype, missing count and a short preview.

Return a comprehensive summary of dataset structure, dimensions, types, and missing values. Args: dataset_id (str): ID of the dataset. Returns: dict: rows, columns, column_names, datatypes_of_columns, no_of_null_in_each_column and summary_of_numerical_columns.

Return the number of rows and columns. Args: dataset_id (str): ID of the dataset. Returns: dict: rows and columns.

Train a decision tree classifier. Easy to interpret and needs no scaling, but overfits easily when max_depth is not limited. The split must have only numerical, complete features. Args: split_id (str): ID of a split made by train_test_split. criterion (str): Split quality measure: "gini", "entropy" or "log_loss". Default is "gini". max_depth (int, optional): Maximum tree depth. Default is None (unlimited); 3-10 is a common range. min_samples_split (int): Minimum rows needed to split a node. Default is 2. min_samples_leaf (int): Minimum rows in a leaf. Default is 1. class_weight (str, optional): "balanced" to counter imbalanced classes. Default is None. random_state (int): Seed. Default is 42. Returns: dict: model_id, train/test metrics (accuracy, precision, recall, F1, ROC-AUC), top features and warnings.

Train a decision tree regressor. Easy to interpret, captures non-linear effects, but overfits easily when max_depth is not limited. The split must have only numerical, complete features (no scaling is needed). Args: split_id (str): ID of a split made by train_test_split. max_depth (int, optional): Maximum tree depth. Default is None (unlimited); 3-10 is a common range. min_samples_split (int): Minimum rows needed to split a node. Default is 2. min_samples_leaf (int): Minimum rows in a leaf. Default is 1. random_state (int): Seed. Default is 42. Returns: dict: model_id, train/test metrics (R2, MAE, MSE, RMSE, MAPE), top features and warnings.

Remove a dataset from the session. Splits and models already made from it are kept. Dataset ids are never reused, so old references cannot point at a different dataset. Args: dataset_id (str): ID of the dataset to delete. Returns: dict: The deleted id and the remaining dataset ids.

Generate statistical summaries for numerical and categorical columns. Args: dataset_id (str): ID of the dataset. Returns: dict: the numerical statistical summary, the number of unique values of each categorical column, and the unique values with their counts for each categorical column.

Detect outliers in numerical columns using the IQR method (values beyond Q1 - 1.5IQR or Q3 + 1.5IQR). Columns where IQR is 0 (constant columns, 0/1 flags, one-hot columns) are skipped and listed. Args: dataset_id (str): ID of the dataset. columns (list[str], optional): Numerical columns to check. Default is all numerical columns. max_values (int): Maximum number of outlier values listed per column (the count is always complete). Default is 50. Returns: dict: outliers_bounds, per-column outlier count with example values, and skipped_columns.

Drop a column from the dataset. Args: dataset_id (str): ID of the dataset. column_name (str): Name of the column to remove. Returns: dict: Dataset ID and deleted column name.

Remove rows or columns containing missing values. Args: dataset_id (str): ID of the dataset. axis (str): "rows" removes rows with at least one missing value (default); "columns" removes columns with at least one missing value. Returns: dict: row and column counts before and after.

Detect duplicate records and report their count. Args: dataset_id (str): ID of the dataset. Returns: dict: duplicated_rows_count (rows that repeat an earlier row).

Train an Elastic Net regression, a mix of L1 (Lasso) and L2 (Ridge) regularisation. The split must have only numerical, complete features. Args: split_id (str): ID of a split made by train_test_split. alpha (float): Overall regularisation strength, greater than 0. Default is 1.0. l1_ratio (float): Mix between 0 (pure Ridge) and 1 (pure Lasso). Default is 0.5. fit_intercept (bool): Learn an intercept. Default is True. max_iter (int): Maximum solver iterations. Default is 10000. scale (bool): Standardise the features first. Strongly recommended. Default is True. Returns: dict: model_id, train/test metrics (R2, MAE, MSE, RMSE, MAPE), top features and warnings.

Encode categorical columns, choosing the encoding automatically. Recommended: run train_test_split first and pass the split_id. If a dataset id is used, pass columns explicitly so the target column is not encoded by accident. Args: dataset_id (str): ID of a dataset or of a split. columns (list[str], optional): Categorical columns to encode. If omitted, all categorical columns. method (str): "auto", "onehot" or "label". With "auto", columns with at most max_onehot_categories categories are one-hot encoded and the rest are label encoded. max_onehot_categories (int): Category limit used by "auto". Default is 10. drop_first (bool): Passed to one-hot encoding. Default is False. Returns: dict: Which columns were one-hot encoded, which were label encoded, and the details.

Export a dataset to CSV, Excel, JSON, Parquet or pickle. Args: dataset_id (str): ID of the dataset. extension_name (str): One of csv, json, xls, xlsx, parquet, pkl. file_name (str): Output file name or path. The extension is added if it is missing, e.g. "cars" -> "cars.csv". Returns: dict: The path that was written, the format and the number of rows and columns.

Split a date/datetime column into numerical parts that models can use. Values that cannot be parsed become missing values (the call fails if more than half cannot be parsed). Args: dataset_id (str): ID of a dataset or of a split. column (str): Date column (text or datetime). features (list[str], optional): Parts to extract. Choose from: year, month, day, dayofweek, dayofyear, weekofyear, quarter, hour, minute, is_weekend, is_month_start, is_month_end, epoch_days. Default is year, month, day, dayofweek, is_weekend, quarter. drop_original (bool): Drop the original date column. Default is True. cyclical (bool): Also add "_sin" and "_cos" columns for cyclic parts (month, dayofweek, hour, ...) so that e.g. December and January are close. Default is False. date_format (str, optional): strftime format such as "%d/%m/%Y". Recommended when dates are not ISO formatted. Returns: dict: The new columns, and any extracted parts that are constant (e.g. hour for date-only data).

Generate and save a bar chart of the most important features of a trained model. Uses feature_importances_ (tree models) or absolute coef_ (linear models). Args: model_id (str): ID of the trained model. top_n (int): Number of top features to show. Default is 15. Returns: dict: Model ID, top features with importance values, and saved image filename.

Remove unhelpful numerical features. Only numerical feature columns are evaluated; other columns are kept. Scored methods need no missing values (run impute_missing first). Args: dataset_id (str): ID of a dataset or of a split. For a split the target is known already. target_column (str, optional): Target column. Required for a plain dataset when method is k_best or mutual_info; it is never removed. method (str): "k_best" (keep the k best by F-test), "mutual_info" (keep the k best by mutual information), "variance" (drop near-constant features) or "correlation" (drop one of each highly correlated pair). k (int): Number of features kept by k_best and mutual_info. Default is 10. threshold (float, optional): Minimum variance for "variance" (default 0.0) or maximum absolute correlation for "correlation" (default 0.9). Returns: dict: Selected and dropped columns, with scores where available.

Replace missing numerical values with the column mean. Args: dataset_id (str): ID of the dataset. col_name (str, optional): Only this column is filled. If omitted, all numerical columns are processed. Returns: dict: how many values were filled in each column.

Replace missing numerical values with the column median. Args: dataset_id (str): ID of the dataset. col_name (str, optional): Only this column is filled. If omitted, all numerical columns are processed. Returns: dict: how many values were filled in each column.

Replace missing category (text) values with the most frequent value of the column. Args: dataset_id (str): ID of the dataset. col_name (str, optional): Only this column is filled. If omitted, all category columns are processed. Returns: dict: how many values were filled in each column (columns with no values at all are skipped).

Find highly correlated pairs of numerical features. Args: dataset_id (str): ID of the dataset. threshold (float): Minimum absolute correlation (0 to 1). Returns: dict: Highly correlated feature pairs and their correlation values.

Analyze relationships between numerical features and a target column. Args: dataset_id (str): ID of the dataset. target_column (str): Name of the target column. Returns: dict: Features ranked by their correlation with the target.

Retrieve metadata of a registered dataset: number of rows and columns and the column names. Args: dataset_id (str): ID of the dataset. Returns: dict: dataset_id, rows, columns and column_names.

Show the current state of a train/test split: shapes, feature columns, data types, missing values and the preprocessing steps applied so far. Args: split_id (str): ID of the split created by train_test_split. Returns: dict: Split details.

Train a gradient boosting classifier (scikit-learn): shallow trees added one at a time, each correcting the errors of the previous ones. Often the most accurate classical model on tabular data. The split must have only numerical, complete features (no scaling is needed). Args: split_id (str): ID of a split made by train_test_split. n_estimators (int): Number of boosting stages. Default is 100. learning_rate (float): Shrinkage per stage, greater than 0. Lower values need more estimators. Default is 0.1. max_depth (int): Depth of each tree. Default is 3. subsample (float): Fraction of rows used per tree, in (0, 1]. Below 1 gives stochastic boosting. Default is 1.0. min_samples_leaf (int): Minimum rows in a leaf. Default is 1. random_state (int): Seed. Default is 42. Returns: dict: model_id, train/test metrics (accuracy, precision, recall, F1, ROC-AUC), top features and warnings.

Train a gradient boosting regressor (scikit-learn): shallow trees added one at a time, each correcting the errors of the previous ones. Often the most accurate classical model on tabular data. The split must have only numerical, complete features (no scaling is needed). Args: split_id (str): ID of a split made by train_test_split. n_estimators (int): Number of boosting stages. Default is 100. learning_rate (float): Shrinkage per stage, greater than 0. Lower values need more estimators. Default is 0.1. max_depth (int): Depth of each tree. Default is 3. subsample (float): Fraction of rows used per tree, in (0, 1]. Below 1 gives stochastic boosting. Default is 1.0. min_samples_leaf (int): Minimum rows in a leaf. Default is 1. random_state (int): Seed. Default is 42. Returns: dict: model_id, train/test metrics (R2, MAE, MSE, RMSE, MAPE), top features and warnings.

Automatically handle missing values based on their percentage.

• up to 5% missing: drop the rows that contain the missing values

• 60% or more missing: drop the column

• 5%-60%: leave unchanged (use the fill_missing_* tools or impute_missing) Args: dataset_id (str): ID of the dataset. Returns: dict: dropped columns, the columns used for row removal and the row counts.

Generate and save a correlation heatmap of all numerical columns. Args: dataset_id (str): ID of the dataset. method (str): Correlation method: pearson, spearman or kendall. Default is pearson. Returns: dict: Dataset ID, method, columns used, and saved image filename.

Generate and save a histogram for a numerical column. Args: dataset_id (str): ID of the dataset. column_name (str): Numerical column to visualize. bins (int): Number of histogram bins. Default is 10. Returns: dict: Dataset ID, column name, and saved image filename.

Fill missing values. Recommended: use a split_id so the fill values come from training data only. Args: dataset_id (str): ID of a dataset or of a split. columns (list[str], optional): Columns to impute. If omitted, all columns with missing values. strategy (str): "auto" (median for numerical, most_frequent for categorical), "mean", "median", "most_frequent", "constant" or "knn" (numerical only). fill_value (int, float or str, optional): Value used by "constant" (default 0 for numerical, "missing" for categorical). n_neighbors (int): Neighbours used by "knn". Default is 5. Returns: dict: Columns imputed, missing counts before, and the value used for each column.

Combine pairs of numerical columns into new features, e.g. price per km or area * rooms. Pass EITHER columns (every pair among them, in the order given) OR pairs (exact pairs). Division by zero gives a missing value. Missing values must be imputed afterwards if the model needs them gone. For squares or cubes of a single column use polynomial_features or transform_feature. Args: dataset_id (str): ID of a dataset or of a split. columns (list[str], optional): Numerical columns; all pairs (a, b) with a before b are created. pairs (list[list[str]], optional): Explicit pairs such as [["km_driven","year"]]. For "divide" the order matters (first / second). operations (list[str], optional): Any of "multiply", "divide", "add", "subtract". Default is ["multiply"]. Names are "x", "div", "plus", "minus". max_new_features (int): Safety limit on how many columns may be created. Default is 50. Returns: dict: The new feature names and how many rows had a zero denominator.

Train a k-nearest-neighbours classifier: a row gets the majority class of its closest training rows. Distance based, so features are standardised by default. Slow on very large data. The split must have only numerical, complete features. Args: split_id (str): ID of a split made by train_test_split. n_neighbors (int): Number of neighbours. Default is 5. weights (str): "uniform" (all neighbours count equally) or "distance" (closer ones count more). Default is "uniform". p (int): Distance type: 1 = Manhattan, 2 = Euclidean. Default is 2. scale (bool): Standardise the features first (strongly recommended). Default is True. Returns: dict: model_id, train/test metrics (accuracy, precision, recall, F1, ROC-AUC) and warnings.

Label encode categorical columns: every category becomes an integer (alphabetical order). Best for high-cardinality columns (e.g. brand) and tree-based models. Categories that only appear in the test data become -1. Args: dataset_id (str): ID of a dataset or of a split. columns (list[str], optional): Categorical columns to encode. If omitted, all categorical columns. Returns: dict: Encoded columns and the category -> integer mapping.

Train a Lasso (L1-regularised) linear regression. It can shrink weak coefficients to exactly zero, so it also acts as feature selection. The split must have only numerical, complete features. Args: split_id (str): ID of a split made by train_test_split. alpha (float): Regularisation strength, greater than 0. Default is 1.0. If the target has a large scale (e.g. prices in the millions) a value of 1 may be too small to have any effect. fit_intercept (bool): Learn an intercept. Default is True. max_iter (int): Maximum solver iterations. Default is 10000. scale (bool): Standardise the features first. Strongly recommended. Default is True. Returns: dict: model_id, train/test metrics (R2, MAE, MSE, RMSE, MAPE), top features and warnings.

Generate and save a learning curve (score vs training set size) for a trained model. Helps to see whether the model is overfitting, underfitting, or needs more data. Args: model_id (str): ID of the trained model. cv (int): Number of cross-validation folds. Default is 5. scoring (str, optional): sklearn scoring name. Defaults to r2 for regression and accuracy for classification. Returns: dict: Model ID, scoring used, training sizes, mean scores, and saved image filename.

Train an ordinary least squares linear regression on a split and evaluate it on train and test. The split must have only numerical, complete features (run encode_categorical / impute_missing first). Args: split_id (str): ID of a split made by train_test_split. fit_intercept (bool): Learn an intercept. Default is True. positive (bool): Force all coefficients to be non-negative. Default is False. scale (bool): Standardise the features first (same predictions, but coefficients become comparable between features). Default is True. Returns: dict: model_id (use it in plots and other model tools), train/test metrics (R2, MAE, MSE, RMSE, MAPE), top features and warnings such as possible overfitting.

Generate and save a line plot showing trends between two columns. Args: dataset_id (str): ID of the dataset. x_column (str): Column for the X-axis. y_column (str): Numerical column for the Y-axis. Returns: dict: Dataset ID, column names, and saved image filename.

Return all column names. Args: dataset_id (str): ID of the dataset. Returns: dict: column_names.

List all datasets currently available in the session. Returns: dict: {"datasets": [metadata of every dataset]}.

Load a CSV file from disk into a new dataset. Args: file_path (str): Full path to the CSV file on the machine running the server. delimiter (str): Column separator. Default is ",". Returns: dict: dataset_id, rows, columns and column_names of the new dataset.

Load a dataset into a pandas DataFrame from uploaded file bytes or from a URL / file path. Supported extensions: .csv, .xls, .xlsx, .json, .parquet. To load a file that is on the machine running the server, prefer load_csv / load_excel / load_json / load_parquet. Args: file (bytes, optional): Raw file content. Needs file_name so the format is known. file_name (str, optional): Name of the file (its extension selects the reader). dataset_url (str, optional): URL (or path) to read from. If the URL has no extension, pass file_name. Returns: dict: dataset_id, rows, columns and column_names of the new dataset.

Load an Excel file (.xls or .xlsx) from disk into a new dataset. Args: file_path (str): Full path to the Excel file on the machine running the server. sheet_name (str, optional): Sheet to read. Default is the first sheet. Returns: dict: dataset_id, rows, columns and column_names of the new dataset.

Load a JSON file from disk into a new dataset (a list of records or a column-oriented object). Args: file_path (str): Full path to the JSON file on the machine running the server. Returns: dict: dataset_id, rows, columns and column_names of the new dataset.

Load a Parquet file from disk into a new dataset. Args: file_path (str): Full path to the Parquet file on the machine running the server. Returns: dict: dataset_id, rows, columns and column_names of the new dataset.

Train a logistic regression classifier (binary or multiclass). A fast, interpretable baseline. The split must have only numerical, complete features (run encode_categorical / impute_missing first). Args: split_id (str): ID of a split made by train_test_split (stratify=True is advisable for classes). C (float): Inverse regularisation strength, greater than 0. Smaller values regularise more. Default is 1.0. max_iter (int): Maximum solver iterations. Default is 1000. fit_intercept (bool): Learn an intercept. Default is True. class_weight (str, optional): "balanced" re-weights classes inversely to their size (helps with imbalance). Default is None. scale (bool): Standardise the features first (recommended; makes coefficients comparable). Default is True. Returns: dict: model_id (use it in plots such as roc_curve), train/test metrics (accuracy, precision, recall, F1, ROC-AUC), top features and warnings (imbalance, overfitting, no better than the majority class).

Log-transform numerical columns. This is the usual fix for right-skewed columns such as prices, counts, distances and incomes. Columns must be passed explicitly so the target is never transformed by accident. Args: dataset_id (str): ID of a dataset or of a split. columns (list[str]): Numerical columns to transform. method (str): "log1p" (log(1 + x), safe for zeros), "log", "log10" or "log2". Default is "log1p". offset (float): Constant added before the log, e.g. to shift columns that contain 0 or negatives. Default is 0. replace (bool): Overwrite the original columns instead of adding "_". Default is False. Returns: dict: The new columns and the skewness before and after (measured on the training rows).

Report memory consumption of the dataset and its columns. Args: dataset_id (str): ID of the dataset. Returns: dict: col_memory_usage_in_mb (per column, including the index) and dataset_memory_usuage_in_mb (total).

Detect and summarize missing values by column. Args: dataset_id (str): ID of the dataset. Returns: dict: null_values_in_columns (column name -> number of missing values).

Normalize numerical columns into a fixed range (min-max scaling), 0 to 1 by default. Args: dataset_id (str): ID of a dataset or of a split. columns (list[str], optional): Numerical columns to normalize. If omitted, all numerical columns. feature_min (float): Lower bound of the new range. Default is 0. feature_max (float): Upper bound of the new range. Default is 1. Returns: dict: Normalized columns with their fitted minimum and maximum values.

One-hot encode categorical columns: every category becomes its own 0/1 column. Best for nominal columns with few categories (e.g. fuel). Categories that only appear in the test data are encoded as all zeros. Args: dataset_id (str): ID of a dataset or of a split. columns (list[str], optional): Categorical columns to encode. If omitted, all categorical columns. drop_first (bool): Drop the first category of each column to avoid redundancy (useful for linear models). Default is False. Returns: dict: Encoded columns and the new column names.

Generate and save a pair plot (scatter matrix with histograms on the diagonal). Args: dataset_id (str): ID of the dataset. columns (list[str], optional): Numerical columns to include (2 to 6). If omitted, the first 5 numerical columns are used. Returns: dict: Dataset ID, columns used, and saved image filename.

Create polynomial and interaction features (e.g. x^2 and x*y) from numerical columns. The selected columns are replaced by the expanded set, which still contains the originals. Missing values must be imputed first. Args: dataset_id (str): ID of a dataset or of a split. columns (list[str], optional): Numerical columns to expand (max 8). If omitted, all numerical columns. degree (int): Polynomial degree, 2 to 4. Default is 2. interaction_only (bool): Only create interactions between columns (no squares). Default is False. include_bias (bool): Add a constant column of ones named "bias". Default is False. Returns: dict: Columns used and the new feature names.

Generate and save the precision-recall curve of a classification model on its test data. Binary models get one curve; multiclass models get one-vs-rest curves. Args: model_id (str): ID of the trained classification model. Returns: dict: Model ID, average precision per class, and saved image filename.

Generate and save an actual vs predicted scatter plot for a regression model on its test data. Points close to the red line are good predictions. Args: model_id (str): ID of the trained regression model. Returns: dict: Model ID, number of points plotted, and saved image filename.

Train a random forest classifier: many decorrelated trees that vote. A strong default for tabular data. The split must have only numerical, complete features (no scaling is needed). Args: split_id (str): ID of a split made by train_test_split. n_estimators (int): Number of trees. Default is 100. max_depth (int, optional): Maximum tree depth. Default is None (unlimited). min_samples_split (int): Minimum rows needed to split a node. Default is 2. min_samples_leaf (int): Minimum rows in a leaf. Default is 1. max_features (str): Features considered per split: "sqrt", "log2" or "all". Default is "sqrt". class_weight (str, optional): "balanced" or "balanced_subsample" to counter imbalanced classes. Default is None. random_state (int): Seed. Default is 42. Returns: dict: model_id, train/test metrics (accuracy, precision, recall, F1, ROC-AUC), top features and warnings.

Train a random forest regressor: many decorrelated trees averaged together. A strong default for tabular data. The split must have only numerical, complete features (no scaling is needed). Args: split_id (str): ID of a split made by train_test_split. n_estimators (int): Number of trees. Default is 100. max_depth (int, optional): Maximum tree depth. Default is None (unlimited). min_samples_split (int): Minimum rows needed to split a node. Default is 2. min_samples_leaf (int): Minimum rows in a leaf. Default is 1. max_features (float): Fraction of features considered at each split, between 0 and 1. Default is 1.0. random_state (int): Seed. Default is 42. Returns: dict: model_id, train/test metrics (R2, MAE, MSE, RMSE, MAPE), top features and warnings.

Rank models that are already trained (by any training tool or compare_models) using their stored metrics. Nothing is retrained. Models are only comparable when they predict the same target; models trained on different splits are ranked too, but flagged because their test rows differ. Args: model_ids (list[str], optional): Models to rank. Default: every trained model (narrowed by split_id / task). split_id (str, optional): Only rank models trained on this split. task (str): "auto", "regression" or "classification". With "auto" all chosen models must be of one task. sort_by (str, optional): Metric to rank by. Regression: r2 (default), mae, mse, rmse, mape_percent. Classification: f1 (default), accuracy, balanced_accuracy, precision, recall, roc_auc, f1_macro. dataset (str): "test" (default) or "train". Ranking by train rewards overfitting; use it for diagnosis only. top_n (int, optional): Only list the best n models (ranks are still computed over all of them). Returns: dict: ranking (rank, model_id, algorithm, score, train and test metrics, generalisation gap), best_model_id, models skipped (no stored metrics) and notes (ties, mixed splits, overfitting).

Compute regression metrics: R2, MAE, MSE, RMSE and MAPE, plus bias, median and maximum absolute error. Use EITHER a trained model (model_id) OR your own lists of true and predicted values. MAPE is given in percent and skips rows whose true value is 0. Args: model_id (str, optional): ID of a trained regression model. Metrics are recomputed from its stored data. dataset (str): "test", "train" or "both". Only used with model_id. Default is "test". y_true (list[float], optional): True values (use together with y_pred). y_pred (list[float], optional): Predicted values, same length as y_true. Returns: dict: The metrics (one block per dataset when dataset="both").

Remove duplicate rows from a dataset (the first occurrence of each row is kept). Args: dataset_id (str): ID of the dataset. Returns: dict: rows before and after, and how many rows were removed.

Remove rows that have an outlier in any of the checked numerical columns (IQR trimming). Rows with missing values are kept. Columns where IQR is 0 (flags, one-hot columns) are skipped. Args: dataset_id (str): ID of the dataset. columns (list[str], optional): Numerical columns to check. Default is all numerical columns (consider excluding the target column). Returns: dict: rows before and after, rows removed, the bounds used and the skipped columns.

Rename a column in the dataset. Args: dataset_id (str): ID of the dataset. column_name (str): Current column name. new_name (str): New column name. Returns: dict: Dataset ID and old/new column names.

Generate and save residual plots for a regression model on its test data: residuals vs predicted values, and a histogram of the residuals. Args: model_id (str): ID of the trained regression model. Returns: dict: Model ID, mean and std of residuals, and saved image filename.

Train a Ridge (L2-regularised) linear regression. Good when features are correlated. The split must have only numerical, complete features. Args: split_id (str): ID of a split made by train_test_split. alpha (float): Regularisation strength, 0 or more. Larger values shrink coefficients more. Default is 1.0. fit_intercept (bool): Learn an intercept. Default is True. scale (bool): Standardise the features first. Strongly recommended for regularised models. Default is True. Returns: dict: model_id, train/test metrics (R2, MAE, MSE, RMSE, MAPE), top features and warnings.

Generate and save the ROC curve of a classification model on its test data. Binary models get one curve; multiclass models get one-vs-rest curves. Args: model_id (str): ID of the trained classification model. Returns: dict: Model ID, AUC score per class, and saved image filename.

Scale numerical columns. Recommended: use a split_id so the scaler is fit on training data only. If a dataset id is used, pass columns explicitly so the target column is not scaled. Args: dataset_id (str): ID of a dataset or of a split. columns (list[str], optional): Numerical columns to scale. If omitted, all numerical columns. method (str): "standard" (mean 0, std 1), "minmax" (0 to 1), "robust" (median/IQR, good with outliers) or "maxabs" (divide by max absolute value). Returns: dict: Scaled columns and the fitted scaler parameters.

Generate and save a scatter plot to visualize the relationship between two numerical columns. Args: dataset_id (str): ID of the dataset. col_name1 (str): Column for the X-axis. col_name2 (str): Column for the Y-axis. Returns: dict: Dataset ID, column names, and saved image filename.

Model-based feature selection: fit a model on the training rows, rank the numerical features by importance and drop the weak ones. (feature_selection in preprocessing uses simple statistical filters instead.) Only numerical features are scored; other columns are kept untouched. Missing values must be imputed first. Args: dataset_id (str): ID of a dataset or of a split. With a split the target is known already. target_column (str, optional): Required when dataset_id is a dataset. method (str): "random_forest" (importance from a random forest) or "lasso" (L1 regularisation, regression only). Default is "random_forest". k (int, optional): Keep the k most important features. threshold (float, optional): Keep features whose normalised importance (importances sum to 1) is at least this value. If neither k nor threshold is given: random_forest keeps features above the average importance, lasso keeps non-zero ones. apply (bool): Drop the unselected features. If False, only report the ranking. Default is True. random_state (int): Seed. Default is 42. Returns: dict: Ranked importances, the selected and dropped features, and whether the data was changed.

Separate a dataset into a features dataset (X) and a target dataset (y). Both are registered as new datasets; the original dataset is not changed. Args: dataset_id (str): ID of the dataset. target_column (str): Name of the target column. Returns: dict: IDs of the new features and target datasets and their columns.

Standardize numerical columns to mean 0 and standard deviation 1 (z-score). Good for linear models, SVM, KNN and anything distance or gradient based. Args: dataset_id (str): ID of a dataset or of a split. columns (list[str], optional): Numerical columns to standardize. If omitted, all numerical columns. Returns: dict: Standardized columns with their fitted means and standard deviations.

Inspect a dataset (or the training part of a split) and propose feature-engineering steps. Nothing is changed. Every suggestion names the tool to call and ready-to-use arguments. Checks: constant and ID-like columns, date columns, skewed columns, outlier-heavy columns, categorical columns, and (when the target is numeric or binary) non-linear and interaction effects. Args: dataset_id (str): ID of a dataset or of a split. target_column (str, optional): Needed for target-aware suggestions when dataset_id is a dataset (a split already knows it). max_suggestions (int): Maximum number of suggestions returned. Default is 15. Returns: dict: Suggestions sorted by priority (tool, arguments, reason) plus general notes.

Train a support vector machine classifier. Powerful on small to medium data, slow on large data (training time grows quickly with the number of rows). Needs scaled features. The split must have only numerical, complete features. Args: split_id (str): ID of a split made by train_test_split. C (float): Regularisation: larger values fit the training data more tightly. Default is 1.0. kernel (str): "rbf", "linear", "poly" or "sigmoid". Default is "rbf"; "linear" is much faster on big data. gamma (str): Kernel width for rbf/poly/sigmoid: "scale" or "auto". Default is "scale". degree (int): Polynomial degree, only used by the "poly" kernel. Default is 3. probability (bool): Also learn class probabilities (needed for ROC-AUC on multiclass, precision-recall and roc_curve plots). It makes training several times slower. Default is True. class_weight (str, optional): "balanced" to counter imbalanced classes. Default is None. random_state (int): Seed (used for the probability calibration). Default is 42. scale (bool): Standardise the features first (strongly recommended). Default is True. Returns: dict: model_id, train/test metrics (accuracy, precision, recall, F1, ROC-AUC) and warnings.

Split a dataset into training and test sets and register them as a split. Do this BEFORE encoding, scaling or imputing, then pass the returned split_id to those tools so they are fit on the training data only (no data leakage). The original dataset is not changed. Args: dataset_id (str): ID of the dataset. target_column (str): Name of the target column. test_size (float): Fraction of rows used for testing (0 to 1). Default is 0.2. random_state (int): Seed for a reproducible split. Default is 42. stratify (bool): Keep the target class proportions in both sets (classification). Default is False. shuffle (bool): Shuffle the rows before splitting. Default is True. Returns: dict: split_id, row counts, feature columns and the inferred task.

Apply a mathematical transformation to one numerical column, usually to reduce skewness. Args: dataset_id (str): ID of a dataset or of a split. column (str): Numerical column to transform. method (str): One of "sqrt" (needs values >= 0), "square", "cube", "reciprocal" (no zeros), "abs", "cbrt" (cube root, works with negatives), "signed_log1p" (log that keeps the sign, works with negatives), "yeo_johnson" (fitted power transform, works with negatives), "boxcox" (fitted, needs values > 0). Fitted methods learn their parameters from the training rows only. new_column (str, optional): Name of the new column. Default is "_". replace (bool): Overwrite the original column instead of adding a new one. Default is False. Returns: dict: The new column and the skewness before and after (measured on the training rows).

Undo the last change made to a dataset by a cleaning or preprocessing tool (the last 5 changes are remembered). Args: dataset_id (str): ID of the dataset. Returns: dict: the restored shape and how many further undo steps are available.

Return unique values for a selected column. Args: dataset_id (str): ID of the dataset. col_name (str): Name of the column. Returns: dict: unique_values, column_name and dataset_id.

Calculate frequency counts for the values of a column, or of every categorical column. Args: dataset_id (str): ID of the dataset. column (str, optional): Column to count. If omitted, every categorical (text) column is counted. top_n (int, optional): Only return the n most frequent values per column. normalize (bool): Return proportions instead of counts. Default is False. Returns: dict: category_columns_unique_values_and_counts, a list with one {column: {value: count}} entry per column.

Generate and save violin plots showing the distribution of numerical data. Args: dataset_id (str): ID of the dataset. column_name (str, optional): Numerical column to visualize. If omitted, generates violin plots for all numerical columns. group_by (str, optional): Column to split column_name by (top 10 groups). Requires column_name. Returns: dict: Dataset ID, columns/groups plotted, and saved image filename.

Train an XGBoost classifier: fast, regularised gradient boosting. Requires the optional xgboost package. Any class labels work (text labels are encoded internally). The split must have only numerical, complete features (no scaling is needed). Args: split_id (str): ID of a split made by train_test_split. n_estimators (int): Number of boosting rounds. Default is 100. learning_rate (float): Shrinkage per round, greater than 0. Default is 0.1. max_depth (int): Depth of each tree. Default is 6. subsample (float): Fraction of rows used per tree, in (0, 1]. Default is 1.0. colsample_bytree (float): Fraction of features used per tree, in (0, 1]. Default is 1.0. random_state (int): Seed. Default is 42. Returns: dict: model_id, train/test metrics (accuracy, precision, recall, F1, ROC-AUC), top features and warnings.

Train an XGBoost regressor: fast, regularised gradient boosting. Requires the optional xgboost package. The split must have only numerical, complete features (no scaling is needed). Args: split_id (str): ID of a split made by train_test_split. n_estimators (int): Number of boosting rounds. Default is 100. learning_rate (float): Shrinkage per round, greater than 0. Default is 0.1. max_depth (int): Depth of each tree. Default is 6. subsample (float): Fraction of rows used per tree, in (0, 1]. Default is 1.0. colsample_bytree (float): Fraction of features used per tree, in (0, 1]. Default is 1.0. random_state (int): Seed. Default is 42. Returns: dict: model_id, train/test metrics (R2, MAE, MSE, RMSE, MAPE), top features and warnings.