AxModelFitter_compare_models
Compare multiple fitted models on the same data and rank them using AIC/BIC to identify the statistically best model.
Instructions
Compare multiple models to find the best one using statistical criteria.
USE CASE: You have several competing models (linear, exponential, polynomial) fitted to the same data.
This tool tells you which model is statistically best.
REQUIRED INPUTS:
- models: List of your fitted models with their loss values and parameter counts
- data_file: Path to your data file (same data used for all models)
- output_data: Which columns contain your output data
- sigma: Noise level (required for MSE models, None for MAE models)
EXAMPLE MODELS INPUT:
[
{"name": "Linear", "loss_value": 0.05, "cost_function_type": "mse", "n_parameters": 2},
{"name": "Exponential", "loss_value": 0.02, "cost_function_type": "mse", "n_parameters": 3}
]
RETURNS: Ranked models with statistical evidence for which is best.
Lower AIC/BIC = better model. Akaike weights show relative model support.
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| n_obs | No | Explicit count of independent residuals for ALL models. If None, infers from output data | |
| sigma | No | REQUIRED noise std dev for diagonal covariance Σ=σ²I applied to ALL models. For mse: provide from domain knowledge. For mae: use None. | |
| models | Yes | List of model dicts: [{'name': 'Model1', 'loss_value': 0.01, 'cost_function_type': 'mse', 'n_parameters': 3}, ...] | |
| data_file | Yes | Path to data file (CSV, Excel, JSON, Parquet). All data must be provided via file. | |
| file_format | No | File format: 'csv', 'excel', 'json', 'parquet' (auto-detect if None) | |
| output_data | Yes | Output column mapping: {'columns': ['y'], 'name': 'y', 'unit': 'volt'} or {'columns': ['y1', 'y2'], 'name': 'y', 'unit': 'volt'} | |
| df_effective | No | Effective degrees of freedom for penalized models (EXCLUDING scale) - applied to ALL models | |
| n_scale_params | No | Number of scale parameters: 1 for single-output, d for d-output with separate scales | |
| aicc_include_scale | No | Include scale parameter in AICc correction (literature varies) | |
| include_scale_param | No | Include scale parameter (σ² or b) in k count |