AxModelFitter_cross_validate_model
Cross-validate your model to test generalization using kfold, shuffle, or custom splits. Check R² consistency across folds to detect overfitting.
Instructions
Test how well your model generalizes to new data using cross-validation.
REQUIRED INPUTS (same as fit_model):
- All model parameters: function_source, parameters, bounds, etc.
- data_file: Path to your data file
- input_data: Maps file columns to input variables
- output_data: Maps file columns to output variables
VALIDATION TYPES:
- 'kfold': Split data into equal parts (good default)
- 'shuffle': Random train/test splits
- 'custom': Specify your own train/test indices
TYPICAL USAGE:
1. Use same parameters as your fit_model call
2. Set validation_strategy='kfold' and n_splits=5
3. Check if test R² values are consistent across folds
INTERPRETATION:
- Consistent high R² across folds: Good generalization
- Large R² variation: Model may be overfitting
- Low average R²: Model not capturing data patterns well
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| bounds | Yes | Parameter/input/output bounds | |
| max_time | No | Maximum optimization time in seconds per fold | |
| n_splits | No | Number of validation folds (for kfold and shuffle) | |
| constants | No | Fixed constants | |
| data_file | Yes | Path to data file (CSV, Excel, JSON, Parquet). All data must be provided via file. | |
| test_size | No | Test set proportion (for shuffle split) | |
| input_data | Yes | Input column mappings: [{'column': 'time', 'name': 't', 'unit': 'second'}, {'column': 'x_col', 'name': 'x', 'unit': 'meter'}] | |
| model_name | Yes | Model name for identification | |
| file_format | No | File format: 'csv', 'excel', 'json', 'parquet' (auto-detect if None) | |
| jit_compile | No | Enable JIT compilation | |
| output_data | Yes | Output column mapping: {'columns': ['signal'], 'name': 'y', 'unit': 'volt'} OR {'columns': ['y1', 'y2'], 'name': 'y', 'unit': 'volt'} | |
| random_state | No | Random seed for reproducibility | |
| custom_splits | No | Custom train/test splits: [{'train': [0,1,2], 'test': [3,4]}, ...] | |
| function_name | Yes | Function name that computes the model output | |
| optimizer_type | No | Optimizer: 'nlopt' (best default), 'scipy' (simple), 'nevergrad' (gradient-free) | nlopt |
| function_source | Yes | JAX function source code using jnp operations | |
| optimizer_config | No | Optimizer config: {'use_gradient': True, 'tol': 1e-6, 'max_function_eval': 1000000} | |
| cost_function_type | No | Cost function: 'mse', 'mae', 'huber', 'relative_mse' | mse |
| initial_parameters | Yes | Initial parameter guesses for optimization on each fold | |
| validation_strategy | No | Validation type: 'kfold', 'shuffle', or 'custom' | kfold |