get_experiment_results
Retrieves an experiment's results to compare candidate and baseline metrics and test cases, revealing improvements or regressions.
Instructions
Reads an Experiment's results: each Metric, candidate against baseline, then each test case.
Find an Experiment with list_experiments. metrics has one entry per Metric of the
Metric Group, in its order: each side's value (or labels for a Label), delta and
delta_pct of the candidate over the baseline, better (which way a change is an
improvement, null when undeclared), change (improved, regressed, unchanged or unknown),
and how many Sessions errored, meaning the Evaluator's own code broke. A test case's value
is the mean over its Sessions, and a Metric is averaged only over the paired_test_cases
both sides have a value for, so check that count before trusting a delta. Say a Metric
improved or regressed only from change; unknown means no judgement, so never infer a
direction from the Metric's name. pipeline_metrics compares DURATION, TOKENS and COST the
same way. Each test case lists the Sessions each side scored and, per Score Metric keyed
{evaluator_id}:{metric_key}, each side's mean; that is where to look for the test cases
behind a regression. Resolve evaluator ids to names with list_evaluators. Results are
partial while status is REPLAYING or SCORING. headline gives each side's Composite
(0 to 1, 1 best) and normalized_means. When comparable is false the two sides used
different formulas: report no change between them and give incomparable_reason; each side's
composite_weights shows the formula it used. aggregates gives each Metric over every Session
a side scored, not just the paired test cases: each side's whole-run entry (a Score's mean, a
Label's shares), delta, share_deltas for a Label and change. Empty until a side's
scoring runs end.
:param experiment_id: The Experiment's id.
:param max_test_cases: Most test cases to return; metrics still cover every test case.
:returns: The results, or an error message.
The output is automatically stored and can be referenced in other functions.
Returns a formatted preview with an object ID (e.g., @obj_123).
Use the object store tools in combination with the object ID to view nested properties of the object.
Use the returned object ID to pass this result to other functions.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| experiment_id | Yes | ||
| max_test_cases | No |