Skip to main content
Glama

get_experiment_results

Retrieves an experiment's results to compare candidate and baseline metrics and test cases, revealing improvements or regressions.

Instructions

Reads an Experiment's results: each Metric, candidate against baseline, then each test case.

Find an Experiment with list_experiments. metrics has one entry per Metric of the Metric Group, in its order: each side's value (or labels for a Label), delta and delta_pct of the candidate over the baseline, better (which way a change is an improvement, null when undeclared), change (improved, regressed, unchanged or unknown), and how many Sessions errored, meaning the Evaluator's own code broke. A test case's value is the mean over its Sessions, and a Metric is averaged only over the paired_test_cases both sides have a value for, so check that count before trusting a delta. Say a Metric improved or regressed only from change; unknown means no judgement, so never infer a direction from the Metric's name. pipeline_metrics compares DURATION, TOKENS and COST the same way. Each test case lists the Sessions each side scored and, per Score Metric keyed {evaluator_id}:{metric_key}, each side's mean; that is where to look for the test cases behind a regression. Resolve evaluator ids to names with list_evaluators. Results are partial while status is REPLAYING or SCORING. headline gives each side's Composite (0 to 1, 1 best) and normalized_means. When comparable is false the two sides used different formulas: report no change between them and give incomparable_reason; each side's composite_weights shows the formula it used. aggregates gives each Metric over every Session a side scored, not just the paired test cases: each side's whole-run entry (a Score's mean, a Label's shares), delta, share_deltas for a Label and change. Empty until a side's scoring runs end. :param experiment_id: The Experiment's id. :param max_test_cases: Most test cases to return; metrics still cover every test case. :returns: The results, or an error message.

The output is automatically stored and can be referenced in other functions. Returns a formatted preview with an object ID (e.g., @obj_123). Use the object store tools in combination with the object ID to view nested properties of the object. Use the returned object ID to pass this result to other functions.

Input Schema

NameRequiredDescriptionDefault
experiment_idYes
max_test_casesNo

Schema Changelog

Changes observed during successful MCP inspections.

  1. Addedv0.1.33

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations at all, the description carries the full burden and does so: results are partial while ``status`` is REPLAYING or SCORING, ``aggregates`` is empty until scoring runs end, ``errored`` counts mean the Evaluator's own code broke, ``unknown`` means no judgement, and comparability failures must be reported as no-change. It also discloses the object-store side effect (auto-stored output, preview with an ``@obj_123`` id), which is behavior an agent must know before calling.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The first sentence is front-loaded with the core purpose and the dense middle is largely necessary return-value documentation given there is no output schema. Some density could be trimmed (the ``pipeline_metrics`` aside, the repeated 'each side's' phrasing) and the trailing object-store boilerplate does not earn much space, but overall the content justifies its length.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema and only 0% parameter coverage, the description compensates by documenting the return structure field by field: ``metrics``, ``delta``/``delta_pct``, ``change``, ``paired_test_cases``, ``pipeline_metrics``, per-test-case Session breakdowns, ``headline``/``normalized_means``, ``comparable`` and ``aggregates``. An agent can both call the tool and interpret its output without further documentation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, so the description must compensate. ``max_test_cases`` is explained with a crucial semantic detail — '``metrics`` still cover every test case' — which prevents the agent from assuming metrics are truncated too. ``experiment_id`` is only restated ('The Experiment's id'), but that parameter is self-evident, so the gap is minor.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The opening sentence gives a specific verb and resource plus the shape of the payload: 'Reads an Experiment's results: each Metric, candidate against baseline, then each test case.' It is clearly a results-retrieval tool, distinguishable from metadata tools like get_experiment or run-management tools like list_experiment_runs.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It routes the agent explicitly: find the experiment with ``list_experiments`` and resolve evaluator ids with ``list_evaluators``. It also gives conditional usage rules ('Say a Metric improved or regressed only from ``change``', 'When ``comparable`` is false ... report no change'). It stops short of naming sibling result tools such as get_experiment_run or get_evaluation_run as alternatives, so it is strong but not fully disambiguating.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Deploy Server

Other Tools