Skip to main content
Glama

Read the flow trace behind one scored item

caliper_evals_runs_item_execution
Read-only

What the flow actually DID on one run item: every step in order (what it said, which tools it called with what arguments, what came back) and the final answer, plus status, duration, and cost. Read this when a low score needs explaining beyond the judge's reasoning — a wrong tool call or an empty tool result is usually the cause, and the fix is different from a prompt fix. Pass the run item's id (NOT itemId) from caliper_evals_runs_get. Items whose outputs were supplied from outside (external evals) have no trace and return not_found. Steps are capped for transport; the run page in Caliper has the full record.

Input Schema

NameRequiredDescriptionDefault
runIdYesRun id.
evalIdYesEval the run belongs to.
runItemIdYesThe run item's `id` from caliper_evals_runs_get (not its dataset itemId).
workspaceNoWorkspace slug. Personal tokens with no default workspace MUST pass this; tokens with a default can override per call. Ignored for workspace API keys.

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observed

TDQS

A4.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover the read-only safety profile, but the description adds genuinely non-obvious behavior: external-eval items have no trace and return not_found, and steps are truncated for transport while the Caliper run page holds the full record. These caveats go beyond what readOnlyHint/destructiveHint/openWorldHint convey.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with what the tool returns, then the when-to-use trigger, then the parameter pitfall, then edge-case caveats. No filler sentences; each one carries distinct information an agent needs.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description carries the full burden of describing the return shape (ordered steps, tool calls and arguments, tool results, final answer, status, duration, cost) and failure modes (not_found for external items, truncated steps). Nothing needed to call or interpret it correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description adds real value by stressing that `runItemId` must be the run item's `id` and NOT its `itemId`, a classic source of wrong-argument failures. The workspace parameter's default/override semantics are left to the schema, but the critical gotcha is surfaced.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource — reads the execution trace (every step, tool call, arguments, tool result, final answer, plus status/duration/cost) for one run item. It is clearly distinguishable from caliper_evals_runs_get, which is cited as the source of the id rather than a competitor for this data.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives an explicit trigger — 'read this when a low score needs explaining beyond the judge's reasoning' — with the diagnostic rationale (wrong tool call or empty tool result). It also tells the agent where to obtain the required id (from caliper_evals_runs_get), leaving nothing to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources