alibi

diagnose_trace

Pinpoint why a long AI-agent run failed by scanning the trace and returning the 3 steps most likely responsible.

Instructions

Find where a long AI-agent run went wrong. Reads the trace chapter by chapter (never all at once), raises a CUSUM alarm, looks back, and returns the 3 steps to read first. trace: path to a .json trace or a Claude Code session .jsonl, or a LangSmith trace id. A trace under the length gate (default 50K tokens) or over the cost ceiling (default 250K tokens) is returned with gated true and a message saying which: it is not analysed and nothing is spent.

Input Schema

TableJSON Schema

Name	Required	Default
`trace`	Yes
`source`	No	auto
`project`	No

Output Schema

TableJSON Schema

Name	Required	Description
`gated`	Yes	True when the trace was not analysed and nothing was spent: it is empty, under the length gate, or over the cost ceiling. `message` says which.
`anchor`	No
`message`	Yes
`n_steps`	Yes
`cost_usd`	No	Billed for this trace in USD; null if the judge logs none.
`suspects`	No
`n_chapters`	No
`judge_calls`	No	Judge calls made for this trace; null if the judge logs none.
`trace_tokens`	Yes
`alarm_chapter`	No
`judge_seconds`	No

Schema Changelog

Changes observed during successful MCP inspections.

Changed9 schema fields changedv0.1.4Sep 23, 2026

addedInput schema / properties / source / enum

Added value: +[
+  "auto",
+  "json",
+  "claude-code",
+  "langsmith"
+]

addedOutput schema / properties / cost_usd / anyOf

Added value: +[
+  {
+    "type": "number"
+  },
+  {
+    "type": "null"
+  }
+]

addedOutput schema / properties / cost_usd / description

Added value: +"Billed for this trace in USD; null if the judge logs none."

removedOutput schema / properties / cost_usd / type
```
Removed value: -"number"
```

addedOutput schema / properties / judge_calls / anyOf

Added value: +[
+  {
+    "type": "integer"
+  },
+  {
+    "type": "null"
+  }
+]

addedOutput schema / properties / judge_calls / description

Added value: +"Judge calls made for this trace; null if the judge logs none."

removedOutput schema / properties / judge_calls / type
```
Removed value: -"integer"
```

addedOutput schema / properties / judge_seconds / anyOf

Added value: +[
+  {
+    "type": "number"
+  },
+  {
+    "type": "null"
+  }
+]

removedOutput schema / properties / judge_seconds / type
```
Removed value: -"number"
```

First observedv0.1.3Sep 20, 2026

TDQS

A4.3/5.0

Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure, and it delivers. It reveals that the tool reads incrementally ('chapter by chapter, never all at once'), uses a CUSUM alarm, performs a lookback, returns exactly 3 steps, and gates traces by token length/cost with a 'gated' flag and no spend. This is unusually rich behavioral context for a description.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is dense but every sentence earns its place: purpose, algorithm behavior, parameter explanation, and gating rules are all covered without redundancy. It is slightly long, but the complexity of the tool justifies the length. Front-loading the purpose sentence is ideal.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Considering the tool's complexity and the absence of annotations, the description is remarkably complete: it explains the algorithm, the gating thresholds, cost behavior, and expected output concept. An output schema exists, so return-value details need not be spelled out. Missing only minor clarifications around optional parameter semantics and normal (non-gated) cost behavior.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It fully explains the required `trace` parameter, including accepted formats (.json, Claude Code .jsonl, LangSmith trace id). However, it provides no semantic guidance for `source` or `project`; the enum on `source` is somewhat self-explanatory given the trace formats, but `project` remains opaque. Partial compensation warrants a middle score.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description starts with a specific, outcome-oriented verb and resource: 'Find where a long AI-agent run went wrong.' It goes beyond a mere label by explaining the analysis approach (chapter-by-chapter reading, CUSUM alarm, lookback) and the concrete deliverable (the 3 steps to read first), leaving no ambiguity about what the tool does.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description clearly frames when the tool is appropriate: for diagnosing long AI-agent runs that went wrong. It also tells the agent about gating behavior for traces that are too short or too costly, which effectively describes conditions under which the tool will not perform analysis. No sibling tools exist to contrast against, so exclusion guidance is unnecessary; the context is clear enough for a 4.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Deploy Server

Other Tools

diagnose_traceA