evaluate_trace
Run an independent reliability check on an agent's multi-step task trace to get a 0-100 score, detected issues, and concrete fixes for the next attempt.
Instructions
Get an independent, deterministic reliability check of a multi-step task you (an agent) just ran. Returns a 0-100 score, the issues found, and concrete fixes to apply on your next attempt.
WHEN TO CALL: after you finish (or give up on) a multi-step task that used tools, before you report "done" to the user, or after a failure you want to learn from. Free, no API key, fast and offline (keyword heuristics, no LLM). Call get_trace_schema first if unsure of the format.
INPUT (all plain strings):
goal: the original task, stated specifically (what success looks like). Vague goals under 20 chars are penalised.
trace: your chronological steps, one per line, e.g. "1. Called search_flights(AMS->LIS) -> returned 3 results" / "2. Observed: ..." . Write what each tool RETURNED ("observed", "returned", "result:"), any error and the retry/fallback you used, and a final "Verified: ..." line.
outcome: the concrete final result you claim, and how it meets the goal.
agent_name: optional identifier for yourself (logging only).
RETURNS JSON: score (int 0-100), dimensions {goal_clarity, observability, error_recovery, goal_alignment}, issues (list), suggestions (list, max 5), summary, version.
THE LOOP: run task -> evaluate_trace -> apply the suggestions to your process -> re-run -> evaluate_trace again and compare scores/dimensions. Scores below ~60 usually mean missing observations, unrecovered errors, or no verification step.
Tool call template
Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.
{
"id": 1,
"jsonrpc": "2.0",
"method": "tools/call",
"params": {
"arguments": {
"goal": "<goal>",
"trace": "<trace>",
"outcome": "<outcome>"
},
"name": "evaluate_trace"
}
}Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| goal | Yes | ||
| trace | Yes | ||
| outcome | Yes | ||
| agent_name | No | unknown |