cot_analyzer
Analyze a Chain-of-Thought (CoT) or reasoning trace from an LLM. Detects step count, logical flow, conclusion presence, backtracking, and estimates reasoning depth. Useful for o1/o3/DeepSeek-R1 evaluation.
Example tool call
This call uses example arguments supplied in the tool’s input schema.
{
"id": 1,
"jsonrpc": "2.0",
"method": "tools/call",
"params": {
"arguments": {
"reasoning": "Step 1: 12 apples minus 5 eaten.\nStep 2: 12 - 5 = 7.\nTherefore, the answer is 7.",
"expected_conclusion": "7"
},
"name": "cot_analyzer"
}
}Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| reasoning | Yes | The CoT / reasoning trace text (e.g. from <think> tags or step-by-step output) | |
| expected_conclusion | No | Expected final answer to check against (optional) |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
| markers | No | ||
| step_count | No | ||
| total_chars | No | ||
| total_lines | No | ||
| has_conclusion | No | ||
| reasoning_depth | No | ||
| backtracking_signals | No | ||
| reasoning_depth_label | No | ||
| conclusion_matches_expected | No |