Skip to main content
Glama

evaluate_decision

Evaluate context, diffs, logs, or planning options and receive calibrated probabilities for binary, categorical, or scoring questions—before committing to lengthy plans or edits.

Instructions

Zero-token-output decision engine powered by TypeSafe Jev. Evaluates context, code diffs, logs, or planning options and returns calibrated probabilities. Use this BEFORE generating long reasoning plans or multi-file edits to pick optimal paths.

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
typeYesDecision format: 'noul' (binary yes/no probability), 'choice' (categorical distribution), or 'score' (ordered rubric).
stateYesThe context, code diff, error log, or task description to evaluate.
apiKeyNoOptional OpenRouter API key. If omitted, the OPENROUTER_API_KEY environment variable will be used.
criteriaYesCriteria map: for 'noul' provide { true: '...', false: '...' }, for 'choice' provide { option1: '...', option2: '...' }, for 'score' provide an array of strings in order.
questionYesThe targeted question to evaluate (e.g., 'Will this change cause a breaking API error?').

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observedv1.0.3

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full disclosure burden. It does disclose notable traits: 'Zero-token-output' (no token generation), 'returns calibrated probabilities' (output behavior), and the 'TypeSafe Jev' engine dependency. However, it does not mention the external OpenRouter API call, API key requirements (beyond the schema), or latency/cost implications — material operational information for an agent deciding whether to invoke the tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, each earning its place: identity (zero-token-output engine), capability (evaluates and returns probabilities), and usage timing (use BEFORE planning or edits). The most distinctive trait is front-loaded. There is slight redundancy between 'zero-token-output decision engine' and 'returns calibrated probabilities', but no real wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema and no annotations, so the description must cover return behavior and operational traits itself; it states probabilities as the general output but does not specify per-type return shapes for 'noul' vs 'choice' vs 'score'. The external OpenRouter dependency and cost/latency implications are left to schema inference. For a 5-param tool with fully documented inputs, the description is adequate but leaves output-format and operational details unstated.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3 and the schema already documents all five parameters, including type-specific criteria formats for 'noul', 'choice', and 'score'. The description marginally reinforces the 'state' parameter by listing the same input kinds (context, code diffs, logs, planning options) but adds no syntax, format, or edge-case detail beyond what the schema provides.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses specific verbs and resources: it 'evaluates context, code diffs, logs, or planning options' and 'returns calibrated probabilities,' clearly identifying the tool as a decision evaluator with a defined input scope and output shape. Minor deduction for jargon like 'Zero-token-output' and 'TypeSafe Jev,' which add character but require interpretation. With no sibling tools to differentiate against, the verb+resource specification carries the clarity burden and succeeds.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The final sentence gives explicit timing guidance: 'Use this BEFORE generating long reasoning plans or multi-file edits to pick optimal paths.' This is concrete contextual direction for when the tool adds value relative to the agent's workflow. No exclusions or when-not-to-use guidance is given, though no sibling tools exist, so the guidance is as complete as the context allows.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Deploy Server

Other Tools