Skip to main content
Glama
Ahesui
by Ahesui

Evaluate with Jev

jev_evaluate
Read-onlyIdempotent

Evaluate shared text or JSON by sending named noul, choice, and score questions in parallel. Returns typed answers, probabilities, confidence, and an auto, review, or escalate action.

Instructions

Escape hatch: send shared state plus named noul/choice/score questions to TypeSafe Jev. Use when no other jev_* recipe fits. Jev does not write code or prose. Questions in one call run in parallel. Returns typed answers, probabilities, confidence, usage, and action auto|review|escalate.

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
modelNoOverride, default typesafe-ai/jev
stateYesShared state to judge: text or JSON
questionsYesNamed noul, choice, and score questions evaluated in parallel

Output Schema

TableJSON Schema
NameRequiredDescriptionDefault
modelYes
usageYes
actionYes
answersYes
coverageYes
truncatedYes

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observedv0.1.0

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover read-only, open-world, idempotent, and non-destructive behavior. The description adds useful behavioral context beyond that: parallel execution of questions, the no-code/no-prose constraint, and the concrete return shape including action auto|review|escalate. No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two compact sentences front-load the purpose and usage guidance. Every clause earns its place: escape-hatch role, selection criterion, capability boundary, execution model, and return summary. No redundant or filler language.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the rich input schema, output schema, and annotations, the description is complete enough for an agent to decide and invoke correctly. It covers when to use the tool, what it cannot do, how questions behave, and what the caller gets back. No critical decision-making information is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents state, questions, and model. The description adds the notion of 'named' questions and parallel execution, but does not materially expand on parameter semantics beyond what the schema provides. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific action—send shared state plus named noul/choice/score questions to TypeSafe Jev—and clearly frames it as an escape hatch. This distinguishes it from sibling jev_* tools by making clear it is the generic fallback when no more specific recipe applies.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides explicit selection guidance: 'Use when no other jev_* recipe fits.' It also sets an exclusion boundary by stating Jev does not write code or prose, which helps the agent avoid misusing this tool for generation tasks better suited to other tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.