Skip to main content
Glama

Run a saved judge

jev_saved_judge

Evaluate new input against a judge saved in your Jev AI account, returning model answers and usage. Optionally specify a revision to detect changed rules with HTTP 409.

Instructions

Evaluate new input with a judge saved in your Jev AI account. Does not change the judge. Optional revision detects changed rules with HTTP 409. Use jev_decide for inline rules. Requires JEV_AI_API_KEY from https://jev-ai.pro/jev-api. Sends input to Jev AI and consumes account credits; rate limits apply. Returns model, answers and usage. No automatic retries.

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
stateYesText, JSON object or array to evaluate. Sent to Jev AI and its model providers.
judgeIdYesID of a saved judge owned by the API-key account.
revisionNoExpected judge revision; omit to use current rules.

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observedv1.0.1

TDQS

A4.9/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Discloses key behaviors beyond annotations: does not change the judge, sends input to Jev AI, consumes account credits, rate limits apply, returns model/answers/usage, and has no automatic retries. It even explains the HTTP 409 revision conflict behavior. No contradiction with the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the core purpose, and every sentence carries distinct information: purpose, non-mutation, revision semantics, sibling routing, auth, side effects, rate limits, return value, and retry policy. Dense but not padded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with no output schema and 3 parameters, this description is complete: it covers auth requirements, side effects, return values, conflict behavior, retry policy, and a sibling alternative. Nothing essential for invoking it correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description adds meaningful context beyond the schema, particularly for the revision parameter, explaining it detects changed rules with HTTP 409, and clarifying that state is sent to Jev AI model providers. This elevates the score.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific action: 'Evaluate new input with a judge saved in your Jev AI account.' It also explicitly names the sibling alternative (jev_decide) for inline rules, making the tool's role clear. The title and body are consistent.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives clear when-to-use context: when using a saved judge, and directs inline-rule use cases to jev_decide. It also covers prerequisite setup (JEV_AI_API_KEY), revision behavior, and operational constraints like rate limits and credit consumption.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.