jev-mcp

Overview Schema Source Related Servers Discussions Admin

Judge many items with one question set

jev_batch

Read-only

Ask one question set across many items such as diff hunks or files in a single call. Returns ranked verdicts, probabilities, and flagged failures for batch review.

Instructions

N items x ONE question set: one request per item (failing tests, diff hunks, files), concurrent with a shared rate gate. Returns rows {item, verdict, p, decision} sorted by p as top-k or only the flagged ones, plus a summary and every failure. items_from builds items on the server: "$diff:staged" (one per hunk) or ["$file:a.ts", ...]. Diff review: preset review_filter with narrow Nouls where true = bad. Not for several questions about one state (jev_ask). If it returns unavailable / rate_limited / budget_exceeded, judge inline; do not retry in a loop. Example: {items_from:"$diff:staged", preset:"review_filter", questions:{no_err:{type:"noul",instructions:"Does diff remove error handling?"}}}

Input Schema

Name	Required	Description
`cache`	No	Use the response cache (default true).
`items`	No	Your items (one request each).
`top_k`	No
`detail`	No	Add per-question answers to every row (more tokens).
`preset`	No	Decision preset: gate (default) \| triage \| review_filter \| strict. Thresholds live in config.
`return`	No	ranked = top_k by p; flagged = only flagged / non-auto items; all. Default: flagged for review_filter, else ranked when rank_by, else all.
`rank_by`	No	Question name to sort by (p descending).
`only_ids`	No	Re-run only these item ids (reruns also auto-label earlier decisions).
`questions`	Yes	One question set asked of every item.
`items_from`	No	Items built on the server: "$diff" / "$diff:staged" / "$diff:<ref>" = one item per hunk {file,hunk,diff}; ["$file:a.ts", ...] = one item per file {path,text}.
`concurrency`	No
`output_path`	No	Also write the returned rows as JSONL here (inside the workspace).
`long_context`	No	Long-context header (262k tokens, slower, less accurate). "auto" uses it only when over the 64k/32k limits.
`stop_on_first_error`	No
`max_total_input_tokens`	No

Output Schema

Name	Required	Description
`error`	No	Only on a failed call (isError): kind, message, fix, retryable, plus cause / loc / status / request_id when known.
`items`	No
`model`	No
`summary`	No
`failures`	No
`warnings`	No
`output_path`	No
`model_overridden`	No

Schema Changelog

Changes observed during successful MCP inspections.

First observedv0.2.0Oct 23, 2026

TDQS

A4.7/5.0

Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond readOnlyHint/openWorldHint, it discloses execution traits an agent cannot infer: concurrent requests behind a shared rate gate, result ordering by p, summary plus every failure, and named failure modes (unavailable / rate_limited / budget_exceeded) with an explicit no-retry policy. Return rows are described even though an output schema exists.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the core contract and tightly packed, with a closing worked example that earns its space. Some phrasing is telegraphic to the point of obscurity ("narrow Nouls where true = bad"), which costs a point on clarity.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given 15 params, nested question objects, and an output schema, the description covers the calling contract, the distinguishing alternative, error handling, and an example. Residual gaps are minor: where preset thresholds come from and what concurrency/cache defaults imply in practice.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 15 params at 73% schema coverage, the description adds real meaning for the tricky ones: items_from forms, return top-k vs flagged semantics, rank_by sorting, and preset selection. It leaves several params (concurrency, cache, top_k, max_total_input_tokens) entirely to the schema, so it enriches rather than fully documents.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Opens with a precise formula: "N items x ONE question set: one request per item", then names concrete item kinds (failing tests, diff hunks, files). It explicitly separates itself from jev_ask by scope (many items vs. several questions about one state), so an agent can route without opening either schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives explicit when-not guidance ("Not for several questions about one state (jev_ask)") and a concrete workflow recipe ("Diff review: preset review_filter..."). It also prescribes fallback behavior for error responses: judge inline, do not retry in a loop.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Deploy Server