jev-mcp

Overview Schema Source Related Servers Discussions Admin

Evaluate many items with Jev

jev.evaluate_batch

Read-only

Batch-evaluate up to 100 items using typed questions in one call to classify, score, rank, or tag each item and return ordered results with usage summaries.

Instructions

Ask TypeSafe Jev the same typed questions about many items (up to 100) in one call, for example to classify a list of tickets or to score every search result and sort them.

items is a list of { "state": ..., "id"?: "...", "images"?: [...] }; each item is judged on its own. questions and the optional model work exactly as in jev.evaluate and apply to every item. Use a choice question to classify, a score or noul question to rank, and several questions to tag.

Each item is a separate provider request and is billed separately. Every item is checked before anything is sent; one invalid item rejects the whole call. Returns results in input order: status "ok" with answers and usage, "error" with the item's own error, or "skipped" when an earlier error (for example authentication or no credits) would also have hit it. summary counts each status and usage adds up the successful items.

images (optional, up to 4 PNG, JPEG or WebP, 4 MiB each) are judged together with the item's state; each is either { "path": "" } for a local file inside a directory the server allows (JEV_IMAGE_DIRS), or { "data": "data:image/png;base64,..." }. Only image-capable models accept images (for example cloudflare/clef or cloudflare/clef-flash through OpenRouter); other models reject them. Never put images inside state.

provider is optional; this server offers only "typesafe".

Input Schema

Name	Required	Description	Default
`items`	Yes
`model`	No
`provider`	No
`questions`	Yes

Output Schema

Name	Required	Description
`usage`	Yes	Sum over successful items; costUsd only when every successful item reported a cost
`results`	Yes	One result per item, in input order
`summary`	Yes
`provider`	Yes	The provider every item was sent to

Schema Changelog

Changes observed during successful MCP inspections.

First observedv0.6.0Oct 28, 2026

TDQS

A4.7/5.0

Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes well beyond readOnlyHint/openWorldHint: discloses per-item billing, all-or-nothing pre-validation ('one invalid item rejects the whole call'), per-item ok/error/skipped statuses with the auth/no-credits cascade rule, and image model capability limits. This is exactly the operational context an agent cannot infer from annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with purpose and usage patterns, then parameters, then provider. Dense but nearly every sentence adds a constraint; the paragraph on image models and the JEV_IMAGE_DIRS aside could be tightened slightly.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Output schema exists, yet the description still summarizes `results`/`summary`/`usage` shapes in a way that clarifies ordering and status semantics. Together with the image, validation, and billing rules, an agent has everything needed to call this correctly on the first try.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description carries the full burden and does so: it documents the `items` element shape, the optional `id`, the two image forms with size/format caps and the JEV_IMAGE_DIRS restriction, that `provider` only accepts 'typesafe', and that images must never be nested in `state`. These are non-obvious constraints absent from the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource — 'Ask TypeSafe Jev the same typed questions about many items (up to 100) in one call' — and names the motivating use cases (classify tickets, score and sort search results). The batch-vs-single distinction from jev.evaluate is made explicit via the reference that questions/model 'work exactly as in jev.evaluate'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives concrete routing advice: use a choice question to classify, a score or noul question to rank, several questions to tag, and points to jev.evaluate for parameter semantics. It doesn't state an explicit exclusion (e.g. 'use jev.evaluate for a single item'), but the one-call/up-to-100 framing makes the boundary obvious.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Deploy Server