Skip to main content
Glama

jev-mcp

A fast, typed-judgment MCP server that lets coding agents (Claude Code first) ask TypeSafe's Jev yes/no, choice, score and box questions, and act on the answer. See SPEC.md for the full design and PROBE-RESULTS.md for live measurements.

What Jev is. Jev is the first System One model from TypeSafe: a model built to be used by machines, not to chat. "Send state and typed questions; get structured answers your code can use directly." It never writes text or code; it returns typed decisions with calibrated probabilities and a separate confidence, so software can act when confidence is high and escalate when it is low. TypeSafe's own comparison on a System One task: 0.114 s vs 8.566 s and $0.000081 vs $0.013880 against an LLM. With this server, the ~110-call live probe cost about $0.002 in total.

What jev-mcp adds. Three tools for coding agents, input refs instead of pasted state ($file:, $diff, $log:), a secret guard that masks credentials before anything is sent, decision presets (auto | confirm | review), caching, rate and cost caps, and calibration from your own feedback. Stdio MCP server, TypeScript, Node >= 22, ESM, @typesafe-ai/sdk 0.7.0 (exact pin).

Question types

Type

Jev returns

Typical use in coding work

Noul

probability that the answer is yes

"Does this diff hunk change behaviour?", gate an action

Choice

one option from a set, a probability per option, confidence

triage a failing test: my change / pre-existing / environment / timing / test bug

Score

a level on an ordered, described scale, probabilities, confidence

rate the risk of a change, grade a review comment

Box

four edges of a region in an image (fractions of width/height)

find a button or field in a screenshot

Ground truth for the API: the TypeSafe docs at https://docs.typesafe.ai (index: https://docs.typesafe.ai/llms.txt) and the schema at https://api.typesafe.ai/openapi.json. Where SPEC.md disagrees with them, the official docs win.

Related MCP server: jev-code

Where Jev helps, from our tests

Measured with this server's stress suite (about 11,000 live calls on jev-1.13.0 / jev-1.14.0, 95% intervals, synthetic and real labelled items). Ranked from most to least useful for coding work:

Rank

Task

Result

Verdict

Final decision

1

Triage of failing tests and logs (what kind of failure is this?)

100% accuracy, AUROC 1.000 on labelled test logs

Use it

Confirmed: same answers as a large model at about 1/80 of the cost. Jev handles it.

2

Clear, one-step classification (language, intent, sentiment, category)

96.7 to 100% accuracy, AUROC 0.999; paraphrases agree 99.3%

Use it

Confirmed: a tie with a large model; disputed severity calls upheld 4 of 4. Jev handles it.

3

Many questions about one state (batch up to 30 per call)

same verdict as separate calls in 99.2% of pairs, 12.4x cheaper, 29.9x faster

Use it

Server default: up to 30 questions per request.

4

Finding a fact in a long file or log

100% needle accuracy from 1 KB to 256 KB (long-context header above ~48 KB of logs)

Use it

Trim first; long context only when a log is over the limit.

5

Gating on confidence (act when sure, ask when not)

at confidence >= 0.9: 100% accuracy on 85% of items

Use it

auto only at p <= 0.1 or >= 0.9 (Choice: confidence >= 0.95); the rest goes to confirm.

6

Scoring on a scale; locating a region in a screenshot

score: 97.5 to 100% exact level; box: mean IoU 0.73, 87% hits

Use it, compare levels, not exact p

Compare levels and bands, never p to two decimals.

7

Screening diff hunks for relevance

94% on security-diff decoys; mismatched bug/hunk pairs rejected 100%

Use it as a filter

First, cheap filter; the final code review stays with tests and a second review.

8

Subjective engineering choices ("A or B?")

winner changed with wording in 19.6% of framed calls, with option order in 17 to 33% of subjective items

Only with an order check and a human

Reversed-order check; a changed winner goes to review, never auto.

9

Three-step reasoning chains

73 to 87% (two steps: 100%)

Split into single steps

One step per question; compute the rest in code.

10

"Is this bug report real?" / "Does this patch fix the bug?"

AUROC 0.64 and 0.57: near chance; accuracy equals the always-yes base rate

Do not use; run the tests, ask a reviewer

No model decides this: tests and a review do. A high p is never proof that a fix is right.

Against a large general-purpose model judging the same items inline, Jev tied on clear classification, was also unusable on the two hard judgments, and cost about 80x less (about $0.00005 vs $0.0039 per item) at about 0.3 s per call.

Jev models compared

The same test sets on every model we could call (2026-10-21). jev-preview answered as jev-1.14.0 at the time, so its column is a second sample of 1.14.0.

Measure

jev-1.13.0

jev-1.14.0

jev-preview

Accuracy, 210 labelled items

96.7%

98.6%

98.1%

AUROC / calibration error (ECE)

0.999 / 0.039

0.998 / 0.049

0.999 / 0.054

Clear items, 10 options: accuracy

100%

99.5%

99.5%

First-position bias, 10 options (none = 10%)

9.2%

10.0%

10.4%

Verdict flips over 20 identical calls

0

0

0

Three-step reasoning

73.3%

86.7%

80.0%

Counting / arithmetic

90.0% / 96.7%

93.3% / 100%

93.3% / 100%

Dates, negation, long numbers, Hebrew = English

100%

100%

100%

Prompt injection inside the state: flip rate

7.3%

2.1%

1.7%

Latency p50 / p95

262 / 326 ms

225 / 278 ms

222 / 280 ms

Pick: jev-1.14.0, pinned by default in this server: same accuracy and calibration as 1.13.0, better on three-step reasoning and prompt injection, and about 15% faster. Pin a version rather than a moving alias, so a model change never shifts your thresholds silently.

Short demo: triage a failing test

The agent pastes nothing; it sends the log line and five labelled options:

jev_ask {"recipe": "classify", "args": {
  "state": "FAIL test/unit/day-usage-lock.test.ts > reclaims a stale lock: expected 5, received undefined. The test passed before the last commit that changed DayUsageStore.add.",
  "options": {"my_change": "Caused by the current diff", "preexisting": "Failed before this change",
              "env_infra": "Environment or infrastructure", "timing": "Timing or ordering", "test_bug": "The test itself is wrong"},
  "none_option": "noul"}}

Live answer (jev-1.14.0):

{"choice": "my_change", "confidence": 1, "probabilities": {"my_change": 1, "preexisting": 0, "env_infra": 0, "timing": 0, "test_bug": 0},
 "usage": {"requests": 1, "input_tokens": 423, "est_cost_usd": 0.000018}}

The agent now knows to look at its own last change first, for less than two thousandths of a cent.

Install

  1. Requirements: Node.js >= 22, git, and a TypeSafe API key (create one at https://console.typesafe.ai/keys).

  2. Clone and build (any folder, any OS):

    git clone https://github.com/ablau911-ux/jev-mcp.git jev-mcp
    cd jev-mcp
    npm install
    npm run build
  3. API key: set the environment variable TYPESAFE_API_KEY, or put the key in the file ~/.typesafe/api-key.txt (Windows: %USERPROFILE%\.typesafe\api-key.txt). The variable wins when it is not empty. The key only goes into the Authorization header and is scrubbed from every output and log.

  4. Register with Claude Code (user scope), from the clone folder. The command stores the absolute path of your clone:

    # bash / zsh
    claude mcp add --scope user jev -- node "$(pwd)/dist/server.js"
    # PowerShell
    claude mcp add --scope user jev -- node "$PWD\dist\server.js"
  5. Any other MCP client: add a stdio server to the client's MCP config. Use the absolute path of your clone:

    {
      "mcpServers": {
        "jev": {
          "command": "node",
          "args": ["/absolute/path/to/jev-mcp/dist/server.js"],
          "env": { "TYPESAFE_API_KEY": "<your key>" }
        }
      }
    }

Tools (exactly three)

Tool

Role

jev_ask

ONE state, N questions in one request. Or a recipe: {recipe, args}. Loads upfront (anthropic/alwaysLoad).

jev_batch

N items x ONE question set, one request per item. items_from: "$diff:staged" (one item per hunk) or ["$file:a.ts", ...]. Returns rows {item, verdict, p, decision} ranked (top-k) or flagged, a summary and every failure.

jev_status

health (startup check), usage (tokens, cost estimate, caps, estimate/actual ratio), models, validate (offline dry run), presets, feedback, calibrate.

Recipes (former helper tools): classify, decide, verify_claim, rerank, self_consistency, find_lines, locate. The server instructions are the router: routing words first, rule 0, a "NOT for" list, one example per recipe and the fallback rule, under 4,096 characters. The full guide is the resource jev://guide.

Every result is concise JSON (structuredContent plus the same text, matching the declared outputSchema): verdict, p, conf, decision (auto | confirm | review), model, an id for feedback. Errors are isError with {kind, message, fix, retryable}; kind is invalid_input | too_large | secret | auth | unavailable | rate_limited | budget_exceeded | cancelled | internal, plus cause, status, request_id and a JSON-path loc. On unavailable, rate_limited or budget_exceeded: judge inline, do not retry in a loop.

Input refs (never paste state)

Ref

Expands to

$file:src/a.ts, $file:src/a.ts#L10-80

the file text (line range, 1-based, inclusive)

$diff, $diff:staged, $diff:HEAD, $diff:<ref>

git diff split per hunk [{file, hunk, diff}]

$log:test.log, $log:test.log#500

the last N lines, stack traces cut to the top 5 non-library frames

Files must be inside the working directory or the client's MCP roots (1 MiB max). Credential files (.env*, *.pem, *.key, id_rsa*, app.ini, .ssh/, .aws/, .typesafe/, tea config.yml, *credentials*, ...) are refused after realpath, with the same message as a missing file; errors never echo the allowed roots.

Safety

  • Secret guard, no override. Every outbound string (state, instructions, criteria, every ref and $diff hunk) is scanned. Matches are replaced by [REDACTED:<kind>] and the count is reported (secrets_redacted). Kinds: PEM keys, AWS, GitHub, Gitea, Slack, JWT, Bearer, generic secret=..., connection strings, every value next to a secret-like key, and live secrets read at runtime (the TypeSafe key, the tea token). No argument, env var or config key turns it off; JEV_SECRET_MODE=refuse only makes it stricter: mode: refuse refuses on any detection; the default is redact. It covers accidental leaks; deliberate obfuscation is out of scope. Threat model, coverage and how to report: SECURITY.md.

  • Injection. Name untrusted fields untrusted_* (and $log / out-of-workspace files count as untrusted): a probe Noul runs in the same request and decisions are capped at confirm (review when the probe fires).

  • Egress. Every call sends data to TypeSafe. Tools carry openWorldHint: true and readOnlyHint: true (Jev changes no local state, which may reduce approval prompts). JEV_CONFIRM_EGRESS=1 adds anthropic/requiresUserInteraction (a prompt on every call, impractical for agent loops). The vendor's /legal terms (retention, region) were not read.

Models, decisions, calibration

  • Text and locate/box are pinned to jev-1.14.0 (live stress run 2026-10-21: prompt-injection flips 2.1% vs 7.3% on 1.13.0). The cache is keyed on the version; aliases are tracked from response.model, and a changed version purges its cache namespace and caps decisions at confirm.

  • Presets in config: gate (default), triage, review_filter (recall filter, flags p >= 0.3), strict (tight bands plus a reversed-order Choice check). Each threshold names its metric and says calibrated: false and tuned_on. The live probe found no first-option bias on clear two-option code Choices, so order checks are opt-in.

  • Calibration: every decision is logged (metadata only) to ~/.typesafe/jev-decisions.jsonl (Windows: %USERPROFILE%\.typesafe\). Labels come from jev_status {action:"feedback", id, label} and from auto-labels (a flagged hunk later edited, a reranked file later read, a rerun that keeps or flips the verdict). jev_status {action:"calibrate", seed_git:true} reports ECE and a Wilson-bound threshold per preset once 30+ labels exist; it never changes config by itself.

Hooks (optional, no agent tokens)

jev-mcp-hook pre-tool-use asks before a Bash command that is not confidently harmless (never auto-allows). jev-mcp-hook post-tool-use adds a warning when tool output looks like injected instructions. Both fail open. Register them in settings.json only if you want them.

Limits and defaults

  • Price $0.042 per million input tokens (estimate). Context 64k per request, 32k per question, refused before sending; long_context: "auto" uses the 262k header when needed.

  • Token estimates use the probe factors: 3.3 chars/token for code and text, 1.4 for JSON structure, 1.2 for Hebrew and other non-Latin text (chars/4 under-counted by 17-69%).

  • Timeouts 5 s per attempt, 15 s per call (fitted inside MCP_TOOL_TIMEOUT when set). Retries: 429, 529, network; other 5xx once; every retry resends the data.

  • Session caps: 2M input tokens, 120 requests per minute, 5M per day, 200k tokens/s.

  • Batch: 6 workers, cap 8, halves on 429/529. Rerank window 16 (cap 64).

  • Images and Box: off unless JEV_IMAGES=1. PNG/JPEG; EXIF removed; out-of-range PNGs resized.

Configuration

~/.typesafe/jev-config.json (Windows: %USERPROFILE%\.typesafe\jev-config.json) overrides any default in src/config/defaults.ts and src/config/prompts.ts, with two floors: the secret guard's patterns and the credential denylist can only be ADDED to (never removed, replaced or switched off), and a model pin changes only through models.override: {text?, locate?} (results then carry model_overridden: true; tool arguments and env vars cannot change a pin). Environment: TYPESAFE_API_KEY, TYPESAFE_BASE_URL (official TypeSafe hosts only), JEV_COMPRESSION (auto|zstd|off|gzip), JEV_CACHE_DIR (disk cache, refused inside a git repo), JEV_BACKEND=mock (offline placeholders for tests and CI), JEV_IMAGES, JEV_SECRET_MODE, JEV_CONFIRM_EGRESS, JEV_LOG_LEVEL, MCP_TOOL_TIMEOUT. Logs go to stderr only.

Checks and probe

npm run lint
npm run typecheck
JEV_LIVE=1 npm run probe      # opt-in live probe (~110 calls, ~$0.002); see PROBE-RESULTS.md

The probe needs a TypeSafe API key and sends synthetic data only.

License

MIT

Available Tools

3 tools
jev_askAsk Jev about one stateA
Read-only

ONE state, N questions: ask every typed question (noul yes/no p, choice label, score level) about one state in one call; answers come back with verdict, p and decision (auto | confirm | review). Use for: judging one failure log, one diff, one file, one command. For many items use jev_batch. Or run a recipe: {recipe:"classify"|"decide"|"verify_claim"|"rerank"|"self_consistency"|"find_lines"|"locate", args}. Not for: counting, dates, arithmetic, test pass/fail (exit code), summaries, judging whether a bug report is real or a patch fixes it. Refs instead of pasting: "$file:a.ts#L1-80", "$diff", "$log:x.log". If it returns unavailable / rate_limited / budget_exceeded, judge inline; do not retry in a loop. Example: {state:{diff:"$diff:staged"}, questions:{leak:{type:"noul",instructions:"Does diff add a hardcoded credential?"}}}

NameRequiredDescriptionDefault
argsNoArguments of the recipe.
cacheNoUse the response cache (default true).
stateNoWhat the questions are about: a string, or an object/array with named fields. Refs are expanded on the server, never paste files: "$file:src/a.ts#L10-80", "$diff" (uncommitted) / "$diff:staged" / "$diff:<ref>", "$log:test.log". Name untrusted fields untrusted_*.
detailNoAdd probabilities and routing reasons (more tokens).
imagesNoImages by placeholder like "<shot>", written inside a state string. Off unless the user set JEV_IMAGES=1.
presetNoDecision preset: gate (default) | triage | review_filter | strict. Thresholds live in config.
recipeNoRun a named recipe instead of raw questions; pass its args. See instructions for one example each.
questionsNoQuestions by unique name (names are never shown to the model). Put every question about this state in one call.
long_contextNoLong-context header (262k tokens, slower, less accurate). "auto" uses it only when over the 64k/32k limits.

Output Schema

NameRequiredDescription
idNo
errorNoOnly on a failed call (isError): kind, message, fix, retryable, plus cause / loc / status / request_id when known.
modelNo
usageNo
recipeNo
answersNo
warningsNo
model_overriddenNo
secrets_redactedNo

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations declare readOnlyHint and openWorldHint, and the description goes well beyond that: it discloses the return shape (verdict/p/decision), the recipe mode, server-side ref expansion, and critically the degradation behavior (unavailable / rate_limited / budget_exceeded) with an instruction not to retry in a loop. That failure-mode guidance is exactly the kind of context structured fields cannot carry.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the core constraint and organized into Use for / Not for / refs / failure behavior / example – every sentence earns its place. The density and shorthand ("noul", "$diff:staged") make it slightly harder to parse on first read, but nothing is padding.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 9-parameter tool with nested objects and an output schema, the description covers the remaining gaps an agent would otherwise stumble on: which state to pass, how to reference files instead of pasting, which recipes exist, and what to do when the service degrades. Nothing needed to invoke it correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3; the description still adds value by naming the three question types (noul yes/no p, choice label, score level), the recipe form with the args convention, and a full worked example call. It stops short of explaining preset, detail, cache, long_context or images beyond what the schema already says.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Opens with a precise verb+resource+cardinality constraint ("ONE state, N questions") and immediately states that answers come back with verdict, p and decision. It explicitly contrasts with the sibling jev_batch for the many-item case, so an agent can route without opening either schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives positive triggers ("judging one failure log, one diff, one file, one command"), an explicit alternative for the many-item case ("For many items use jev_batch"), and a concrete not-for list excluding counting, dates, arithmetic, exit codes and patch-correctness judgments. This is the strongest form of routing guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

jev_batchJudge many items with one question setA
Read-only

N items x ONE question set: one request per item (failing tests, diff hunks, files), concurrent with a shared rate gate. Returns rows {item, verdict, p, decision} sorted by p as top-k or only the flagged ones, plus a summary and every failure. items_from builds items on the server: "$diff:staged" (one per hunk) or ["$file:a.ts", ...]. Diff review: preset review_filter with narrow Nouls where true = bad. Not for several questions about one state (jev_ask). If it returns unavailable / rate_limited / budget_exceeded, judge inline; do not retry in a loop. Example: {items_from:"$diff:staged", preset:"review_filter", questions:{no_err:{type:"noul",instructions:"Does diff remove error handling?"}}}

NameRequiredDescriptionDefault
cacheNoUse the response cache (default true).
itemsNoYour items (one request each).
top_kNo
detailNoAdd per-question answers to every row (more tokens).
presetNoDecision preset: gate (default) | triage | review_filter | strict. Thresholds live in config.
returnNoranked = top_k by p; flagged = only flagged / non-auto items; all. Default: flagged for review_filter, else ranked when rank_by, else all.
rank_byNoQuestion name to sort by (p descending).
only_idsNoRe-run only these item ids (reruns also auto-label earlier decisions).
questionsYesOne question set asked of every item.
items_fromNoItems built on the server: "$diff" / "$diff:staged" / "$diff:<ref>" = one item per hunk {file,hunk,diff}; ["$file:a.ts", ...] = one item per file {path,text}.
concurrencyNo
output_pathNoAlso write the returned rows as JSONL here (inside the workspace).
long_contextNoLong-context header (262k tokens, slower, less accurate). "auto" uses it only when over the 64k/32k limits.
stop_on_first_errorNo
max_total_input_tokensNo

Output Schema

NameRequiredDescription
errorNoOnly on a failed call (isError): kind, message, fix, retryable, plus cause / loc / status / request_id when known.
itemsNo
modelNo
summaryNo
failuresNo
warningsNo
output_pathNo
model_overriddenNo

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond readOnlyHint/openWorldHint, it discloses execution traits an agent cannot infer: concurrent requests behind a shared rate gate, result ordering by p, summary plus every failure, and named failure modes (unavailable / rate_limited / budget_exceeded) with an explicit no-retry policy. Return rows are described even though an output schema exists.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the core contract and tightly packed, with a closing worked example that earns its space. Some phrasing is telegraphic to the point of obscurity ("narrow Nouls where true = bad"), which costs a point on clarity.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given 15 params, nested question objects, and an output schema, the description covers the calling contract, the distinguishing alternative, error handling, and an example. Residual gaps are minor: where preset thresholds come from and what concurrency/cache defaults imply in practice.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 15 params at 73% schema coverage, the description adds real meaning for the tricky ones: items_from forms, return top-k vs flagged semantics, rank_by sorting, and preset selection. It leaves several params (concurrency, cache, top_k, max_total_input_tokens) entirely to the schema, so it enriches rather than fully documents.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Opens with a precise formula: "N items x ONE question set: one request per item", then names concrete item kinds (failing tests, diff hunks, files). It explicitly separates itself from jev_ask by scope (many items vs. several questions about one state), so an agent can route without opening either schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives explicit when-not guidance ("Not for several questions about one state (jev_ask)") and a concrete workflow recipe ("Diff review: preset review_filter..."). It also prescribes fallback behavior for error responses: judge inline, do not retry in a loop.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

jev_statusJev health, usage and modelsA
Read-only

Jev health and housekeeping, no judgments: health (is Jev reachable, which models are pinned), usage (tokens, cost estimate, caps, estimate/actual ratio), models (aliases and versions), validate (offline dry run of a jev_ask input), presets (decision thresholds), feedback (label a past decision), calibrate (fit thresholds from labels). Free except health/models refresh (one GET).

NameRequiredDescriptionDefault
idNofeedback: a decision id from a result (id, or id.<question>).
labelNofeedback: true when the verdict was right.
resetNousage: clear the session counters after reporting.
actionNohealth (default) | usage | models | validate | presets | feedback | calibrate.
presetNocalibrate: only this preset.
refreshNohealth/models: check again now.
requestNovalidate: a jev_ask input {state, questions, images?}; checked offline, nothing sent.
seed_gitNocalibrate: first seed eval cases from git history (fix-test commits).

Output Schema

NameRequiredDescription
okNo
errorNoOnly on a failed call (isError): kind, message, fix, retryable, plus cause / loc / status / request_id when known.
actionNo

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint and openWorldHint, so the description's main added value is the cost model: "Free except health/models refresh (one GET)" plus the note that validate is checked offline with "nothing sent." It does not spell out side effects of \"feedback\" or \"calibrate\" (they persist labels/thresholds, and \"reset\" clears counters), which sits in mild tension with readOnlyHint, though these are internal bookkeeping rather than changes to the user's resources.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

One dense opening sentence front-loads the action inventory, followed by the single most decision-relevant caveat about cost. It is compact with no filler, though the parenthetical chain is packed enough that a skim reader may lose the action-to-parameter mapping.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

All seven actions are named and briefly defined, the cost/refresh caveat is disclosed, and an output schema exists so return values need not be described. The remaining gap is routing guidance against jev_ask/jev_batch, which the description does not address.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and every parameter already carries an action-prefixed description, so the schema does the heavy lifting. The description echoes the same action-to-parameter pairings (feedback/label, health/models/refresh) but adds no syntax or format detail beyond what the schema states.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific scope ("Jev health and housekeeping, no judgments") and then enumerates all seven actions with parenthetical definitions, so an agent can tell what each invocation does. It differentiates itself from jev_ask only obliquely via "validate (offline dry run of a jev_ask input)"; there is no explicit statement that it is the status/introspection counterpart to the query tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The action list implies when each mode applies, and "validate (offline dry run of a jev_ask input)" links it to the ask workflow, but there is no explicit "use this when X, use jev_ask/jev_batch when Y" routing. An agent must infer the split from the sibling names alone.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 3 tool updatesv0.2.0
    • First observedjev_ask
    • First observedjev_batch
    • First observedjev_status

TDQS

A4.1/5.0

Scored across 3 tools

Disambiguation5/5

The three tools split cleanly along orthogonal axes: jev_ask (one state, many questions), jev_batch (many items, one question set), and jev_status (health/housekeeping, no judgments). Descriptions explicitly include 'Not for' clauses that steer agents away from the wrong tool (e.g., 'For many items use jev_batch', 'Not for several questions about one state').

Naming Consistency5/5

All names follow a single predictable pattern: the shared `jev_` prefix plus a short snake_case noun (ask, batch, status). No mixed conventions or verb-style drift.

Tool Count4/5

Three tools is on the lean side, but each occupies a distinct role (single judgment, batched judgment, meta/ops) and nothing feels redundant. A slightly larger surface (e.g., a recipe-listing tool) might round it out, but the current set is well-scoped.

Completeness4/5

The surface covers the full judgment lifecycle: single calls, concurrent batch calls, offline validation, threshold presets, feedback labeling, and calibration. Minor gaps exist — no dedicated tool to list or inspect past decisions/history, and recipe discovery is only implied through the `recipe` arg rather than being a first-class operation.

Maintenance

ActivityMaintained
ResponsivenessNo issues

Related MCP Connectors

Related MCP Servers

  • A
    license
    Not graded
    quality
    A
    maintenance
    Enables coding agents to send natural language and JSON application state to a judge tool that returns typed judgments (yes/no probability, choice, or score) with model and usage metadata, and fails gracefully without blocking workflows.
    12 npm
    5
    Apache 2.0
  • A
    license
    A
    quality
    C
    maintenance
    Enables agents to call typed code-review and content-moderation decision tools, returning structured verdicts, probabilities, and confidence-gated actions.
    2
    MIT