jev-mcp
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@jev-mcptriage this failing test: my change, environment, or flaky timing?"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
jev-mcp
A fast, typed-judgment MCP server that lets coding agents (Claude Code first) ask TypeSafe's Jev
yes/no, choice, score and box questions, and act on the answer.
See SPEC.md for the full design and PROBE-RESULTS.md for live measurements.
What Jev is. Jev is the first System One model from TypeSafe: a model built to be used by machines, not to chat. "Send state and typed questions; get structured answers your code can use directly." It never writes text or code; it returns typed decisions with calibrated probabilities and a separate confidence, so software can act when confidence is high and escalate when it is low. TypeSafe's own comparison on a System One task: 0.114 s vs 8.566 s and $0.000081 vs $0.013880 against an LLM. With this server, the ~110-call live probe cost about $0.002 in total.
What jev-mcp adds. Three tools for coding agents, input refs instead of pasted state (
$file:,$diff,$log:), a secret guard that masks credentials before anything is sent, decision presets (auto | confirm | review), caching, rate and cost caps, and calibration from your own feedback. Stdio MCP server, TypeScript, Node >= 22, ESM,@typesafe-ai/sdk0.7.0 (exact pin).
Question types
Type | Jev returns | Typical use in coding work |
Noul | probability that the answer is yes | "Does this diff hunk change behaviour?", gate an action |
Choice | one option from a set, a probability per option, confidence | triage a failing test: my change / pre-existing / environment / timing / test bug |
Score | a level on an ordered, described scale, probabilities, confidence | rate the risk of a change, grade a review comment |
Box | four edges of a region in an image (fractions of width/height) | find a button or field in a screenshot |
Ground truth for the API: the TypeSafe docs at https://docs.typesafe.ai (index:
https://docs.typesafe.ai/llms.txt) and the schema at https://api.typesafe.ai/openapi.json.
Where SPEC.md disagrees with them, the official docs win.
Related MCP server: jev-code
Where Jev helps, from our tests
Measured with this server's stress suite (about 11,000 live calls on jev-1.13.0 / jev-1.14.0, 95% intervals, synthetic and real labelled items). Ranked from most to least useful for coding work:
Rank | Task | Result | Verdict | Final decision |
1 | Triage of failing tests and logs (what kind of failure is this?) | 100% accuracy, AUROC 1.000 on labelled test logs | Use it | Confirmed: same answers as a large model at about 1/80 of the cost. Jev handles it. |
2 | Clear, one-step classification (language, intent, sentiment, category) | 96.7 to 100% accuracy, AUROC 0.999; paraphrases agree 99.3% | Use it | Confirmed: a tie with a large model; disputed severity calls upheld 4 of 4. Jev handles it. |
3 | Many questions about one state (batch up to 30 per call) | same verdict as separate calls in 99.2% of pairs, 12.4x cheaper, 29.9x faster | Use it | Server default: up to 30 questions per request. |
4 | Finding a fact in a long file or log | 100% needle accuracy from 1 KB to 256 KB (long-context header above ~48 KB of logs) | Use it | Trim first; long context only when a log is over the limit. |
5 | Gating on confidence (act when sure, ask when not) | at confidence >= 0.9: 100% accuracy on 85% of items | Use it |
|
6 | Scoring on a scale; locating a region in a screenshot | score: 97.5 to 100% exact level; box: mean IoU 0.73, 87% hits | Use it, compare levels, not exact p | Compare levels and bands, never p to two decimals. |
7 | Screening diff hunks for relevance | 94% on security-diff decoys; mismatched bug/hunk pairs rejected 100% | Use it as a filter | First, cheap filter; the final code review stays with tests and a second review. |
8 | Subjective engineering choices ("A or B?") | winner changed with wording in 19.6% of framed calls, with option order in 17 to 33% of subjective items | Only with an order check and a human | Reversed-order check; a changed winner goes to |
9 | Three-step reasoning chains | 73 to 87% (two steps: 100%) | Split into single steps | One step per question; compute the rest in code. |
10 | "Is this bug report real?" / "Does this patch fix the bug?" | AUROC 0.64 and 0.57: near chance; accuracy equals the always-yes base rate | Do not use; run the tests, ask a reviewer | No model decides this: tests and a review do. A high p is never proof that a fix is right. |
Against a large general-purpose model judging the same items inline, Jev tied on clear classification, was also unusable on the two hard judgments, and cost about 80x less (about $0.00005 vs $0.0039 per item) at about 0.3 s per call.
Jev models compared
The same test sets on every model we could call (2026-10-21). jev-preview answered as jev-1.14.0 at the
time, so its column is a second sample of 1.14.0.
Measure | jev-1.13.0 | jev-1.14.0 | jev-preview |
Accuracy, 210 labelled items | 96.7% | 98.6% | 98.1% |
AUROC / calibration error (ECE) | 0.999 / 0.039 | 0.998 / 0.049 | 0.999 / 0.054 |
Clear items, 10 options: accuracy | 100% | 99.5% | 99.5% |
First-position bias, 10 options (none = 10%) | 9.2% | 10.0% | 10.4% |
Verdict flips over 20 identical calls | 0 | 0 | 0 |
Three-step reasoning | 73.3% | 86.7% | 80.0% |
Counting / arithmetic | 90.0% / 96.7% | 93.3% / 100% | 93.3% / 100% |
Dates, negation, long numbers, Hebrew = English | 100% | 100% | 100% |
Prompt injection inside the state: flip rate | 7.3% | 2.1% | 1.7% |
Latency p50 / p95 | 262 / 326 ms | 225 / 278 ms | 222 / 280 ms |
Pick: jev-1.14.0, pinned by default in this server: same accuracy and calibration as 1.13.0, better on three-step reasoning and prompt injection, and about 15% faster. Pin a version rather than a moving alias, so a model change never shifts your thresholds silently.
Short demo: triage a failing test
The agent pastes nothing; it sends the log line and five labelled options:
jev_ask {"recipe": "classify", "args": {
"state": "FAIL test/unit/day-usage-lock.test.ts > reclaims a stale lock: expected 5, received undefined. The test passed before the last commit that changed DayUsageStore.add.",
"options": {"my_change": "Caused by the current diff", "preexisting": "Failed before this change",
"env_infra": "Environment or infrastructure", "timing": "Timing or ordering", "test_bug": "The test itself is wrong"},
"none_option": "noul"}}Live answer (jev-1.14.0):
{"choice": "my_change", "confidence": 1, "probabilities": {"my_change": 1, "preexisting": 0, "env_infra": 0, "timing": 0, "test_bug": 0},
"usage": {"requests": 1, "input_tokens": 423, "est_cost_usd": 0.000018}}The agent now knows to look at its own last change first, for less than two thousandths of a cent.
Install
Requirements: Node.js >= 22, git, and a TypeSafe API key (create one at https://console.typesafe.ai/keys).
Clone and build (any folder, any OS):
git clone https://github.com/ablau911-ux/jev-mcp.git jev-mcp cd jev-mcp npm install npm run buildAPI key: set the environment variable
TYPESAFE_API_KEY, or put the key in the file~/.typesafe/api-key.txt(Windows:%USERPROFILE%\.typesafe\api-key.txt). The variable wins when it is not empty. The key only goes into theAuthorizationheader and is scrubbed from every output and log.Register with Claude Code (user scope), from the clone folder. The command stores the absolute path of your clone:
# bash / zsh claude mcp add --scope user jev -- node "$(pwd)/dist/server.js"# PowerShell claude mcp add --scope user jev -- node "$PWD\dist\server.js"Any other MCP client: add a stdio server to the client's MCP config. Use the absolute path of your clone:
{ "mcpServers": { "jev": { "command": "node", "args": ["/absolute/path/to/jev-mcp/dist/server.js"], "env": { "TYPESAFE_API_KEY": "<your key>" } } } }
Tools (exactly three)
Tool | Role |
| ONE state, N questions in one request. Or a recipe: |
| N items x ONE question set, one request per item. |
|
|
Recipes (former helper tools): classify, decide, verify_claim, rerank,
self_consistency, find_lines, locate. The server instructions are the router: routing
words first, rule 0, a "NOT for" list, one example per recipe and the fallback rule, under
4,096 characters. The full guide is the resource jev://guide.
Every result is concise JSON (structuredContent plus the same text, matching the declared
outputSchema): verdict, p, conf, decision (auto | confirm | review), model, an
id for feedback. Errors are isError with {kind, message, fix, retryable}; kind is
invalid_input | too_large | secret | auth | unavailable | rate_limited | budget_exceeded | cancelled | internal, plus cause, status, request_id and a JSON-path loc.
On unavailable, rate_limited or budget_exceeded: judge inline, do not retry in a loop.
Input refs (never paste state)
Ref | Expands to |
| the file text (line range, 1-based, inclusive) |
|
|
| the last N lines, stack traces cut to the top 5 non-library frames |
Files must be inside the working directory or the client's MCP roots (1 MiB max). Credential
files (.env*, *.pem, *.key, id_rsa*, app.ini, .ssh/, .aws/, .typesafe/, tea
config.yml, *credentials*, ...) are refused after realpath, with the same message as a
missing file; errors never echo the allowed roots.
Safety
Secret guard, no override. Every outbound string (state, instructions, criteria, every ref and
$diffhunk) is scanned. Matches are replaced by[REDACTED:<kind>]and the count is reported (secrets_redacted). Kinds: PEM keys, AWS, GitHub, Gitea, Slack, JWT, Bearer, genericsecret=..., connection strings, every value next to a secret-like key, and live secrets read at runtime (the TypeSafe key, the tea token). No argument, env var or config key turns it off;JEV_SECRET_MODE=refuseonly makes it stricter: mode: refuse refuses on any detection; the default is redact. It covers accidental leaks; deliberate obfuscation is out of scope. Threat model, coverage and how to report: SECURITY.md.Injection. Name untrusted fields
untrusted_*(and$log/ out-of-workspace files count as untrusted): a probe Noul runs in the same request and decisions are capped atconfirm(reviewwhen the probe fires).Egress. Every call sends data to TypeSafe. Tools carry
openWorldHint: trueandreadOnlyHint: true(Jev changes no local state, which may reduce approval prompts).JEV_CONFIRM_EGRESS=1addsanthropic/requiresUserInteraction(a prompt on every call, impractical for agent loops). The vendor's/legalterms (retention, region) were not read.
Models, decisions, calibration
Text and locate/box are pinned to
jev-1.14.0(live stress run 2026-10-21: prompt-injection flips 2.1% vs 7.3% on 1.13.0). The cache is keyed on the version; aliases are tracked fromresponse.model, and a changed version purges its cache namespace and caps decisions atconfirm.Presets in config:
gate(default),triage,review_filter(recall filter, flags p >= 0.3),strict(tight bands plus a reversed-order Choice check). Each threshold names its metric and sayscalibrated: falseandtuned_on. The live probe found no first-option bias on clear two-option code Choices, so order checks are opt-in.Calibration: every decision is logged (metadata only) to
~/.typesafe/jev-decisions.jsonl(Windows:%USERPROFILE%\.typesafe\). Labels come fromjev_status {action:"feedback", id, label}and from auto-labels (a flagged hunk later edited, a reranked file later read, a rerun that keeps or flips the verdict).jev_status {action:"calibrate", seed_git:true}reports ECE and a Wilson-bound threshold per preset once 30+ labels exist; it never changes config by itself.
Hooks (optional, no agent tokens)
jev-mcp-hook pre-tool-use asks before a Bash command that is not confidently harmless
(never auto-allows). jev-mcp-hook post-tool-use adds a warning when tool output looks like
injected instructions. Both fail open. Register them in settings.json only if you want them.
Limits and defaults
Price $0.042 per million input tokens (estimate). Context 64k per request, 32k per question, refused before sending;
long_context: "auto"uses the 262k header when needed.Token estimates use the probe factors: 3.3 chars/token for code and text, 1.4 for JSON structure, 1.2 for Hebrew and other non-Latin text (chars/4 under-counted by 17-69%).
Timeouts 5 s per attempt, 15 s per call (fitted inside
MCP_TOOL_TIMEOUTwhen set). Retries: 429, 529, network; other 5xx once; every retry resends the data.Session caps: 2M input tokens, 120 requests per minute, 5M per day, 200k tokens/s.
Batch: 6 workers, cap 8, halves on 429/529. Rerank window 16 (cap 64).
Images and Box: off unless
JEV_IMAGES=1. PNG/JPEG; EXIF removed; out-of-range PNGs resized.
Configuration
~/.typesafe/jev-config.json (Windows: %USERPROFILE%\.typesafe\jev-config.json) overrides any default in src/config/defaults.ts
and src/config/prompts.ts, with two floors: the secret guard's patterns and the credential
denylist can only be ADDED to (never removed, replaced or switched off), and a model pin
changes only through models.override: {text?, locate?} (results then carry
model_overridden: true; tool arguments and env vars cannot change a pin). Environment:
TYPESAFE_API_KEY, TYPESAFE_BASE_URL (official TypeSafe hosts only), JEV_COMPRESSION
(auto|zstd|off|gzip), JEV_CACHE_DIR (disk cache, refused inside a git repo),
JEV_BACKEND=mock (offline placeholders for tests and CI), JEV_IMAGES, JEV_SECRET_MODE,
JEV_CONFIRM_EGRESS, JEV_LOG_LEVEL, MCP_TOOL_TIMEOUT. Logs go to stderr only.
Checks and probe
npm run lint
npm run typecheck
JEV_LIVE=1 npm run probe # opt-in live probe (~110 calls, ~$0.002); see PROBE-RESULTS.mdThe probe needs a TypeSafe API key and sends synthetic data only.
License
MIT
Available Tools
3 toolsjev_askAsk Jev about one stateARead-only
ONE state, N questions: ask every typed question (noul yes/no p, choice label, score level) about one state in one call; answers come back with verdict, p and decision (auto | confirm | review). Use for: judging one failure log, one diff, one file, one command. For many items use jev_batch. Or run a recipe: {recipe:"classify"|"decide"|"verify_claim"|"rerank"|"self_consistency"|"find_lines"|"locate", args}. Not for: counting, dates, arithmetic, test pass/fail (exit code), summaries, judging whether a bug report is real or a patch fixes it. Refs instead of pasting: "$file:a.ts#L1-80", "$diff", "$log:x.log". If it returns unavailable / rate_limited / budget_exceeded, judge inline; do not retry in a loop. Example: {state:{diff:"$diff:staged"}, questions:{leak:{type:"noul",instructions:"Does diff add a hardcoded credential?"}}}
| Name | Required | Description | Default |
|---|---|---|---|
| args | No | Arguments of the recipe. | |
| cache | No | Use the response cache (default true). | |
| state | No | What the questions are about: a string, or an object/array with named fields. Refs are expanded on the server, never paste files: "$file:src/a.ts#L10-80", "$diff" (uncommitted) / "$diff:staged" / "$diff:<ref>", "$log:test.log". Name untrusted fields untrusted_*. | |
| detail | No | Add probabilities and routing reasons (more tokens). | |
| images | No | Images by placeholder like "<shot>", written inside a state string. Off unless the user set JEV_IMAGES=1. | |
| preset | No | Decision preset: gate (default) | triage | review_filter | strict. Thresholds live in config. | |
| recipe | No | Run a named recipe instead of raw questions; pass its args. See instructions for one example each. | |
| questions | No | Questions by unique name (names are never shown to the model). Put every question about this state in one call. | |
| long_context | No | Long-context header (262k tokens, slower, less accurate). "auto" uses it only when over the 64k/32k limits. |
Output Schema
| Name | Required | Description |
|---|---|---|
| id | No | |
| error | No | Only on a failed call (isError): kind, message, fix, retryable, plus cause / loc / status / request_id when known. |
| model | No | |
| usage | No | |
| recipe | No | |
| answers | No | |
| warnings | No | |
| model_overridden | No | |
| secrets_redacted | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations declare readOnlyHint and openWorldHint, and the description goes well beyond that: it discloses the return shape (verdict/p/decision), the recipe mode, server-side ref expansion, and critically the degradation behavior (unavailable / rate_limited / budget_exceeded) with an instruction not to retry in a loop. That failure-mode guidance is exactly the kind of context structured fields cannot carry.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with the core constraint and organized into Use for / Not for / refs / failure behavior / example – every sentence earns its place. The density and shorthand ("noul", "$diff:staged") make it slightly harder to parse on first read, but nothing is padding.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 9-parameter tool with nested objects and an output schema, the description covers the remaining gaps an agent would otherwise stumble on: which state to pass, how to reference files instead of pasting, which recipes exist, and what to do when the service degrades. Nothing needed to invoke it correctly is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3; the description still adds value by naming the three question types (noul yes/no p, choice label, score level), the recipe form with the args convention, and a full worked example call. It stops short of explaining preset, detail, cache, long_context or images beyond what the schema already says.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Opens with a precise verb+resource+cardinality constraint ("ONE state, N questions") and immediately states that answers come back with verdict, p and decision. It explicitly contrasts with the sibling jev_batch for the many-item case, so an agent can route without opening either schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives positive triggers ("judging one failure log, one diff, one file, one command"), an explicit alternative for the many-item case ("For many items use jev_batch"), and a concrete not-for list excluding counting, dates, arithmetic, exit codes and patch-correctness judgments. This is the strongest form of routing guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
jev_batchJudge many items with one question setARead-only
N items x ONE question set: one request per item (failing tests, diff hunks, files), concurrent with a shared rate gate. Returns rows {item, verdict, p, decision} sorted by p as top-k or only the flagged ones, plus a summary and every failure. items_from builds items on the server: "$diff:staged" (one per hunk) or ["$file:a.ts", ...]. Diff review: preset review_filter with narrow Nouls where true = bad. Not for several questions about one state (jev_ask). If it returns unavailable / rate_limited / budget_exceeded, judge inline; do not retry in a loop. Example: {items_from:"$diff:staged", preset:"review_filter", questions:{no_err:{type:"noul",instructions:"Does diff remove error handling?"}}}
| Name | Required | Description | Default |
|---|---|---|---|
| cache | No | Use the response cache (default true). | |
| items | No | Your items (one request each). | |
| top_k | No | ||
| detail | No | Add per-question answers to every row (more tokens). | |
| preset | No | Decision preset: gate (default) | triage | review_filter | strict. Thresholds live in config. | |
| return | No | ranked = top_k by p; flagged = only flagged / non-auto items; all. Default: flagged for review_filter, else ranked when rank_by, else all. | |
| rank_by | No | Question name to sort by (p descending). | |
| only_ids | No | Re-run only these item ids (reruns also auto-label earlier decisions). | |
| questions | Yes | One question set asked of every item. | |
| items_from | No | Items built on the server: "$diff" / "$diff:staged" / "$diff:<ref>" = one item per hunk {file,hunk,diff}; ["$file:a.ts", ...] = one item per file {path,text}. | |
| concurrency | No | ||
| output_path | No | Also write the returned rows as JSONL here (inside the workspace). | |
| long_context | No | Long-context header (262k tokens, slower, less accurate). "auto" uses it only when over the 64k/32k limits. | |
| stop_on_first_error | No | ||
| max_total_input_tokens | No |
Output Schema
| Name | Required | Description |
|---|---|---|
| error | No | Only on a failed call (isError): kind, message, fix, retryable, plus cause / loc / status / request_id when known. |
| items | No | |
| model | No | |
| summary | No | |
| failures | No | |
| warnings | No | |
| output_path | No | |
| model_overridden | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond readOnlyHint/openWorldHint, it discloses execution traits an agent cannot infer: concurrent requests behind a shared rate gate, result ordering by p, summary plus every failure, and named failure modes (unavailable / rate_limited / budget_exceeded) with an explicit no-retry policy. Return rows are described even though an output schema exists.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with the core contract and tightly packed, with a closing worked example that earns its space. Some phrasing is telegraphic to the point of obscurity ("narrow Nouls where true = bad"), which costs a point on clarity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 15 params, nested question objects, and an output schema, the description covers the calling contract, the distinguishing alternative, error handling, and an example. Residual gaps are minor: where preset thresholds come from and what concurrency/cache defaults imply in practice.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 15 params at 73% schema coverage, the description adds real meaning for the tricky ones: items_from forms, return top-k vs flagged semantics, rank_by sorting, and preset selection. It leaves several params (concurrency, cache, top_k, max_total_input_tokens) entirely to the schema, so it enriches rather than fully documents.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Opens with a precise formula: "N items x ONE question set: one request per item", then names concrete item kinds (failing tests, diff hunks, files). It explicitly separates itself from jev_ask by scope (many items vs. several questions about one state), so an agent can route without opening either schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives explicit when-not guidance ("Not for several questions about one state (jev_ask)") and a concrete workflow recipe ("Diff review: preset review_filter..."). It also prescribes fallback behavior for error responses: judge inline, do not retry in a loop.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
jev_statusJev health, usage and modelsARead-only
Jev health and housekeeping, no judgments: health (is Jev reachable, which models are pinned), usage (tokens, cost estimate, caps, estimate/actual ratio), models (aliases and versions), validate (offline dry run of a jev_ask input), presets (decision thresholds), feedback (label a past decision), calibrate (fit thresholds from labels). Free except health/models refresh (one GET).
| Name | Required | Description | Default |
|---|---|---|---|
| id | No | feedback: a decision id from a result (id, or id.<question>). | |
| label | No | feedback: true when the verdict was right. | |
| reset | No | usage: clear the session counters after reporting. | |
| action | No | health (default) | usage | models | validate | presets | feedback | calibrate. | |
| preset | No | calibrate: only this preset. | |
| refresh | No | health/models: check again now. | |
| request | No | validate: a jev_ask input {state, questions, images?}; checked offline, nothing sent. | |
| seed_git | No | calibrate: first seed eval cases from git history (fix-test commits). |
Output Schema
| Name | Required | Description |
|---|---|---|
| ok | No | |
| error | No | Only on a failed call (isError): kind, message, fix, retryable, plus cause / loc / status / request_id when known. |
| action | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint and openWorldHint, so the description's main added value is the cost model: "Free except health/models refresh (one GET)" plus the note that validate is checked offline with "nothing sent." It does not spell out side effects of \"feedback\" or \"calibrate\" (they persist labels/thresholds, and \"reset\" clears counters), which sits in mild tension with readOnlyHint, though these are internal bookkeeping rather than changes to the user's resources.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
One dense opening sentence front-loads the action inventory, followed by the single most decision-relevant caveat about cost. It is compact with no filler, though the parenthetical chain is packed enough that a skim reader may lose the action-to-parameter mapping.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
All seven actions are named and briefly defined, the cost/refresh caveat is disclosed, and an output schema exists so return values need not be described. The remaining gap is routing guidance against jev_ask/jev_batch, which the description does not address.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% and every parameter already carries an action-prefixed description, so the schema does the heavy lifting. The description echoes the same action-to-parameter pairings (feedback/label, health/models/refresh) but adds no syntax or format detail beyond what the schema states.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific scope ("Jev health and housekeeping, no judgments") and then enumerates all seven actions with parenthetical definitions, so an agent can tell what each invocation does. It differentiates itself from jev_ask only obliquely via "validate (offline dry run of a jev_ask input)"; there is no explicit statement that it is the status/introspection counterpart to the query tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The action list implies when each mode applies, and "validate (offline dry run of a jev_ask input)" links it to the ask workflow, but there is no explicit "use this when X, use jev_ask/jev_batch when Y" routing. An agent must infer the split from the sibling names alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
3 tool updates
v0.2.0- First observed
jev_ask - First observed
jev_batch - First observed
jev_status
TDQS
Scored across 3 tools
The three tools split cleanly along orthogonal axes: jev_ask (one state, many questions), jev_batch (many items, one question set), and jev_status (health/housekeeping, no judgments). Descriptions explicitly include 'Not for' clauses that steer agents away from the wrong tool (e.g., 'For many items use jev_batch', 'Not for several questions about one state').
All names follow a single predictable pattern: the shared `jev_` prefix plus a short snake_case noun (ask, batch, status). No mixed conventions or verb-style drift.
Three tools is on the lean side, but each occupies a distinct role (single judgment, batched judgment, meta/ops) and nothing feels redundant. A slightly larger surface (e.g., a recipe-listing tool) might round it out, but the current set is well-scoped.
The surface covers the full judgment lifecycle: single calls, concurrent batch calls, offline validation, threshold presets, feedback labeling, and calibration. Minor gaps exist — no dedicated tool to list or inspect past decisions/history, and recipe discovery is only implied through the `recipe` arg rather than being a first-class operation.
Maintenance
Related MCP Connectors
Calibrated judgments for text: yes/no probabilities, picks from your options, or scores.
Human judgment for AI agents: discover capabilities, get quotes, and track paid human tasks.
Give any AI agent a way to ask a person — for approval, a decision, an answer or a review.
- TaskrizOAuthcom.taskriz
Let your AI agent ask a person for a decision, a date or a photo, and get a structured answer back.
Related MCP Servers
- AlicenseAqualityCmaintenanceProvides agents with fast, typed, calibrated decision tools for classification, scoring, yes/no checks, and gating risky tool calls.5319 npm3MIT
- AlicenseNot gradedqualityBmaintenanceProvides coding agents with typed classification, yes/no checks, scoring, ranking, and question-answering tools that return calibrated probabilities for fast, reliable decisions.224 npm46MIT
- AlicenseNot gradedqualityAmaintenanceEnables coding agents to send natural language and JSON application state to a judge tool that returns typed judgments (yes/no probability, choice, or score) with model and usage metadata, and fails gracefully without blocking workflows.12 npm5Apache 2.0
- AlicenseAqualityCmaintenanceEnables agents to call typed code-review and content-moderation decision tools, returning structured verdicts, probabilities, and confidence-gated actions.2MIT