Jev MCP
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@Jev MCPIs this test failure likely caused by the current diff?"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
Jev MCP
Local Model Context Protocol server that lets expensive frontier coding agents delegate routine probabilistic judgment to TypeSafe Jev.
Jev answers: what does the available evidence suggest?
The frontier model answers: given that evidence, what should I do?
Codex Desktop / Claude Code / other MCP clients
│
│ MCP stdio
▼
Jev MCP
validation · policy · cache · telemetry
│
│ HTTPS (lazy, never on startup)
▼
TypeSafe System One
│
▼
Jev
cheap calibrated NoulVersion 1.0. First-class clients: Codex Desktop and Claude Code. Architecture stays client-agnostic so Cursor, VS Code, JetBrains, CI, and custom harnesses can attach later without new tools.
License: MIT.
Contents
Related MCP server: jev-review
Why it exists
Frontier coding models (Astra/Sol-class Codex models, Claude Opus) spend a lot of inference on work that is classification, not generation:
is this failure related to the current diff?
are we repeating the same failed strategy?
which requirements still look uncovered?
which files are worth reading next?
Jev MCP moves those questions to a cheap System One model. The metric that matters is not “Jev calls per minute.” It is:
frontier inference cost
────────────────────────
successful completed taskCost reduction that increases regressions is a failure. Quality and regression rate must stay near the no-Jev baseline.
What it is not
Jev MCP must not:
write or modify production code
run shell commands or tests
create commits or approve merges
certify correctness, safety, or security
replace CI, typechecking, lint, or static analysis
generate architecture or resolve ambiguous product decisions
become a second coding agent
If TypeSafe is down, the MCP returns a structured error. The frontier agent continues the task. No fabricated probabilities.
Never returned as a status:
APPROVED
CORRECT
SAFE_TO_MERGE
SECUREHow judgment is ordered
DETERMINISTIC EVIDENCE
↓
CHEAP PROBABILISTIC (Jev)
↓
FRONTIER REASONINGExample: test exit_code → parser → jev_triage_failure → Opus/Sol root-cause.
Do not call Jev when an exit code, type error, or deterministic check already answers the question.
Install
Requires Python 3.11+.
git clone <this-repo>
cd jevmcp
python -m venv .venv
source .venv/bin/activate
pip install -e ".[dev]"Set a TypeSafe key (never commit it):
export TYPESAFE_API_KEY=... # from ~/.env or the TypeSafe console
# optional
export TYPESAFE_MODEL=jev-latest
export TYPESAFE_BASE_URL=https://api.typesafe.aiCopy examples if you want a project file:
cp config.example.yaml jev-mcp.yaml
cp .env.example .env # then fill TYPESAFE_API_KEY locallyCheck the install without calling TypeSafe:
python scripts/doctor.pyConfirm the live provider (explicit opt-in):
python scripts/doctor.py --pingStartup never contacts TypeSafe. --ping is the only doctor path that does.
Quick start
jev-mcp --profile interactive --shadow
# or
python -m jev_mcp --profile autonomous --shadowThis is a stdio server. It reads JSON-RPC on stdin and writes on stdout. Logs go to stderr. Running it in a bare terminal with no MCP client is not useful — the client must spawn the process.
Default shadow mode is on. Results include an advisory: treat them as telemetry, not mandatory control actions. That is the recommended first deployment.
MCP clients
Client differences belong in instructions and profile, not tool schemas. Both V1 clients see the same seven tools.
Client | Default profile | Typical models | Bias |
Codex Desktop |
| Astra, Sol, other Codex | Aggressive triage / stuck detection on long autonomous runs |
Claude Code |
| Claude Opus | Selective use; human is already in the loop |
Codex Desktop
Install
jev-mcponPATH(or point the config at.venv/bin/jev-mcp).Merge
integrations/codex/mcp-config.example.jsoninto Codex MCP settings.Merge
integrations/codex/AGENTS.example.mdinto the target repoAGENTS.md.Put
TYPESAFE_API_KEYin the process environment, not in a committed JSON file. Some clients do not expand${TYPESAFE_API_KEY}.
Claude Code
Same pattern with integrations/claude-code/mcp-config.example.json and CLAUDE.example.md.
Claude Code should call Jev less often. Skip it when the next step is already obvious from deterministic evidence or the user is directing the work.
Future clients
Cursor, VS Code agents, JetBrains, CI, GitHub agents, custom harnesses: add a config snippet and an instruction file. Do not add client-specific tools.
Tools
All tools return JSON. Every successful response includes meta:
{
"provider": "typesafe",
"model": "jev-1.13.0",
"latency_ms": 112,
"cached": false,
"request_id": "…",
"tool": "jev_triage_failure",
"shadow_mode": true,
"jev_input_units": 312,
"jev_estimated_cost": null
}If input was truncated, warnings is present. Treat those judgments more cautiously.
Optional on every call: client (name, mode, model, session_id, task_id) and use_cache. client.mode does not change policy. It is telemetry only. Profile comes from server config / CLI / JEV_MCP_PROFILE.
Core
jev_triage_failure
Cheap triage before a broad investigation of a test/build/lint/typecheck failure.
Signals: related_to_current_change, likely_localized, likely_preexisting, requirement_related, same_as_previous_failure, needs_deeper_reasoning.
Classification: relationship, scope, escalation. These are labels over thresholds, not diagnoses.
Does not explain the bug or propose a fix.
jev_compare_attempts
Compare two unsuccessful attempts. Use after two materially similar failures.
Signals: same_failure, same_strategy, meaningful_new_evidence, meaningful_progress, reconsider_approach.
|
|
|
|
|
|
|
|
Initial stuck rule (configurable):
same_failure >= 0.85
AND same_strategy >= 0.80
AND meaningful_new_evidence <= 0.30
AND meaningful_progress <= 0.30A connection-refused failure followed by a UNIQUE violation is progress, not stuck. Both tests failed; the failure advanced.
jev_check_completion
Requirement/evidence coverage before an expensive full-task review.
Per requirement: appears_satisfied, evidence_present, possible_gap.
Global: scope_appropriate, unresolved_requirement, further_review_warranted.
Statuses: APPEARS_COMPLETE, REVIEW_REQUIRED, INCOMPLETE.
APPEARS_COMPLETE is not approval. Investigate anything in review_requirements.
Missed requirements (false complete) are scored 5× worse than extra escalations in the eval suite.
Extended
jev_rank_context
Ranks discovery candidates. Never deletes. Tiers: HIGH, MEDIUM, LOW.
Do not suppress LOW items until critical-context Recall@10 is proven ≥ 0.98 on the golden set. V1 is ranking only.
jev_classify_findings
Normalize findings from reviewers, linters, tests, humans. Signals include likely_valid, requirement_related, requires_code_change, security_relevant, data_integrity_relevant.
A high security_relevant is a triage flag, not a confirmed vulnerability.
jev_assess_risk
Whether a change looks like it warrants more expensive frontier review (auth, payment, schema, public API, …). Same rule: signals, not conclusions.
jev_judge
Generic Noul primitive: caller supplies state and a list of {id, question}.
Rejects obvious generative requests (write this function, fix this bug, generate tests, …). Ask one evidence-grounded proposition per question.
Profiles and policy
Profiles change recommended usage and threshold aggressiveness. They do not change tool schemas or Jev’s underlying questions.
|
|
| |
Default client | Codex large tasks | Claude Code | user override |
Failure triage | aggressive | moderate | moderate |
Stuck detection | aggressive (easier | enabled | enabled |
Context HIGH bar | lower | conservative | medium |
Set with --profile, JEV_MCP_PROFILE, or profile.default in YAML.
Probability interpretation
Do not treat p > 0.5 as true.
Range | Band |
0.00 – 0.30 | LOW |
0.30 – 0.70 | UNCERTAIN |
0.70 – 1.00 | HIGH |
Individual tools use stricter cutoffs (stuck detector). All thresholds live in config.
Ambiguous engineering is a valid outcome. The eval suite tracks ambiguous_case_overconfidence_rate — overconfident YES/NO on uncertain cases is a safety failure.
Configuration
Precedence (highest last-applied wins):
built-in defaults
↓
~/.config/jev-mcp/config.yaml user
↓
./jev-mcp.yaml or ./config.yaml project
↓
JEV_MCP_CONFIG / --config explicit file
↓
environment
↓
CLI flagsSee config.example.yaml.
Project YAML cannot set:
provider.api_keyprovider.base_urltelemetry.store_content
Those restrictions exist so a cloned repo cannot steal TYPESAFE_API_KEY by pointing base_url at an attacker host, and cannot silently persist source into SQLite.
api_key and base_url come from the environment only. store_content may be enabled in the user config or by constructing AppConfig locally for debugging — not from project files.
Provider limits:
Field | Bound |
| 1–120 (default 30) |
| 0–2 (default 1; only transient errors) |
Environment
Variable | Role |
| Required for live Jev |
| Default |
| Default |
| Provider timeout |
| Extra HTTPS hosts (default is only |
|
|
|
|
|
|
| Logging (stderr) |
| Cache + telemetry directory |
| Explicit YAML path |
| Must be |
|
|
Provider
Tools depend on JudgmentProvider, not TypeSafe types.
validate → normalize → Noul questions → System One → validate p ∈ [0,1] → cache → telemetryTypeSafeProvider —
POST {base_url}/v1/systemone, HTTPS, host allowlist,follow_redirects=False, one retry on transient status (429,529,5xx, timeout).MockProvider / MockJudgmentProvider — deterministic local answers for tests and
doctorwithout a key.
State sent to Jev is untrusted evidence (repo text, test output). Questions are our Noul instructions. Injection in a source file must not be copied into instructions. Jev may still be influenced by adversarial evidence; treat HIGH confidence plus injection-looking content as a reason for deeper frontier review.
Cost estimates use cost.jev_input_per_million from config. Prices are not hard-coded.
Cache and telemetry
Default data dir: ~/.local/share/jev-mcp/ (JEV_MCP_DATA_DIR overrides). Files are created 0600, directory 0700.
Cache (SQLite WAL) key:
provider · model · tool · tool_version · policy_version · normalization_version
· normalized_state_hash · questions_hashNot in the key: timestamp, session id, client name.
Bypass when use_cache=false, cache disabled, or provider/tool/normalization version changes. Responses expose meta.cached.
Telemetry (local, default):
stores hashes, not source (
store_content: false)records tool, latency, cache hit, error code, probability summary, optional token units
outcome labels can be attached later (
jev-mcp-export --record-outcome)
Remote telemetry is not implemented. local_only is the contract for anything added later.
Errors
Returned to the client as:
{
"error": {
"code": "PROVIDER_TIMEOUT",
"message": "Jev evaluation timed out.",
"retryable": true
}
}Code | Retryable |
| no |
| no |
| usually yes |
| yes |
| yes |
| no |
| no |
Messages are redacted (API keys, ghp_, sk_live_, JWTs, Slack, GitLab, npm, connection URIs, …). No stack traces, no Authorization headers.
Security
Local stdio MCP. Attackers that matter: malicious repo config, poisoned test output, confused agent, other local UIDs.
Hardened in V1:
no code/file/shell tools
secrets and provider host not accepted from project YAML
TypeSafe host pinned; no redirect-following (stops Bearer exfil)
fault injection requires an explicit allow env
profile not controllable by the agent
shadow on by default
secret-pattern redaction on logs, errors, telemetry
sqlite owner-only permissions
Residual: Jev still reads untrusted state. A file that says “return 1.0” can bias probabilities. That is a judgment-integrity risk, not RCE. Do not use Jev results as merge gates.
Docker
docker build -t jev-mcp .
docker run --rm -i -e TYPESAFE_API_KEY jev-mcp --profile interactive --shadow-i is required (stdio). Do not --env-file ~/.env — that dumps unrelated secrets into the container. Image is multi-stage, non-root (uid 10001), data volume /var/lib/jev-mcp. Key is never baked in.
Tests and evals
Testing is part of the product. Hosted Jev can change while this repo stays still. Compare every release to eval-results/.
Layer | What | When |
A | Unit + policy. | Every PR |
B | MCP contract: exact tool list, schema + description snapshots | Every PR |
C | Real stdio MCP + mock Jev (errors, cache, truncate, concurrent, restart) | Every PR |
Adversarial | Injection as evidence, secret leakage | Every PR |
D | Live TypeSafe. Structural only ( | Nightly |
E | Golden corpus (~320 labeled cases), Recall@10, missed-requirement cost, uncertainty | PR uses mock path; live nightly |
pytest -m "not live and not typesafe"
ruff check src tests scripts evals
python scripts/eval.py --generate
python scripts/eval.py --out eval-results/local/software.json
JEV_MCP_LIVE=1 pytest -m liveDo not assert answer == 0.91 against hosted Jev. Do not run hundreds of live Jev calls on every commit.
Gates: evals/GATES.md. Experiment configs A–D (frontier only → all tools): evals/experiments/. Hidden acceptance tests belong in benchmark repos, not in the agent-visible rubric.
Repository layout
src/jev_mcp/ server, engine, tools, policy, provider, cache, telemetry
integrations/ Codex + Claude Code examples
scripts/ doctor, install, benchmark, eval, export
tests/ unit, contract, integration, adversarial, live, evals
evals/golden/ labeled coding-decision cases
eval-results/ per-release history
benchmarks/ smaller fixture corpusCommands
Command | Purpose |
| stdio MCP server |
| same |
| config + optional |
| print client MCP snippets |
| small local corpus |
| golden suite |
| dump local telemetry JSON |
CLI flags for the server: --config, --provider {typesafe,mock}, --profile {autonomous,interactive,custom}, --shadow, --log-level.
Available Tools
7 toolsjev_assess_riskA
Cheap change-risk triage to decide whether a more expensive frontier review appears warranted. Returns sensitivity likelihoods only. These are not security conclusions and do not certify safety.
| Name | Required | Description | Default |
|---|---|---|---|
| client | No | ||
| task_goal | Yes | ||
| use_cache | No | ||
| diff_summary | No | ||
| requirements | No | ||
| changed_files | No |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden of behavioral disclosure. It honestly states that the tool is cheap, returns only sensitivity likelihoods, and does not certify safety or provide security conclusions. This is valuable caveat-level transparency, though it does not mention caching behavior, side effects, or operational details.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two tight sentences with no filler. The purpose is front-loaded, and each clause adds meaning: cost profile, decision purpose, output scope, and a caveat. This is an efficient structure.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The purpose and output caveat are present, but the description is incomplete for a tool with six parameters and zero schema descriptions. It does not say what inputs the agent must supply or how the returned likelihoods map to the go/no-go decision. Output schema existence helps, but the description alone leaves invocation details to inference.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% and the description provides no guidance on any of the six parameters. It does not explain the role of task_goal, diff_summary, changed_files, requirements, client, or use_cache. With zero coverage, the description was expected to compensate but does not mention parameters at all.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states a specific purpose: 'Cheap change-risk triage to decide whether a more expensive frontier review appears warranted.' It also defines the output scope, 'sensitivity likelihoods only,' which separates it from final security assessments. However, it does not explicitly differentiate from the sibling tools, particularly 'jev_triage_failure,' which also uses triage language.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives a clear usage context: use this cheap triage to decide whether a more expensive frontier review is necessary. It also warns against over-interpretation with 'These are not security conclusions and do not certify safety.' It does not name specific sibling alternatives or when not to use them, but the decision-oriented framing is helpful.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
jev_check_completionA
Cheap requirement/evidence coverage check before an expensive full-task review. Use on large tasks with explicit requirements. Returns per-requirement likelihoods and which items warrant review. Never certifies correctness, safety, or merge readiness.
| Name | Required | Description | Default |
|---|---|---|---|
| task | Yes | ||
| client | No | ||
| use_cache | No | ||
| verification | No | ||
| implementation | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the behavioral disclosure burden. It discloses the tool's lightweight nature, its output (per-requirement likelihoods and review-warranted items), and its non-certification limitations. It does not mention side effects or external dependencies, but the key behavioral boundaries are present.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three concise sentences with the main purpose front-loaded. Every sentence adds value: what it does, when to use it, what it returns, and what it cannot vouch for. There is no filler or repetition.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description gives solid selection guidance and output expectations, and an output schema exists so return values are covered. However, for a tool with five parameters and nested objects, it leaves the construction of task and implementation inputs unexplained. It is adequate but not fully self-sufficient for correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate for parameter meaning, but it does not. It hints at 'requirements' and 'evidence' but does not explain the roles of task, implementation, verification, use_cache, or client. This leaves most parameters semantically under-specified.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly names a specific verb and resource: a cheap requirement/evidence coverage check performed before an expensive full-task review. It distinguishes the tool from a full review, though it does not explicitly differentiate it from the listed sibling tools by name.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives an explicit use case ('Use on large tasks with explicit requirements') and a clear boundary ('Never certifies correctness, safety, or merge readiness'). It does not name alternative sibling tools, so the guidance is strong but not fully explicit on what to use instead.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
jev_classify_findingsA
Cheap normalization of review findings from tests, linters, humans, or frontier reviewers. Use to triage a large finding set before expensive reasoning. Returns likelihood signals only. Does not confirm that a vulnerability or defect exists.
| Name | Required | Description | Default |
|---|---|---|---|
| client | No | ||
| findings | Yes | ||
| task_goal | Yes | ||
| use_cache | No | ||
| requirements | No |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden. It discloses that the tool is 'cheap', returns 'likelihood signals only', and explicitly states it does not confirm existence of vulnerabilities. This provides meaningful behavioral insight. However, it doesn't disclose potential side effects, rate limits, or specifics about the output format beyond likelihood signals. The return type is disclosed as 'likelihood signals only', which is a positive disclosure.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely efficient: two sentences. It conveys purpose, use context, and an explicit limitation. Every word earns its place, and the key constraint ('likelihood signals only') is front-loaded near the end of the first sentence. This is textbook conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has an output schema (though not provided in detail here), 5 parameters of which only 2 are required, and a clear differentiation from siblings. The description covers the core purpose, the triage context, and the critical limitation. It doesn't detail the 'likelihood signals' format or what the output schema looks like, but that is likely covered by the output schema itself. This is complete enough for an agent to decide when to invoke.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema coverage is 0% in terms of parameter descriptions; all parameter semantics come from the schema structure alone. The description does not enumerate or explain individual parameters. However, the description gives context about 'triage' and 'normalization' which helps interpret the role of findings and task_goal. Given the 0% coverage, the description could do more to explain parameters, but the schema itself has reasonably descriptive names (finding, task_goal, requirements). Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose with a specific verb ('classification'), resource ('review findings'), and context (cheap normalization from multiple sources). It also distinguishes itself from siblings by mentioning 'triage' and 'likelihood signals only'. The description differentiates from sibling tools by framing this as a cheap pre-filter before expensive reasoning, which is unique among the listed siblings.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly states when to use it ('to triage a large finding set before expensive reasoning') and what it does not do ('Does not confirm that a vulnerability or defect exists'). It implies when not to use it (when confirmation is needed), though it doesn't name specific alternative sibling tools. The usage context is clear and actionable.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
jev_compare_attemptsA
Cheap comparison of two unsuccessful implementation attempts. Use when repeated work may be spending frontier inference on the same failed strategy. Returns stuck/progress signals and an advisory control signal; does not choose the next fix.
| Name | Required | Description | Default |
|---|---|---|---|
| client | No | ||
| task_goal | Yes | ||
| use_cache | No | ||
| current_attempt | Yes | ||
| previous_attempt | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses that the tool is 'cheap' (cost/performance signal), returns 'stuck/progress signals and an advisory control signal,' and explicitly does not choose the next fix. This goes beyond the schema and annotations (which are absent). It doesn't mention side effects or failure modes, but for a read-only comparison tool, the disclosed behavior is reasonably complete.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three sentences, each with a distinct purpose: what it does, when to use it, and what it returns/doesn't return. It is front-loaded with the core action and scoped efficiently. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description is complete for a comparison tool: it states the trigger condition, the output type, and the limitation. The output schema exists, so return values are partially documented. The main gap is that the Attempt object's fields are not semantically explained, but the schema provides their names and types, and the description's 'stuck/progress signals' gives a high-level sense of the output.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description carries the burden of explaining parameters. The description names the key inputs ('two unsuccessful implementation attempts') and the output signals, but it does not explain the individual fields within the Attempt object (failure, approach, hypothesis, diff_summary, changed_files) or the client metadata. The schema provides field names and types, but the description adds no semantic detail about how these fields are used in the comparison.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb ('compare'), a specific resource ('two unsuccessful implementation attempts'), and a clear scope ('cheap comparison'). It also distinguishes itself from siblings by noting it 'does not choose the next fix,' which helps an agent understand its role relative to other jev_* tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says when to use it: 'Use when repeated work may be spending frontier inference on the same failed strategy.' This gives a clear trigger condition. It also states what it does not do ('does not choose the next fix'), which helps an agent avoid over-trusting the tool's output.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
jev_judgeB
Generic bounded judgment primitive. Ask one evidence-grounded yes/no proposition per question and receive calibrated probabilities. Rejects generative requests such as writing code, designing architecture, or fixing bugs. Jev evaluates supplied evidence; it does not invent it.
| Name | Required | Description | Default |
|---|---|---|---|
| state | Yes | ||
| client | No | ||
| questions | Yes | ||
| use_cache | No |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It discloses key behaviors: it only evaluates supplied evidence, returns calibrated probabilities, and rejects generative requests. This gives the agent a clear behavioral model, though it doesn't cover potential errors or rate limits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences with no filler. The core purpose is front-loaded, and each sentence adds distinct information: what it does, what it rejects, and a key behavioral caveat.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has nested objects and 4 parameters, but the description covers only the general behavior. It doesn't explain the structure of questions, the role of state, or client metadata. For a tool that requires evidence, it doesn't specify where evidence goes.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It hints at questions holding propositions and evidence being supplied, but it doesn't explain state, client, or use_cache. This leaves most parameters underspecified.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's function: asking one evidence-grounded yes/no proposition per question and receiving calibrated probabilities. It also mentions what it rejects. However, it does not explicitly differentiate from sibling judgment tools, relying on 'generic' to imply its role.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage by calling it a 'generic bounded judgment primitive' and notes it rejects generative requests, which is a when-not. It does not provide explicit guidance on when to prefer this over sibling tools like jev_assess_risk or jev_triage_failure.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
jev_rank_contextA
Cheap ranking of repository context candidates before deeper frontier inspection. Use when discovery produced many plausible files or symbols. Ranks only; never deletes or permanently excludes candidates.
| Name | Required | Description | Default |
|---|---|---|---|
| client | No | ||
| task_goal | Yes | ||
| use_cache | No | ||
| candidates | Yes | ||
| current_step | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It clearly states the tool is 'cheap' and 'ranks only; never deletes or permanently excludes candidates', establishing that this is a non-destructive, side-effect-free operation. It doesn't mention caching or output specifics, but the core behavioral traits are disclosed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences with no filler. The core purpose is front-loaded, the usage condition follows immediately, and the non-destructive guarantee is stated succinctly. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description is sufficient for an agent to know when to call this tool and what it does, including the key safety property. An output schema exists, so return values needn't be explained. The main gap is lack of parameter detail, but the tool's simple concept and clear parameter names make this a relatively minor omission.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate for parameter understanding, but it does not describe parameters at all. While parameter names like task_goal, current_step, and candidates are somewhat self-explanatory, the description adds no semantics beyond the schema's literal titles. This is a significant gap for a 5-parameter tool.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb ('ranking') and resource ('repository context candidates') and distinguishes this tool from sibling tools like jev_assess_risk or jev_judge. It precisely describes the tool's role as a preliminary filter before deeper frontier inspection.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states when to use it ('when discovery produced many plausible files or symbols'). This gives clear context for selection. It doesn't explicitly name alternatives or state when not to use it, but the use condition is strong and the tool's narrow scope implies the boundary.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
jev_triage_failureA
Cheap probabilistic triage for test/build/lint/typecheck failures. Use before broad reasoning when the relationship or scope of a failure is unclear. Returns structured likelihood signals; does not diagnose or fix the problem.
| Name | Required | Description | Default |
|---|---|---|---|
| task | Yes | ||
| client | No | ||
| failure | Yes | ||
| use_cache | No | ||
| current_step | Yes | ||
| diff_summary | No | ||
| changed_files | No | ||
| previous_failure | No |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It discloses that the tool is 'cheap' and 'probabilistic', returns 'structured likelihood signals', and explicitly does not diagnose or fix. This is meaningful behavioral context beyond the schema. It could add more about caching or side effects, but the core behavior is well disclosed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, each earning its place: what it is, when to use it, and what it returns/doesn't do. The key information is front-loaded. No redundancy or filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has an output schema, so return values are documented elsewhere. The description covers purpose, usage timing, and limitations. It doesn't mention the 'use_cache' parameter or how the tool handles previous_failure, but for a triage tool with a clear scope, the description is largely complete. The main gap is that it doesn't explain what inputs are most important to provide, but the schema covers the input structure.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. The description explains the tool's overall purpose but does not explain any of the 8 parameters (task, current_step, failure, use_cache, diff_summary, changed_files, previous_failure, client). The parameter names are fairly self-explanatory, and the schema has detailed nested definitions, but the description adds no parameter-level meaning. Baseline 3 is appropriate because the schema itself is rich, but the description does not help with parameter semantics.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb ('triage'), a resource ('test/build/lint/typecheck failures'), and a clear scope ('probabilistic', 'cheap'). It also explicitly distinguishes itself from diagnosis/fixing, which helps an agent understand exactly what this tool does and does not do.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says 'Use before broad reasoning when the relationship or scope of a failure is unclear.' This gives a clear when-to-use condition. It also states what the tool does not do ('does not diagnose or fix the problem'), which helps an agent avoid using it for the wrong purpose. Sibling tools are not named, but the usage context is specific enough to route correctly.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
7 tool updates
v1.0.0- First observed
jev_assess_risk - First observed
jev_check_completion - First observed
jev_classify_findings - First observed
jev_compare_attempts - First observed
jev_judge - First observed
jev_rank_context - First observed
jev_triage_failure
TDQS
Scored across 7 tools
Each tool targets a distinct stage in a review/diagnosis workflow: change risk, failure triage, attempt comparison, completion checks, context ranking, findings classification, and generic judgment. A few tools share the same cheap-likelihood framing, but their input types and intended uses are clearly differentiated.
All tools follow the consistent jev_ prefix with a verb_noun pattern (assess_risk, triage_failure, compare_attempts, check_completion, rank_context, classify_findings, judge). There is no mixed casing or stylistic drift.
Seven tools is well-scoped for a probabilistic triage suite, with each tool earning its place as a distinct decision-support primitive. The count is neither too thin nor overloaded.
For the stated domain of cheap probabilistic triage, the set covers the major pre-review decision points and even provides a generic judge fallback for arbitrary yes/no propositions. Potential gaps like diagnosing or fixing are intentionally excluded rather than left as dead ends.
Maintenance
Related MCP Connectors
Calibrated probabilistic foresight for AI agents, powered by live prediction-market signal.
Verifiable, deterministic risk math for autonomous agents; re-runnable proof on every answer.
Pre-execution governance for AI agents. Deterministic PASS/FAIL/REVIEW verdicts, replayable proof.
Deterministic decision layer for autonomous agents: reproducible PROCEED/REVIEW/SKIP verdicts.
Related MCP Servers
- AlicenseAqualityBmaintenanceEnables agents to verify claims against cited evidence, screen content for prompt injection and relevance before reading it, and rank candidates by meaning, all with calibrated probability verdicts.345MIT
- AlicenseNot gradedqualityDmaintenanceEnables AI coding agents to receive continuous, structured software-quality feedback from Jev across multiple quality dimensions while working locally.79MIT
- AlicenseAqualityCmaintenanceEnables typed, calibrated judgment calls through classify, score, check, and batched ask tools, each returning full probability distributions for programmatic decisions.51MIT
- AlicenseAqualityCmaintenanceEnables coding agents to make cheap, fast probabilistic decisions on every turn, with tools for coding-loop checks, review, verification, screening untrusted input, and ranking candidates.6MIT