jev-judge-mcp

An MCP server giving a coding agent one tool, judge, for consulting TypeSafe's Jev: narrow typed questions about supplied evidence, answered with calibrated probabilities instead of prose.

Jev complements the active model; it does not replace it.

Install

Two things get installed: the MCP server, which gives your agent the judge tool, and the skill, which teaches it to design a call worth making. They install separately; npx skills handles skills only and never touches MCP servers.

Let your agent install it

Paste this into Claude Code, Codex, OpenCode, or any other coding agent. Nothing to clone or configure first.

Install the Jev judge MCP tool and its skill into yourself, the coding agent reading this.

1. Work out which agent you are, and run only your own line:

   • Claude Code: claude plugin marketplace add gecm0/jev-judge-mcp then claude plugin install typesafe@jev. That registers the server and the skill together, so skip step 2.

   • Codex: codex mcp add jev -- npx -y jev-judge-mcp

   • OpenCode: opencode mcp add jev -- npx -y jev-judge-mcp

   • Any other agent: register npx -y jev-judge-mcp as a stdio MCP server in its config.

2. Install the skill: npx skills add gecm0/jev-judge-mcp --skill jev

3. Install the companion skill this one refers to: npx skills add typesafe-ai/skills --skill typesafe-ai

4. Report back which agent you registered with, plus this instruction: get a key from https://console.typesafe.ai/, export TYPESAFE_API_KEY in the shell that launches you, and restart. Leave the key in the environment, where the config references it by name.

You are done when judge shows up in your own tool list after that restart.

Or do it by hand

The server is plain MCP over stdio and needs no vendor-specific environment, so any client works. For one without an mcp add command, put this in its config:

{
  "mcpServers": {
    "jev": { "command": "npx", "args": ["-y", "jev-judge-mcp"] }
  }
}

Codex uses ~/.codex/config.toml with a [mcp_servers.jev] table; OpenCode v2 uses opencode.jsonc with mcp.servers.jev and "type": "local". Both also accept codex mcp add and opencode mcp add, which write the right shape for the version you have.

The skill is a plain SKILL.md directory that Claude Code, Codex and OpenCode all read. npx skills places it for you; copying skills/jev/ into ~/.codex/skills/ or .opencode/skills/ works too. Copy the directory, not just the file: skill IDs and relative paths come from it.

To run a working copy instead of the published one, clone the repo, pnpm install, and point the config at node /absolute/path/to/mcp/server.mjs.

API key

Get one from TypeSafe. Set TYPESAFE_API_KEY in the environment that launches your client, then restart it. Do not paste the key into chat and do not commit it. This server does not load .env files.

Claude Code and OpenCode can reference the variable from their config (${TYPESAFE_API_KEY} and {env:TYPESAFE_API_KEY}), so the key never lands in a config file. Codex takes literal strings in [mcp_servers.jev.env], so prefer exporting the variable in the shell that starts it.

TYPESAFE_MODEL optionally pins a version, for example jev-1.13.0. The default, jev-latest, follows TypeSafe's stable alias and can change over time. The response reports the model the API actually used.

Without a key the tool is still listed, and fails with a clear message before making a request.

Related MCP server: Jev MCP

What it contains

MCP has no hook for always-on usage guidance beyond the tool description, so the guidance is split in two, with each fact in exactly one place: the tool description holds what prevents a wrong action (when to reach for it, and how to misread a result), since it is loaded on every turn; the skill holds question design, and is reached through a pointer at the end of the tool description. Adding a fact to both is a regression, not redundancy.

Use

Ask your agent, for example:

Use Jev to classify this bug report and assess its severity in one call: Export crashes in Safari but works in Chrome.

The agent supplies evidence and independent questions:

{
  "state": { "report": "Export crashes in Safari but works in Chrome." },
  "questions": {
    "team": {
      "type": "choice",
      "instructions": "Which team should handle the report?",
      "criteria": {
        "engineering": "Broken functionality",
        "other": "None of the listed teams fits"
      }
    },
    "all_browsers": {
      "type": "noul",
      "instructions": "Does the report say every browser is affected?"
    },
    "severity": {
      "type": "score",
      "instructions": "How severe is the reported defect?",
      "criteria": [
        "Cosmetic; functionality still works",
        "Broken functionality with an available workaround",
        "Blocking defect without an available workaround"
      ]
    }
  }
}

• Choice: one of 2-255 named options, plus probabilities and confidence.

• Noul: probability of yes, from 0 to 1. No separate confidence. Optional criteria describes the true and false cases.

• Score: a probability-weighted position from 0 to levels.length - 1, with 2-10 descriptive levels, probabilities, confidence, and a legend.

Instructions and criterion descriptions may also be JSON objects or arrays. Choice descriptions may be null when the option name is sufficient. Question IDs only identify results; Jev does not see them during inference. Questions run independently over the same state and cannot use each other's answers.

Boundaries

• Every invocation sends the supplied state and questions to https://api.typesafe.ai/v1/systemone and consumes TypeSafe API usage. It never automatically reads files or forwards session history. Only send data allowed by your project's sharing policy, never credentials.

• Jev supplies judgments, not research, generated explanations, proof, or permission to execute actions. Confidence measures distribution concentration, not truth. It cannot select a candidate you omitted. Include a no-match option where needed.

• Jev accepts text/JSON, not images or audio. English is currently its strongest language.

• The server uses native fetch with a 30-second deadline and honours MCP cancellation. Errors are explicit, with no automatic retry and no fallback model. Wait before retrying rate-limit or overload errors. Error bodies are never echoed into the transcript, because they can contain submitted evidence.

• TypeSafe documents 64k tokens per request and 32k for state plus the longest question for Jev 1.13. The service enforces the budget; this server does not approximate it.

• TypeSafe billing is separate from your agent's own model billing. Calls and results are retained in normal session history.

• Output above 2000 lines or 50 KB is truncated, with the full response written to a private temporary file (mode 0600). That file can contain sensitive evidence echoed back in rubric descriptions; remove it when no longer needed.

Verify

pnpm install
pnpm test

Tests use simulated HTTP responses and need no API key. They cover the advertised JSON Schema, credential handling, request batching, per-question answer validation, HTTP and network failures, cancellation, output truncation, and the stdio entry point with tools/list and tools/call routing. They do not measure Jev's accuracy or prove your account can reach the live service.

Protocol smoke test, also without a key:

printf '%s\n%s\n%s\n' \
 '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-06-18","capabilities":{},"clientInfo":{"name":"smoke","version":"0"}}}' \
 '{"jsonrpc":"2.0","method":"notifications/initialized"}' \
 '{"jsonrpc":"2.0","id":2,"method":"tools/list"}' | node mcp/server.mjs

For a live test after configuring the key, ask your agent:

Call judge with state "The sky is blue" and a noul question asking whether the text mentions a colour. Show the probability and the model the API returned.

Release

CI runs the suite on every push and pull request. Publishing is a tag:

# bump "version" in package.json, commit, then
git tag v0.5.1 && git push origin v0.5.1

.github/workflows/publish.yml re-runs the tests, refuses a tag that disagrees with package.json, and publishes to npm through trusted publishing, so there is no NPM_TOKEN secret and provenance is attached automatically. Configure the trusted publisher once under the package's settings on npmjs.com, pointing it at this repository and publish.yml.

References

HTTP API · Question types · Confidence