jev-mcp-python
# jev-mcp-python
[](https://github.com/PyModel/jev-mcp/actions/workflows/ci.yml)
`jev-mcp-python` is a Python MCP server that exposes TypeSafe's Jev model as ten judgment tools. Its wire behavior is frozen against a TypeScript reference server, 0.5.0, through recorded parity fixtures. Jev answers a typed question with a probability distribution. Policy in this server turns a validated answer into `auto`, `review`, or `escalate`.
Wire behavior is pinned to the TypeScript reference through recorded parity fixtures. Vocabulary is in `CONTEXT.md`. Decisions are in `docs/adr/`.
## Install
Python 3.12 or newer, and [uv](https://docs.astral.sh/uv/).
```sh
uvx --from git+https://github.com/PyModel/jev-mcp jev-mcp-python
```
From a checkout:
```sh
uv sync --all-extras
uv run jev-mcp-python
```
The default transport is stdio. Logs go to stderr. Settings come from the environment. The server does not read a `.env` file.
```sh
export TYPESAFE_API_KEY=...
export JEV_PROVIDER=typesafe
export JEV_MCP_MODEL=jev-1.13.0
```
`JEV_PROVIDER=auto` uses the first configured credential, in this order: typesafe, openrouter, cloudflare, compatible. The Vercel provider is not supported. HTTP is experimental (`JEV_MCP_TRANSPORT=streamable-http`, bound to `127.0.0.1:8000` unless you change the host and port).
## Tools
Published in the reference order:
- `jev_verify` checks each claim against the evidence you supply.
- `jev_screen` judges fetched text before an agent reads it.
- `jev_find` ranks candidates against a plain-language query.
- `jev_classify` assigns each item to one class from a shared catalog.
- `jev_decide` chooses among bounded alternatives, and can decline.
- `jev_rerank` scores every candidate's relevance and returns them sorted.
- `jev_compare` judges whether two passages state the same fact, contradict, or state different facts.
- `jev_extract` lets your regex find the matches, then asks Jev to pick among them.
- `jev_review` scores a proposed patch against the request.
- `jev_gate` reviews a patch and verifies completion claims in one call.
## Bench
Numbers below are copied from `evals/reports/bench150.md`. Accuracy is not measured: 0 items were labeled. Speed and time use the 43 pairs where both arms reached the model. 107 with-Jev runs never reached the local model. The report counts those as connection errors, and does not count them as Jev misses. Of the 43 runs that reached the model, 42 got a Jev answer (98%).
| measure | before (no Jev) | after (with Jev) |
|---|---|---|
| accuracy | not measured (0 labeled items) | not measured (0 labeled items) |
| speed (items/min) | 3.38 | 0.79 |
| time consumed (s) | 762.71 | 3266.66 |
| Jev connection | | 42/43 (98%) |
| pairs where both arms ran | 43 | 43 |
| pairs recorded | 150 | 150 |
Stop: all 150 pairs recorded.
The page with the same charts is `evals/reports/bench150.html`. Raw transcripts are not in this repo.
## Agent pilot
The P8 pilot ran on 2026-09-21 with Claude Code CLI 2.1.278, model `claude-sonnet-5`, effort medium. One run per task and arm. Tasks were `t1-bulk-boundary`, `t2-slugify`, and `t3-durations`. No B or C run called a Jev tool. The report says the table cannot tell the arms apart, and that the cost gap is run noise. Full table and per-run rows: `evals/reports/p8-pilot.md`.
| measure | A: no Jev | B: this server | C: TypeScript server 0.5.0 |
|---|---|---|---|
| runs (failed) | 3 (0) | 3 (0) | 3 (0) |
| correct tasks | 3/3 | 3/3 | 3/3 |
| cost per correct task | $0.0336 | $0.0362 | $0.0326 |
| total cost | $0.1008 | $0.1085 | $0.0977 |
| Jev tool calls | 0 | 0 | 0 |
| wall time, s (sum) | 30.9 | 33.7 | 30.6 |
All 9 runs finished. Total spent: $0.3070 of the $25.00 cap.
## Develop
```sh
uv sync --all-extras
make ci
```
`make ci` runs lint, typecheck, unit, property, policy coverage, contract, parity, security, build, and the smoke test. `make eval` is the offline scorer check. `make eval-live`, `make security-live`, and `JEV_AB_LIVE=1 make ab` call paid services and stay off CI.
Contribution notes are in `CONTRIBUTING.md`.
License: MIT.
TDQS
Scored across 10 tools
Each tool has a distinct verb and purpose: verify claims, screen for injection, find best match, classify, decide, rerank, compare, extract, review, and gate. Even similar tools like jev_find and jev_rerank are clearly differentiated (one picks a single best, the other orders all), and jev_review vs jev_gate are explicitly separated by the presence of claim verification.
All tools follow a consistent 'jev_' prefix with a lowercase descriptive verb (verify, screen, find, classify, decide, rerank, compare, extract, review, gate). The naming pattern is uniform and predictable, making it easy for an agent to infer function from the name.
10 tools is well within the ideal 3-15 range. Each tool covers a distinct capability within the Jev evaluation domain, and none feel redundant or unnecessary. The count is appropriately scoped for the server's purpose.
The tool set provides comprehensive coverage of evaluation tasks: claim verification, prompt-injection screening, semantic search, classification, decision-making, reranking, pairwise comparison, structured extraction, patch review, and gated review with claim verification. Cross-references between tools (e.g., using jev_gate for patches with claims, jev_review without) indicate a well-thought-out and complete surface with no obvious gaps for the stated domain.