GenericJevMCP via DiffusionGemma

日本語 · Validation · Provenance

Structured yes/no, choice, and score decisions through MCP, a terminal, or HTTP. A CPU adapter reads bounded answer-token slots from a shared DiffusionGemma NVFP4 backend, returning candidate probabilities and measured elapsed time without generating an essay for each decision.

This is an experimental, self-hosted implementation of Jev-style decisions. The internal API name dg-bert is historical: no BERT model is loaded. The calling assistant is independent of the decision backend.

Architecture

MCP host / terminal → Python client → optional SSH → CPU adapter :8011
                                                        ↓
                                          DiffusionGemma / vLLM :8010

Text, optional evidence, or one image can be evaluated with multiple questions in a joint read or with isolated sequential questions. Ordinary generation on port 8010 and decisions share the same loaded weights. Decisions use one diffusion step per read; ordinary generation uses the denoising schedule.

Related MCP server: mcp-confidence

Repository layout

.
├── jev/             Python API, decision logic, CLI, calibration
├── mcp/             MCP stdio server
├── scripts/         Model preparation, service lifecycle, monitoring
├── tests/           CPU and MCP tests
│   └── live/        Opt-in checks against a running GPU service
├── runtime/         Pinned vLLM overlay and build-time checks
├── examples/        Request examples
├── skills/          Optional assistant skill
└── docs/            Validation scope

Run Python entry points from the repository root: python -m jev.client, python -m jev.calibration, and python scripts/service.py. MCP sets its Python working directory to this root. Local .env, client-config.json, state/ and corpus/ remain at the root and stay untracked.

Requirements

Provision storage for approximately 19 GB of model files, Docker layers, and runtime caches. Weights, credentials and private corpora are not distributed. Other GPU types, x86 hosts and upstream runtime versions are outside the tested deployment.

GPU server setup

Run on the Linux GPU host. Review the model card and model terms before downloading. The code license does not replace model terms.

git clone https://github.com/Bizuayeu/GenericJevMCP-via-DiffusionGemma.git
cd GenericJevMCP-via-DiffusionGemma
python3 -m venv .venv
. .venv/bin/activate
pip install -r requirements.txt

python scripts/prepare_model.py --download
# Or verify an existing snapshot:
# python scripts/prepare_model.py --snapshot /absolute/cache/path/snapshots/REVISION
python scripts/service.py init --download-status state/download-status.json

docker build -t generic-jev:local .
export JEV_IMAGE=$(docker image inspect generic-jev:local --format '{{.Id}}')
python scripts/service.py start-backend
# Wait until ready; first compilation can take longer than warm inference.
curl --fail http://127.0.0.1:8010/health
python scripts/service.py start-adapter
curl --fail http://127.0.0.1:8011/health

The preparation command downloads the pinned revision under ~/.cache/huggingface and verifies filenames/sizes. Initialization generates local credentials with mode 0600 without replacing an existing .env. No corpus is required.

Keep JEV_IMAGE set to your built image ID for service operations and monitoring. The source default is the original tested image ID, not an image already present on your machine. The build verifies the pinned base and overlay hashes.

Optionally run python scripts/monitor.py in another terminal with the same JEV_IMAGE. It stops matching managed GPU containers when host memory falls below the configured reserve; it is not an all-allocation OOM guarantee. Neither services nor monitor are registered for OS autostart. Stop with python scripts/service.py stop-adapter / stop-backend. When settings change, stop and retain/rename the old container before recreating it: the service refuses silent configuration replacement.

Three decision types

python -m jev.client decide --question 'Does water contain hydrogen?' --format text
python -m jev.client decide --question 'Capital of Japan?' --choices Tokyo Osaka Kyoto --format text
python -m jev.client decide --request examples/three-types.json

For score probabilities [0.1, 0.2, 0.7], the result is 0×0.1 + 1×0.2 + 2×0.7 = 1.6. The legend maps indices to labels. For every type, confidence is the largest candidate probability; when the answer is no, noul and confidence differ.

{
  "state": "The package contains three red balls.",
  "questions": {
    "contains_red": {"type": "noul", "instructions": "Does it contain a red ball?"},
    "color": {"type": "choice", "instructions": "What color?", "criteria": {"red": null, "blue": null}},
    "explicitness": {"type": "score", "instructions": "How explicitly is the color stated?", "criteria": ["not stated", "implied", "explicit"]}
  }
}

State is optional: model knowledge is enabled by default. Add --state 'text', repeatable --state-file file.md, or --image photo.png. Files are read by the client and images are encoded as data URLs. A path/URL inside state is not fetched. Existing state and state-file cannot be combined. Failed reads and oversized inputs are errors, never silent truncation.

For inline JSON use --request-json '…' or UTF-8 stdin with --request-json -. PowerShell stdin preserves quotes:

$OutputEncoding = [System.Text.UTF8Encoding]::new($false)
@'
{"questions":{"answer":{"type":"noul","instructions":"Does water contain hydrogen?"}}}
'@ | python -X utf8 -m jev.client decide --request-json - --format text

Remote clients

Clone the source on the client too. Use your SSH alias and the actual server checkout path:

python -m jev.client --ssh-config /path/to/ssh_config --ssh-host spark --remote-root /opt/GenericJevMCP decide --question 'Capital of Japan?' --choices Tokyo Osaka Kyoto

Alternatively set JEV_SSH_CONFIG, JEV_SSH_HOST, JEV_REMOTE_ROOT, or put ssh_config, ssh_host, remote_root in an ignored client-config.json in the repository root. CLI flags override environment variables, which override the file. Without an SSH config, calls are local. API keys remain on the server.

MCP setup

Run npm ci in the checkout on the MCP host. Configure your host with absolute paths:

{
  "mcpServers": {
    "jev": {
      "command": "node",
      "args": ["/absolute/path/GenericJevMCP-via-DiffusionGemma/mcp/server.mjs"],
      "env": {
        "JEV_PYTHON": "/absolute/path/to/python",
        "JEV_SSH_CONFIG": "/absolute/path/to/ssh_config",
        "JEV_SSH_HOST": "spark",
        "JEV_REMOTE_ROOT": "/opt/GenericJevMCP"
      },
      "timeoutSeconds": 240
    }
  }
}

Omit SSH variables on the GPU host. Host formats vary; timeoutSeconds is the tested Antigravity setting, allowing for the client's 200-second SSH timeout. The bridge starts the fixed Python client with shell:false. Tool callers cannot select an executable or change the SSH destination.

Call decide with:

{"request":{"questions":{"answer":{"type":"noul","instructions":"Does water contain hydrogen?"}}}}

Optional top-level arguments are image (one local path) and state_files (UTF-8 paths). The MCP host reads them and transmits their content to the configured GPU server.

For Antigravity, register this server, copy skills/jev to ~/.gemini/antigravity-cli/skills/jev, and restart. Use /jev [AdditionalInput] <OutputCategory>. To auto-allow only this tool, add mcp(jev/decide) to permissions.allow in the host settings. No all-command permission is required. The tool description carries probability semantics instead of repeating a disclaimer in each result.

Probability calibration and abstention

Probabilities are uncalibrated and normalized within the supplied candidates, not probabilities of factual correctness. A missing best answer can still yield a confident selection. Diagnostics expose label mass, label entropy, whether the vocabulary argmax is a permitted label, and read count. If every read's argmax falls outside the labels, the answer is null.

With sources_only:true, an extra evidence-sufficiency decision nulls unsupported answers. That gate is itself a model judgment. Explicit retrieval with no match abstains without inference.

jev/calibration.py fits temperature on labeled examples and evaluates accuracy, NLL and ECE on disjoint context groups:

[{"group":"document-001","probabilities":[0.9,0.1],"correct":0}]

python -m jev.calibration --fit records/fit.json --evaluate records/evaluation.json

Use deployment-matched model, prompt, candidate order, mode and samples. Questions from one source share a group; overlapping fit/evaluation groups are rejected. Do not invent probabilities for abstentions. The upstream-derived grid is 0.2–4.0 in 0.05 steps; improvement on held-out data must be measured.

This tool is an offline evaluator, not automatic serving calibration. It does not modify the API, which retains calibrated:false. No real domain calibration dataset has been validated here; the numerical tests use synthetic distributions. This is distinct from NVFP4 quantization calibration.

Latency breakdown

Individual Windows→SSH→GB10 observations, 2026-09-20:

The MCP example's full assistant turn took 8.472 s. These are single observations, not percentiles or an optimization comparison. The slow file example does not isolate file-read cost. No-match retrieval has zero reads and can omit decision time.

Local HTTP probes: four short questions took 0.108 s joint/warm and 0.374 s separate/warm. Two-question image probes took 4.341 s on first shape use and 0.326–0.335 s warm. Compilation and concurrency matter. Evidence and limits.

One Spark-class machine: capacity and limits

Verified hardware: one GB10 system with 128 GB unified memory. These are deployed settings, not a maximum-capacity benchmark:

Startup admission estimate: 19 GiB weights + 6 KV + 10 sampler transient + 12 host reserve = 47 GiB available. This is not measured peak consumption. The PyTorch cap does not constrain every other library allocation.

Co-residency with a separate Gemma 26B NVFP4 backend and mixed generation/decision calls were tested. Host available memory was about 43 GiB after later vision validation, not a measurement of every startup peak. Maximum-context quality, saturation throughput and long-duration stability are unverified. The 16-question/26-candidate limits are also bounded by the answer canvas and 128 unique label-token IDs: not every maximal combination is valid.

Retrieval, modes, and HTTP

The generic path needs no corpus. jev/corpus.py is a site-specific HTTrack importer, not a universal document importer. No source archive is distributed. An optional corpus/surei.jsonl can contain your own entries with id, title and text plus optional provenance. Query retrieval uses Japanese character-bigram BM25; number lookup retains the original 1–91 convention. Restart the adapter after corpus changes.

POST /v1/systemone on loopback port 8011 requires Bearer authentication. The /v1/chat/completions wrapper accepts two messages: system = schema JSON, user = state JSON. Free-form generation uses port 8010. Use SSH forwarding for remote HTTP access.

Mode joint is the default; separate processes independent question inputs sequentially with a stable seed and per-question adaptive reads. Samples auto uses four reads if initial label entropy exceeds 0.1 nats or argmax lies outside labels. Fixed samples: 1–32. Isolation and seed do not guarantee bit-identical GPU results.

Tests

pip install -r requirements.txt
# Download only the pinned tokenizer, not model shards.
python -c "import json, shutil; from huggingface_hub import hf_hub_download; m=json.load(open('models.lock.json'))['diffusion']; shutil.copyfile(hf_hub_download(m['model'],'tokenizer.json',revision=m['revision']), 'tests/tokenizer.json')"
python -m unittest discover -s tests -v
npm ci
npm test

Runtime overlay checks run during Docker build. Live scripts tests/live/verify_stack.py and tests/live/verify_rag.py retain companion-model/private-corpus assumptions; they are not fresh-install acceptance tests. CPU tests and generic examples need no private corpus. See validation for the actual published verification scope.

With a running configured GPU service, npm run smoke exercises MCP initialize, tool discovery and all three decision types. It does not start the GPU backend.

License and provenance

Code: Apache-2.0. NOTICE.md credits the pinned vLLM forks, mmastrac/djev-spark, and open-alternative-jev. Model weights have their own terms, including the Gemma terms referenced by the NVIDIA model card. This is not an official Google or NVIDIA product.