decision-lite

中文文档

A Jev-compatible decision layer, as an MCP server — backed by a pluggable model instead of TypeSafe's proprietary one.

TypeSafe's Jev is a "System One" model: it answers typed questions (noul / choice / score) with probability distributions and confidence, in ~100ms, for routing / escalation / gating. It is closed-source, waitlist-only, and priced per call. decision-lite reproduces the public contract and the workflow — not the model.

Backends

Related MCP server: relaycore

What it does / does not do

Contract

// request
{
  "state": "customer email text or a JSON object",
  "questions": {
    "is_urgent":  { "type": "noul",   "instructions": "needs reply within 1h?",
                    "criteria": { "true": "yes", "false": "no" } },
    "department": { "type": "choice", "instructions": "route to which team",
                    "criteria": { "billing": "invoices/refunds", "technical": "bugs/how-to" } },
    "risk_level": { "type": "score",  "instructions": "customer anger level",
                    "criteria": ["calm", "frustrated", "very angry"] }
  }
}

// response
{
  "model": "glm-5.3-flash",
  "mode": "verbalized",
  "decision_id": "…uuid…",
  "answers": {
    "is_urgent":  { "type": "noul", "noul": 0.95,
                    "confidence": 0.9, "probabilities": { "true": 0.95, "false": 0.05 } },
    "department": { "type": "choice", "choice": "billing", "confidence": 0.74,
                    "probabilities": { "billing": 0.87, "technical": 0.13 } },
    "risk_level": { "type": "score", "score": 1.04, "confidence": 0.92,
                    "legend": { "0": "calm", "1": "frustrated", "2": "very angry" },
                    "probabilities": { "0": 0, "1": 0.96, "2": 0.04 } }
  },
  "usage": { "prompt_tokens": 412, "completion_tokens": 68 }
}

Confidence is a margin, not a correctness probability — same semantics as Jev: noul → |p − 0.5| × 2; choice/score → top-1 minus top-2 probability. A confident wrong answer is still wrong; that's why record_outcome exists.

Modes

needle backend (default) — free, local, calibrated head

Each question is posed to a persistent cactus-needle bridge process as a single-tool call; Needle's byte-level grammar constrains the answer argument to the enum, and its learned head returns confidence. One ~121M model in ~30MB RAM, zero API cost.

Caveats discovered by probing (worth knowing):

• Judgment calls land in suppressed_calls: Needle's grounding gate withholds calls whose arguments aren't literal text spans — i.e. nearly every noul/choice/score judgment. decision-lite still uses the suppressed answer (flagged meta.suppressed: true); the reported confidence is the real head score.

• Distribution is synthesized: Needle returns top-answer + scalar confidence, not a distribution. decision-lite sets P(chosen) = max(c, 1/n) and spreads the rest — enough for ranking and gating, not a true posterior.

• Non-English input: Cactus measured correct non-English calls at confidence 0.0. For any non-ASCII state or question, Needle answers include meta.confidence_unreliable: true; the confidence is still returned but should not be trusted. For Chinese states prefer DECISION_BACKEND=openai.

• First run downloads the engine from Hugging Face (~30MB). Set HF_ENDPOINT=https://hf-mirror.com if huggingface.co is unreachable. DECISION_NEEDLE_GENERATION=2 selects the older 45M model.

laya backend — trained decision model, multilingual

DECISION_BACKEND=laya spawns python/laya_bridge.py (same JSONL pattern as the needle bridge) wrapping laya's Router. Unlike every other backend, laya returns a real per-option probability distribution — a ModernBERT encoder places a [MASK] per option and scores them all in one forward pass — plus answer_confidence (max-p, the temperature-scaled quantity laya calibrates against) alongside an entropy-based confidence. The multilingual checkpoint is selected automatically for non-Latin script (e.g. Chinese states), so unlike needle there is no confidence_unreliable caveat for CJK. Requires pip install laya (pulls torch; ~2.5GB) and a ~643MB checkpoint download on first use. DECISION_LAYA_MODEL=english|multilingual pins a checkpoint instead of script-based routing.

When the runtime selects the laya backend it asks the bridge to preload checkpoints in the background — Router.preload() on every registered checkpoint, or just the pinned one — so the first evaluate does not pay a cold ~10s-per-checkpoint build. This spawns the python bridge at server start even if evaluate is never called; set DECISION_LAYA_WARMUP=0 to keep lazy loading. Warmup failure is non-fatal (a stderr note, then lazy loading as before).

openai backend — verbalized / sampling / logprobs

• verbalized (default): one call per evaluate — model emits JSON probabilities for all questions. Probabilities are the model's verbalized opinion — systematically overconfident on weak models.

• sampling: DECISION_SAMPLES (default 7) independent votes per question at DECISION_TEMPERATURE (default 0.8); distribution = observed vote frequency. Invalid samples dropped and reported (samples.requested/valid/invalid). Less biased than verbalized, still not calibrated — a confidently-wrong model votes wrong every time.

• logprobs: one low-temperature call per question requests logprobs=true and top_logprobs=20. Choices are mapped to one-token codes (A–T); returned token log-probabilities are exponentiated and normalized over the valid options, avoiding multi-token option-label bias. This is a genuine next-token distribution over the codes, not calibrated correctness probability. It requires an endpoint that returns chat-completion logprobs; Ollama 0.32.9 was verified to do so. If there are more than 20 options or the endpoint omits any option code from its top candidates, decision-lite falls back to sampling and reports meta.distribution: "sampling_fallback". HTTP errors (including unsupported request fields) are surfaced rather than hidden. Qwen3 model names automatically send reasoning_effort: "none" so thinking tokens do not consume the short answer budget.

Setup

pip install cactus-needle   # default backend (Python >=3.9)
npm install
npm start                   # stdio MCP server

中文：使用 Ollama 本地 Qwen3

ollama pull qwen3:0.6b
$env:DECISION_BACKEND = "openai"
$env:DECISION_BASE_URL = "http://127.0.0.1:11434/v1"
$env:DECISION_MODEL = "qwen3:0.6b"
$env:DECISION_MODE = "logprobs"  # also supports verbalized or sampling
npm start

Needle 的非英文 confidence 不可靠；中文输入建议使用上面的 OpenAI-compatible 后端。logprobs 一般比多次 sampling 省调用，并避免让小模型自报概率，但只适用于确实返回 top_logprobs 的本地服务；需要兼容时可改用 verbalized，或用 sampling 观察多次回答的一致性。

Multi-platform MCP and plugin install

Build the portable package, inspect its target catalog, then install only the clients you want:

npm run build
node src/cli.mjs list-targets
node src/cli.mjs install --target claude,cursor --scope project --dry-run
node src/cli.mjs install --target claude --scope project
node src/cli.mjs uninstall --target claude --scope project

The installer exposes 22 targets: native MCP configs (Claude Code, Codex, OpenCode, Qwen Code, Reasonix, Kilo, WorkBuddy, Devin); skill targets (Trae, pi, OMP, DSH); and Agent Plugin targets (Copilot, Cursor, Kiro, OpenClaw, Hermes, VS Code, ChatGPT/Codex, Grok, NanoClaw, and other compatible clients). Use --target all --dry-run to preview every adapter. JSON/TOML configuration edits are key-level merges with backups; unparseable or conflicting user entries are left untouched and reported as manual.

OpenClaw and DSH also have native plugin exports, built from the same runtime as the stdio MCP server. Some GUI/closed clients only support manual registration; the adapter reports that rather than guessing. The package is not yet published on npm (registry lookup currently returns 404): local installs point at the built runtime under .decision-lite/plugin; generated mcp.json uses npx -y decision-lite mcp for use after publication. The installer never stores DECISION_API_KEY; add it directly to the host's MCP environment when needed. uninstall --purge-config requires --target all and removes only decision-lite-marked package files if all adapters report clean removal.

Config is env vars:

MCP client config

Any stdio-capable MCP client registers the same way:

// once published on npm
"decision-lite": {
  "command": "npx",
  "args": ["-y", "decision-lite", "mcp"]
}

// from a source checkout — and to pick a backend/model:
"decision-lite": {
  "command": "node",
  "args": ["<path-to-repo>/src/index.mjs"],
  "env": {
    "DECISION_BACKEND": "openai",
    "DECISION_BASE_URL": "https://open.bigmodel.cn/api/paas/v4",
    "DECISION_API_KEY": "your-key",
    "DECISION_MODEL": "glm-5.3-flash",
    "DECISION_MODE": "verbalized"
  }
}

Condition-driven question inputs

An optional conditions v1 block makes the policy explicit while preserving the existing Jev answer shape:

{
  "type": "noul",
  "instructions": "Should this be prioritized?",
  "criteria": { "true": "priority required", "false": "normal queue" },
  "conditions": {
    "version": 1,
    "facts": {
      "deadline": {
        "type": "choice",
        "instructions": "Classify the explicitly stated deadline.",
        "criteria": { "within_hour": "within one hour", "later": "later or unstated" }
      }
    },
    "rules": [
      { "id": "near-deadline", "when": [{ "fact": "deadline", "op": "eq", "value": "within_hour" }], "then": "true" }
    ],
    "default": "false"
  }
}

Facts reuse noul, choice, and score. Rules are OR by list, AND within each when array, and first-match wins. Supported operators are eq, neq, in, not_in; numeric comparisons apply to score facts. The default is mandatory and must be a valid output option. This version standardizes the policy sent to the model; it does not locally execute rules or change how confidence/probabilities are produced. No urgency or risk thresholds are built in.

MCP tools

Condition input v1

To express which facts should lead to an outcome, add an optional conditions table to a question. It reuses Jev question types to define facts, then maps fact combinations to one of that question's existing output options:

{
  "type": "noul",
  "instructions": "Should this be prioritized?",
  "criteria": { "true": "priority required", "false": "normal queue" },
  "conditions": {
    "version": 1,
    "facts": {
      "deadline": {
        "type": "choice",
        "instructions": "Classify the explicitly stated deadline.",
        "criteria": { "within_hour": "explicitly within one hour", "later": "later or no deadline" }
      },
      "active_harm": {
        "type": "noul",
        "instructions": "Is financial, account-security, health, or physical harm happening now?",
        "criteria": { "true": "active harm", "false": "no active harm" }
      }
    },
    "rules": [
      { "id": "near-deadline", "when": [{ "fact": "deadline", "op": "eq", "value": "within_hour" }], "then": "true" },
      { "id": "active-harm", "when": [{ "fact": "active_harm", "op": "eq", "value": "true" }], "then": "true" }
    ],
    "default": "false"
  }
}

Rules are evaluated in array order; every clause in a rule must match, and the first matching rule wins. Multiple rules express OR. Supported operators are eq, neq, in, not_in, and lt/lte/gt/gte for score facts. default must be one of the question's output options. The configured backend extracts the typed facts, then Decision Lite applies the rules locally. Output probabilities are composed from fact distributions assuming independence; meta.conditions records the fact distributions, rule masses, and default mass. If a fact is missing, uniform, or marked confidence-unreliable, the declared default is used. Fact-option combinations are capped at 4096. This makes policy application explicit, but does not calibrate the model's fact extraction or the resulting probabilities. No business thresholds are built in, define them per domain.

Calibration loop: call evaluate → get decision_id → when ground truth is known, record_outcome → decision_stats shows whether "confidence 0.9" answers are actually right ~90% of the time. If the gap is large, raise thresholds or switch backend/mode.

eval harness

node src/cli.mjs eval --dataset fixtures/eval_zh_triage.json runs a labeled fixture through the configured backend and reports accuracy, expected calibration error, mean confidence, and latency (mean/p50/p95), plus per-question accuracy. The report carries a dataset hash so two runs are comparable. Baseline on the 10-example Chinese triage fixture with qwen3:0.6b + logprobs: accuracy 0.43, ECE 0.50, mean confidence 0.94 — a small hand-built set, not a benchmark; use it to compare backends/prompts, not to claim production accuracy.

Known limits

• Confidence ≠ correctness. No backend here is guaranteed calibrated; only logged outcomes tell the truth.

• Needle's confidence is English-calibrated — Cactus measured correct non-English calls at 0.0. For Chinese states use the openai backend or ignore the score.

• Needle is a judgment tool, not a judge of prose — it picks options reliably but won't explain why. 121M models don't reason; don't ask it multi-factor tradeoffs.

• Synthesized distribution: needle mode reports one answer + scalar confidence; the probability spread is constructed, not measured.

• Sampling mode still isn't calibrated — it reflects model self-consistency, not real-world accuracy.

• Small/local models may emit garbage that fails parsing; invalid samples are counted and dropped (sampling) or trigger per-question fallback (verbalized).

Repo layout

src/contract.mjs        question validation, distribution normalization, Jev-shaped answers, margin confidence
src/backend.mjs         OpenAI-compatible client; verbalized, sampling, and logprobs modes
src/needle-backend.mjs  Needle 3 backend — spawns python bridge, JSONL protocol
src/laya-backend.mjs    Laya backend — trained decision model, real distributions
src/eval.mjs            eval harness: accuracy/ECE/latency over a JSON fixture
src/runtime.mjs         shared handlers used by MCP, OpenClaw, and DSH
src/installer.mjs       CLI dispatch for 22 target adapters
src/targets.mjs         native MCP, skill, and Agent Plugin installers
python/needle_bridge.py cactus-needle wrapper (one agent per question)
python/laya_bridge.py   laya Router wrapper (auto-routes checkpoints by script)
src/log.mjs             append-only JSONL decision log + calibration stats
src/index.mjs           MCP stdio server
build.mjs               bundles CLI/native plugins and generates identity files

npm test runs the offline node:test suite, including fake-home installer dry-runs and config round trips. It builds the portable plugin package before testing. node fixtures/e2e_needle.mjs — real end-to-end with Needle 3 (downloads engine on first run).

Honest references

• TypeSafe Jev announcement: https://typesafe.ai/blog/introducing-system-one-models-and-jev

• Jev request/response contract: https://developers.cloudflare.com/ai/models/typesafe/jev/ , https://docs.typesafe.ai

• No official Jev paper or reproducible architecture is public as of writing. Third-party RLCD analyses exist but are reverse engineering, not spec.