decision-lite
Allows using a local Ollama server as an OpenAI-compatible decision backend, e.g., running Qwen3 models for Chinese-language states.
Provides a decision backend using any OpenAI-compatible chat endpoint, supporting verbalized, sampling, and logprobs modes to answer typed questions with probability distributions and confidence.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@decision-lite State: 'Refund missing, very upset.' Is it urgent, route to billing or technical, anger level?"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
decision-lite
A Jev-compatible decision layer, as an MCP server — backed by a pluggable model instead of TypeSafe's proprietary one.
TypeSafe's Jev is a "System One" model: it answers typed questions (noul / choice / score) with probability distributions and confidence, in ~100ms, for routing / escalation / gating. It is closed-source, waitlist-only, and priced per call. decision-lite reproduces the public contract and the workflow — not the model.
Backends
| engine | cost | confidence source |
| cactus-needle — Needle 3, 121M local model, ~30MB engine, fully offline after first download | free | learned calibrated head (real score, not margin) |
| Laya — non-autoregressive decision model; one forward pass scores all options, multilingual checkpoint reads CJK | free (local; ~643MB multilingual checkpoint, torch CPU) | real per-option distribution + temperature-scaled |
| any OpenAI-compatible chat endpoint — GLM-5.3-Flash (~¥0.5/M), DeepSeek, Ollama | provider-priced | verbalized opinion, vote frequency, or token logprobs |
Related MCP server: poordjaevin
What it does / does not do
✅ Same request/response contract as Jev |
|
✅ Pluggable backends | local Needle 3 by default; any OpenAI-compatible endpoint via |
✅ Three OpenAI modes |
|
✅ Calibration tracking | every decision is logged; attach ground truth later; |
❌ Not a reproduction of Jev's internals | Jev's architecture is unpublished. No public paper, weights, or training details exist. decision-lite does not claim otherwise. |
❌ Not calibrated out of the box | Needle's confidence is a learned head (English-calibrated; measured unreliable on non-English). OpenAI-mode probabilities are the model's opinion. Calibration is earned through logged outcomes — same as Jev, except Jev's refit is inside their black box. |
Contract
// request
{
"state": "customer email text or a JSON object",
"questions": {
"is_urgent": { "type": "noul", "instructions": "needs reply within 1h?",
"criteria": { "true": "yes", "false": "no" } },
"department": { "type": "choice", "instructions": "route to which team",
"criteria": { "billing": "invoices/refunds", "technical": "bugs/how-to" } },
"risk_level": { "type": "score", "instructions": "customer anger level",
"criteria": ["calm", "frustrated", "very angry"] }
}
}// response
{
"model": "glm-5.3-flash",
"mode": "verbalized",
"decision_id": "…uuid…",
"answers": {
"is_urgent": { "type": "noul", "noul": 0.95,
"confidence": 0.9, "probabilities": { "true": 0.95, "false": 0.05 } },
"department": { "type": "choice", "choice": "billing", "confidence": 0.74,
"probabilities": { "billing": 0.87, "technical": 0.13 } },
"risk_level": { "type": "score", "score": 1.04, "confidence": 0.92,
"legend": { "0": "calm", "1": "frustrated", "2": "very angry" },
"probabilities": { "0": 0, "1": 0.96, "2": 0.04 } }
},
"usage": { "prompt_tokens": 412, "completion_tokens": 68 }
}Confidence is a margin, not a correctness probability — same semantics as Jev:
noul → |p − 0.5| × 2; choice/score → top-1 minus top-2 probability. A confident wrong answer is still wrong; that's why record_outcome exists.
Modes
needle backend (default) — free, local, calibrated head
Each question is posed to a persistent cactus-needle bridge process as a single-tool call; Needle's byte-level grammar constrains the answer argument to the enum, and its learned head returns confidence. One ~121M model in ~30MB RAM, zero API cost.
Caveats discovered by probing (worth knowing):
Judgment calls land in
suppressed_calls: Needle's grounding gate withholds calls whose arguments aren't literal text spans — i.e. nearly every noul/choice/score judgment. decision-lite still uses the suppressed answer (flaggedmeta.suppressed: true); the reported confidence is the real head score.Distribution is synthesized: Needle returns top-answer + scalar confidence, not a distribution. decision-lite sets
P(chosen) = max(c, 1/n)and spreads the rest — enough for ranking and gating, not a true posterior.Non-English input: Cactus measured correct non-English calls at confidence 0.0. For any non-ASCII state or question, Needle answers include
meta.confidence_unreliable: true; the confidence is still returned but should not be trusted. For Chinese states preferDECISION_BACKEND=openai.First run downloads the engine from Hugging Face (~30MB). Set
HF_ENDPOINT=https://hf-mirror.comif huggingface.co is unreachable.DECISION_NEEDLE_GENERATION=2selects the older 45M model.
laya backend — trained decision model, multilingual
DECISION_BACKEND=laya spawns python/laya_bridge.py (same JSONL pattern as the needle bridge) wrapping laya's Router. Unlike every other backend, laya returns a real per-option probability distribution — a ModernBERT encoder places a [MASK] per option and scores them all in one forward pass — plus answer_confidence (max-p, the temperature-scaled quantity laya calibrates against) alongside an entropy-based confidence. The multilingual checkpoint is selected automatically for non-Latin script (e.g. Chinese states), so unlike needle there is no confidence_unreliable caveat for CJK. Requires pip install laya (pulls torch; ~2.5GB) and a ~643MB checkpoint download on first use. DECISION_LAYA_MODEL=english|multilingual pins a checkpoint instead of script-based routing.
When the runtime selects the laya backend it asks the bridge to preload checkpoints in the background — Router.preload() on every registered checkpoint, or just the pinned one — so the first evaluate does not pay a cold ~10s-per-checkpoint build. This spawns the python bridge at server start even if evaluate is never called; set DECISION_LAYA_WARMUP=0 to keep lazy loading. Warmup failure is non-fatal (a stderr note, then lazy loading as before).
openai backend — verbalized / sampling / logprobs
verbalized(default): one call perevaluate— model emits JSON probabilities for all questions. Probabilities are the model's verbalized opinion — systematically overconfident on weak models.sampling:DECISION_SAMPLES(default 7) independent votes per question atDECISION_TEMPERATURE(default 0.8); distribution = observed vote frequency. Invalid samples dropped and reported (samples.requested/valid/invalid). Less biased than verbalized, still not calibrated — a confidently-wrong model votes wrong every time.logprobs: one low-temperature call per question requestslogprobs=trueandtop_logprobs=20. Choices are mapped to one-token codes (A–T); returned token log-probabilities are exponentiated and normalized over the valid options, avoiding multi-token option-label bias. This is a genuine next-token distribution over the codes, not calibrated correctness probability. It requires an endpoint that returns chat-completion logprobs; Ollama 0.32.9 was verified to do so. If there are more than 20 options or the endpoint omits any option code from its top candidates, decision-lite falls back to sampling and reportsmeta.distribution: "sampling_fallback". HTTP errors (including unsupported request fields) are surfaced rather than hidden. Qwen3 model names automatically sendreasoning_effort: "none"so thinking tokens do not consume the short answer budget.
Setup
pip install cactus-needle # default backend (Python >=3.9)
npm install
npm start # stdio MCP serverLocal Qwen3 via Ollama
ollama pull qwen3:0.6b
$env:DECISION_BACKEND = "openai"
$env:DECISION_BASE_URL = "http://127.0.0.1:11434/v1"
$env:DECISION_MODEL = "qwen3:0.6b"
$env:DECISION_MODE = "logprobs" # also supports verbalized or sampling
npm startNeedle's confidence is unreliable on non-English input; for non-English states use the OpenAI-compatible backend above or laya. logprobs is cheaper than repeated sampling and avoids asking a small model to self-report probabilities, but only works with endpoints that actually return top_logprobs; fall back to verbalized for compatibility, or sampling to observe vote consistency.
Multi-platform MCP and plugin install
Build the portable package, inspect its target catalog, then install only the clients you want:
npm run build
node src/cli.mjs list-targets
node src/cli.mjs install --target claude,cursor --scope project --dry-run
node src/cli.mjs install --target claude --scope project
node src/cli.mjs uninstall --target claude --scope projectThe installer exposes 22 targets: native MCP configs (Claude Code, Codex, OpenCode, Qwen Code, Reasonix, Kilo, WorkBuddy, Devin); skill targets (Trae, pi, OMP, DSH); and Agent Plugin targets (Copilot, Cursor, Kiro, OpenClaw, Hermes, VS Code, ChatGPT/Codex, Grok, NanoClaw, and other compatible clients). Use --target all --dry-run to preview every adapter. JSON/TOML configuration edits are key-level merges with backups; unparseable or conflicting user entries are left untouched and reported as manual.
OpenClaw and DSH also have native plugin exports, built from the same runtime as the stdio MCP server. Some GUI/closed clients only support manual registration; the adapter reports that rather than guessing. The package is not yet published on npm (registry lookup currently returns 404): local installs point at the built runtime under .decision-lite/plugin; generated mcp.json uses npx -y decision-lite mcp for use after publication. The installer never stores DECISION_API_KEY; add it directly to the host's MCP environment when needed. uninstall --purge-config requires --target all and removes only decision-lite-marked package files if all adapters report clean removal.
Config is env vars:
var | default | meaning |
|
|
|
|
| python executable for the needle/laya bridge |
|
| needle model generation (3=121M, 2=45M) |
| auto | pin |
|
| preload laya checkpoints at server start ( |
|
| OpenAI-compatible endpoint (openai backend) |
| empty | bearer key (omit for local) |
|
| model name (openai backend) |
|
|
|
|
| votes per question in sampling mode |
|
| sampling temperature |
|
| append-only decision log |
MCP client config
Any stdio-capable MCP client registers the same way:
// once published on npm
"decision-lite": {
"command": "npx",
"args": ["-y", "decision-lite", "mcp"]
}
// from a source checkout — and to pick a backend/model:
"decision-lite": {
"command": "node",
"args": ["<path-to-repo>/src/index.mjs"],
"env": {
"DECISION_BACKEND": "openai",
"DECISION_BASE_URL": "https://open.bigmodel.cn/api/paas/v4",
"DECISION_API_KEY": "your-key",
"DECISION_MODEL": "glm-5.3-flash",
"DECISION_MODE": "verbalized"
}
}Condition-driven question inputs
An optional conditions v1 block makes the policy explicit while preserving the existing Jev answer shape:
{
"type": "noul",
"instructions": "Should this be prioritized?",
"criteria": { "true": "priority required", "false": "normal queue" },
"conditions": {
"version": 1,
"facts": {
"deadline": {
"type": "choice",
"instructions": "Classify the explicitly stated deadline.",
"criteria": { "within_hour": "within one hour", "later": "later or unstated" }
}
},
"rules": [
{ "id": "near-deadline", "when": [{ "fact": "deadline", "op": "eq", "value": "within_hour" }], "then": "true" }
],
"default": "false"
}
}Facts reuse noul, choice, and score. Rules are OR by list, AND within each when array, and first-match wins. Supported operators are eq, neq, in, not_in; numeric comparisons apply to score facts. The default is mandatory and must be a valid output option. This version standardizes the policy sent to the model; it does not locally execute rules or change how confidence/probabilities are produced. No urgency or risk thresholds are built in.
MCP tools
tool | input | output |
|
| Jev-shaped |
|
|
|
|
| confidence buckets vs observed accuracy |
Condition input v1
To express which facts should lead to an outcome, add an optional conditions table to a question. It reuses Jev question types to define facts, then maps fact combinations to one of that question's existing output options:
{
"type": "noul",
"instructions": "Should this be prioritized?",
"criteria": { "true": "priority required", "false": "normal queue" },
"conditions": {
"version": 1,
"facts": {
"deadline": {
"type": "choice",
"instructions": "Classify the explicitly stated deadline.",
"criteria": { "within_hour": "explicitly within one hour", "later": "later or no deadline" }
},
"active_harm": {
"type": "noul",
"instructions": "Is financial, account-security, health, or physical harm happening now?",
"criteria": { "true": "active harm", "false": "no active harm" }
}
},
"rules": [
{ "id": "near-deadline", "when": [{ "fact": "deadline", "op": "eq", "value": "within_hour" }], "then": "true" },
{ "id": "active-harm", "when": [{ "fact": "active_harm", "op": "eq", "value": "true" }], "then": "true" }
],
"default": "false"
}
}Rules are evaluated in array order; every clause in a rule must match, and the first matching rule wins. Multiple rules express OR. Supported operators are eq, neq, in, not_in, and lt/lte/gt/gte for score facts. default must be one of the question's output options. The configured backend extracts the typed facts, then Decision Lite applies the rules locally. Output probabilities are composed from fact distributions assuming independence; meta.conditions records the fact distributions, rule masses, and default mass. If a fact is missing, uniform, or marked confidence-unreliable, the declared default is used. Fact-option combinations are capped at 4096. This makes policy application explicit, but does not calibrate the model's fact extraction or the resulting probabilities. No business thresholds are built in, define them per domain.
Calibration loop: call evaluate → get decision_id → when ground truth is known, record_outcome → decision_stats shows whether "confidence 0.9" answers are actually right ~90% of the time. If the gap is large, raise thresholds or switch backend/mode.
eval harness
node src/cli.mjs eval --dataset fixtures/eval_zh_triage.json runs a labeled fixture through the configured backend and reports accuracy, expected calibration error, mean confidence, and latency (mean/p50/p95), plus per-question accuracy. The report carries a dataset hash so two runs are comparable. Baseline on the 10-example Chinese triage fixture with qwen3:0.6b + logprobs: accuracy 0.43, ECE 0.50, mean confidence 0.94 — a small hand-built set, not a benchmark; use it to compare backends/prompts, not to claim production accuracy.
Known limits
Confidence ≠ correctness. No backend here is guaranteed calibrated; only logged outcomes tell the truth.
Needle's confidence is English-calibrated — Cactus measured correct non-English calls at 0.0. For Chinese states use the openai backend or ignore the score.
Needle is a judgment tool, not a judge of prose — it picks options reliably but won't explain why. 121M models don't reason; don't ask it multi-factor tradeoffs.
Synthesized distribution: needle mode reports one answer + scalar confidence; the probability spread is constructed, not measured.
Sampling mode still isn't calibrated — it reflects model self-consistency, not real-world accuracy.
Small/local models may emit garbage that fails parsing; invalid samples are counted and dropped (sampling) or trigger per-question fallback (verbalized).
Repo layout
src/contract.mjs question validation, distribution normalization, Jev-shaped answers, margin confidence
src/backend.mjs OpenAI-compatible client; verbalized, sampling, and logprobs modes
src/needle-backend.mjs Needle 3 backend — spawns python bridge, JSONL protocol
src/laya-backend.mjs Laya backend — trained decision model, real distributions
src/eval.mjs eval harness: accuracy/ECE/latency over a JSON fixture
src/runtime.mjs shared handlers used by MCP, OpenClaw, and DSH
src/installer.mjs CLI dispatch for 22 target adapters
src/targets.mjs native MCP, skill, and Agent Plugin installers
python/needle_bridge.py cactus-needle wrapper (one agent per question)
python/laya_bridge.py laya Router wrapper (auto-routes checkpoints by script)
src/log.mjs append-only JSONL decision log + calibration stats
src/index.mjs MCP stdio server
build.mjs bundles CLI/native plugins and generates identity filesnpm test runs the offline node:test suite, including fake-home installer dry-runs and config round trips. It builds the portable plugin package before testing. node fixtures/e2e_needle.mjs — real end-to-end with Needle 3 (downloads engine on first run); node fixtures/e2e_mcp.mjs — a full JSON-RPC smoke session against the built stdio server.
Honest references
TypeSafe Jev announcement: https://typesafe.ai/blog/introducing-system-one-models-and-jev
Jev request/response contract: https://developers.cloudflare.com/ai/models/typesafe/jev/ , https://docs.typesafe.ai
No official Jev paper or reproducible architecture is public as of writing. Third-party RLCD analyses exist but are reverse engineering, not spec.
License
Available Tools
3 toolsdecision_statsB
Calibration report: confidence buckets vs observed accuracy from recorded outcomes. Shows whether high-confidence answers are actually right that often.
Inputs: no named parameters.
evaluateA
Jev-compatible decision call. Send a state and typed questions (noul/choice/score); get back answers with probability distributions and confidence. Use for routing, classification, scoring, escalation gating — not for open-ended generation.
Inputs: state (required), questions (required).
record_outcomeB
Attach ground truth to a past evaluate() call for calibration tracking. outcome: boolean applied to all answers, or {questionId: boolean}.
Inputs: outcome (required), decision_id (required), note.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
TDQS
Scored across 3 tools
The three tools map cleanly to distinct phases: making a decision (evaluate), providing feedback (record_outcome), and inspecting calibration (decision_stats). There is no overlap in purpose and each has a clearly bounded role.
evaluate is a bare verb, record_outcome is verb_noun, and decision_stats is a noun phrase, so conventions are mixed. Each name is still readable and self-explanatory, but there is no single predictable pattern.
Three tools form a tight, complete decision-calibration loop with no filler. The count is well matched to the narrow stated purpose.
The evaluate/record_outcome/decision_stats cycle covers the core lifecycle of making, labeling, and auditing decisions. Minor gaps exist (e.g., no way to list or retrieve past evaluate calls), but essential workflows are covered.
Maintenance
Related MCP Connectors
MCP server providing access to the Scorecard API to evaluate and optimize LLM systems.
Hosted MCP server for LLM cost estimation, model comparison, and budget-aware routing.
MCP server for OnceAsk, the AI-native current-address layer for people and agents.
MCP server for building and testing AI agents with multi-model experimentation and insights.
Related MCP Servers
- AlicenseNot gradedqualityBmaintenanceEnables local typed decision-making over MCP Streamable HTTP, running Laya and Von models to turn arbitrary state into choice, score, and yes/no answers with probabilities.3Do What The F*ck You Want To Public
- AlicenseNot gradedqualityBmaintenanceExposes local MCP tools for typed, calibrated LLM decisions — gating risky actions, yes/no judging, classification, ordinal rating, and multi-question decisions in one pass. Runs on the model Devin's CLI already provides via ACP (or a fully offline local backend), so agents get calibrated confidence with no extra API key, no Ollama, and no waitlist.52 PyPI6MIT
- AlicenseAqualityBmaintenanceLets MCP clients ask typed questions — yes/no, choice, or score — about structured state, including images, and receive calibrated probabilities from System One decision models such as TypeSafe Jev and Cloudflare Clef. Supports TypeSafe, OpenRouter, and local providers selectable per request, with input validation and cost reporting.6219 npmMIT
- AlicenseCqualityCmaintenanceAn MCP server that records the decision and categorisation steps inside agent workflows — inputs, choice and outcome — then serves confident decisions without an LLM call, routing uncertain ones to a hosted model and only training a private probe when reports show it's warranted. It reports agreement, coverage, savings, drift and hosted spend, and includes a drop-in gateway so pipelines making separate LLM classification calls can be answered without code changes.26MIT