umpirix
# OpenJev
[](https://github.com/caelum0x/openjev/stargazers)
**Typed decisions instead of prompt-and-parse: an open-source, self-hostable service that
turns text plus typed questions into calibrated `choice` / `score` / `noul` answers.**
Instead of prompting an LLM and parsing prose, you POST application state plus a set of
typed questions and get back typed judgments with probabilities that ordinary code can
compose. The default engine is a local sentence-embedding classifier (gte-small, int8 ONNX,
CPU-only) with a lexical TF-IDF fallback that needs no model at all, so it runs anywhere
Node runs. Accuracy and calibration (ECE) are measured on public datasets, not assumed.
This repository is the whole project in one place: the API server + web console, the
native engine, the SDKs, the agent skills, and the `otsone` model toolkit used to train
and serve our own model family. Licensed under **Apache-2.0** (the SDK, adapter and skills
subdirectories derived from TypeSafe AI's MIT SDKs keep their MIT licenses; see
[Licensing](#licensing)).
> **Status.** OpenJev is self-host only. There is no official hosted OpenJev service, and
> the SDK packages are not yet published to npm or PyPI: install them from this repository
> (see [SDKs](#sdks)). The `openjev.com` domain and the PyPI name `openjev` belong to
> unrelated projects.
## Already use the TypeSafe SDK? Change one line
The official `typesafe-sdk` (Python) and `@typesafe-ai/sdk` (JavaScript) packages work
against an OpenJev server unmodified. Point the base URL at your server and keep your code:
```bash
export TYPESAFE_BASE_URL=http://localhost:8080 # with or without a trailing /v1
export TYPESAFE_API_KEY=anything # the SDK requires a value; a self-hosted server ignores it
```
```python
from typesafe_sdk import Choice, TypeSafeClient
with TypeSafeClient() as client:
response = client.system_one(
state="I was charged twice for order A-104, please refund the duplicate.",
questions={"team": Choice(instructions="Which team?", criteria={"billing": None, "technical": None})},
)
print(response.choices["team"].choice, response.request_id)
```
Checked on 2026-11-11 with `typesafe-sdk` 0.7.5 and `@typesafe-ai/sdk` 0.7.0 against a local
server: `choice`, `score` and `noul` questions, several questions in one request, string and
object `state`, `models.list()`, request ids, error classes for a rejected request, and the
zstd-compressed request bodies both SDKs send for bodies of 1 KiB or more. The runs are
[`examples/official-sdk.py`](./examples/official-sdk.py) and
[`examples/official-sdk.mjs`](./examples/official-sdk.mjs).
Not supported: image assets (multipart requests get a 415) and `box` questions (400). An
unknown model name, including the SDK default `jev-latest`, is answered by the local
`system-one` engine, and the response's `model` field says so. Details:
[`/docs/typesafe-sdk`](./src/lib/content/docs.ts).
## 60-second quickstart
With Docker (API + console on http://localhost:3000, plus the optional laya-onnx sidecar):
```bash
git clone https://github.com/caelum0x/openjev && cd openjev
docker compose up --build
```
Or with Node.js 22+ (dev server on http://localhost:8080):
```bash
git clone https://github.com/caelum0x/openjev && cd openjev
npm install
npm run embed-download # optional: ~33 MB gte-small model; without it the lexical engine answers
npm run dev
```
Then ask it something (use port 3000 for Docker):
```bash
curl -s localhost:8080/api/v1/systemone -H 'content-type: application/json' -d '{
"model": "system-one",
"state": "I was charged twice for order A-104, please refund the duplicate",
"questions": {
"department": { "type": "choice", "instructions": "Which team should handle this?",
"criteria": { "billing": "payments, refunds", "technical": "bugs, outages", "sales": "pricing" } },
"urgency": { "type": "score", "instructions": "How urgent is this?",
"criteria": ["low", "medium", "high", "critical"] },
"is_refund": { "type": "noul", "instructions": "Is the customer asking for a refund?" }
}
}'
```
Every response says which engine answered (`engine.kind` is `embedding` or `lexical`).
Nothing is required to boot: no API key, no database (an in-memory PGlite is used unless
you set `DATABASE_URL`), no model download.
### No server: classify from the command line
`openjev classify` labels text with the engine straight from this checkout. No server, no
API key, no model download. It uses Node.js built-in type stripping (Node.js 22.6+):
```bash
git clone https://github.com/caelum0x/openjev && cd openjev && npm install
# one text per line on stdin
node scripts/cli.mjs classify \
"bug:crash error broken fails,feature:request add option,praise:love great thanks" \
< examples/feedback.txt
# CSV or JSONL in, CSV or JSONL out; rows under the threshold are left blank for review
node scripts/cli.mjs classify bug,feature,praise --file data.csv --column text --threshold 0.6 --out results.csv
node scripts/cli.mjs classify bug,feature,praise --file data.jsonl --field text --json
```
Each row gets a label, a probability and the engine that answered. Labels are a
comma-separated list; a description after a colon tells the engine when a label applies,
which matters most for the lexical engine. With the gte-small model installed
(`npm run embed-download`) the embedding engine answers instead. A row where the engine
cannot tell the labels apart is left blank and marked for review.
**Lexical-engine accuracy, measured, not claimed.** Without the embedding model, the
lexical engine's only signal for a bare label (`bug,feature,praise`, no `:description`) is
the label word itself plus a small built-in synonym lexicon
(`src/lib/jev/lexicon.ts`) — if a label's word isn't in that lexicon, texts that don't
literally contain the word collapse to a tie and get left blank for review, and texts
that happen to share a *different* label's vocabulary (e.g. a praise message that
mentions a fixed "crash") can be scored confidently wrong. `scripts/classify-eval.mjs`
measures this on a 65-row synthetic gold set (`examples/classify-eval.jsonl`, built for
this repo, not scraped) across three label sets (`bug,feature,praise` /
`positive,negative,neutral` / `spam,sales,support`), including rows tagged
`shared_vocab: true` for ones that deliberately reuse another label's cue words.
Run it: `npm run classify-eval`.
Before this pass: `feature` and `praise` had zero lexicon coverage and `positive` /
`negative` / `neutral` had none at all, so most rows either tied (positive → 32.3%
accuracy counting ties as wrong, 67.7% on the rows it actually committed to) or, when a
bug-shaped word like "crash" appeared anywhere, confidently mislabeled a praise or
feature row as a bug. Adding two small synonym groups to the existing lexicon (positive
sentiment + praise cues; feature-request cues) — no model, no training, no change to the
scoring math — moved the same suite to **49.2% accuracy counting ties as wrong, 71.1% on
decided rows**, with the `positive,negative,neutral` set (previously almost all ties) at
35% and the `shared_vocab: true` subset at 47.5%. `npm run gold` (the product-wide 61-case
regression suite) still passes 61/61 — the fix is additive new synonym entries, not a
change to any existing one.
**Real limit, not fixed here:** a lexical bag-of-words engine cannot fully resolve a
praise sentence that mentions "crash" ("thanks for fixing the crash") from a bug report —
that needs the embedding engine or richer per-call descriptions. `--threshold` trades
coverage for precision on the lexical engine: in this suite, `--threshold 0.55` raises
accuracy on the rows it still commits to from 71.1% to 83.8% by abstaining on 28/65 rows
instead of 20/65 (`npm run classify-eval` prints the full sweep). For labels the lexicon
doesn't know, a `name:description` (as in the stdin example above) still outperforms
both, because it gives the lexical engine real words to match on instead of relying on
the built-in lexicon.
## MCP server
Agents that speak the Model Context Protocol can call typed decisions as tools. The running
server exposes `POST /v1/mcp` (stateless Streamable HTTP) with four tools: `choice`, `score`,
`noul` and `decide` (several typed questions in one call). No API key is needed for a
self-hosted server.
```bash
# Claude Code, HTTP
claude mcp add --transport http openjev http://localhost:8080/v1/mcp
# Claude Code, stdio (forwards to the server at OPENJEV_BASE_URL)
claude mcp add openjev --env OPENJEV_BASE_URL=http://localhost:8080 -- node /path/to/openjev/scripts/cli.mjs mcp
```
For Cursor, add `{ "mcpServers": { "openjev": { "url": "http://localhost:8080/v1/mcp" } } }` to
`.cursor/mcp.json`. Tool reference and a curl check: `/integrations/mcp` in the console
(source: [`src/lib/content/integrations.ts`](./src/lib/content/integrations.ts)).
---
## The three model families
The same `POST /v1/systemone` contract runs on any of three families:
| Family | Models | Runs on | Needs |
|--------|--------|---------|-------|
| **OpenJev (ours)** | `system-one` (sentence-embedding engine; lexical TF-IDF fallback), `otsone-1`, `otsone-mini` | anywhere Node runs; `system-one` in-process on CPU; otsone via the ONNX (CPU) or MLX (Apple silicon) sidecar | for `system-one`, the 33 MB gte-small model (`npm run embed-download`; without it the lexical engine answers and says so); the otsone sidecar + locally trained weights for `otsone-*` |
| **Laya (open weights)** | `laya-onnx`, `laya-mlx` | local sidecar (ONNX on Linux/CPU, MLX on Apple silicon) | the Laya checkpoints from Hugging Face (Convai Innovations, Apache-2.0) |
| **Cloud** | `system-one-cloud` | any wire-compatible hosted System One v1 endpoint | `OPENJEV_API_KEY` + endpoint; degrades to local when unset |
Neural models health-gate: if their sidecar is absent, the request is answered by the
local engine and the response says so (`requested` vs `served_by`). An optional
escalation model (`openjev-grok`) routes hard decisions to OpenAI / xAI / Anthropic
when you supply a provider key.
### Measured accuracy of the default engine
Zero-shot (the engine sees only the class names), full test splits, from
[`src/lib/content/leaderboard.json`](src/lib/content/leaderboard.json) and rendered at
`/leaderboard`. To reproduce (the datasets are downloaded at run time into `.cache/`;
the embedding column needs the pinned gte-small model):
```bash
npm install
npm run embed-download
LEADERBOARD_PUBLIC=1 LEADERBOARD_PUBLIC_SAMPLE=all npm run leaderboard
```
| Test split | `system-one` embedding engine (gte-small) | lexical fallback |
|---|---|---|
| Banking77 test (3,080 rows, 77 intents): accuracy / ECE | 62.4% / 0.035 | 38.7% / 0.475 |
| CLINC150 plus test (4,500 in-scope rows, 150 intents): accuracy / ECE | 67.4% / 0.055 | 11.9% / 0.842 |
| CLINC150 plus test, 1,000 out-of-scope rows: AUROC of `ood` | 0.910 | 0.683 |
| same: out-of-scope flagged at the shipped threshold (in-scope false alarms) | 71.1% (9.5%) | 0.0% (0.0%) |
| same: out-of-scope flagged at a 5% false-alarm rate (threshold tuned on test) | 57.4% | 0.1% |
| AG News test (7,600 rows, 4 topics): accuracy / ECE | 75.0% / 0.053 | 35.7% / 0.425 |
The engine's constants were fitted on held-out data only (Banking77 train, CLINC150
validation, half of the in-repo gold suite); AG News was never used for fitting. On the
small in-repo gold suite, which the lexical engine was tuned on, the lexical engine is
ahead (85.6% vs 82.5% accuracy; 94.7% vs 61.3% of off-topic probe pairs flagged); treat
the gold suite as a regression check, not a benchmark.
See [`docs/COMPARISON.md`](./docs/COMPARISON.md) for how these numbers, and our otsone
checkpoints, stack up against TypeSafe Jev, the open-source Kev model and the OpenAI
Decisions API — with sources, and the gaps honestly flagged.
---
## The trust layer
What a System One model alone does not give you:
- **Out of scope and abstain.** Every answer carries an `ood` score (on the
embedding engine, a fitted detector over similarity to the options and their
centroid plus a Mahalanobis-style distance; on other backends, distance of the
state from the question's vocabulary, modulated by answer entropy), and
off-topic inputs lose their confidence instead of coming back at 0.99. Set
`abstain: true` (request or question) to get `abstained: true` and, for Choice,
the reserved `__none__` option.
- **`auto` cascade.** `model: "auto"` answers locally (otsone when its sidecar is
up, else System One) and escalates only low-confidence or out-of-scope
questions to an LLM: Groq (free tier), Ollama (local), or OpenAI/xAI/Anthropic
when keyed. The response's `escalation` block lists what escalated and why.
- **Calibration on your labels.** `POST /api/v1/calibrations` fits a temperature
or isotonic map per question from your labelled rows (CSV/JSONL upload in
`/calibrations`, or straight from an eval run). With 20+ labels per question a
stratified quarter is held out: the map is fitted on the rest and the
before/after accuracy, ECE and Brier are measured on labels it never saw.
Pass `calibration_id` to apply it. The evals page imports labelled CSV/JSONL
(validated against the question schema, with a preview) and has a reliability
diagram and a threshold picker.
- **Shadow mode.** `POST /api/v1/shadow` runs next to your production LLM and
logs agreement; `/shadow` shows agreement by confidence and projected savings.
Send `incumbent: { kind: "label", answers }` to log human ground truth instead.
- **Drift alerts.** Each calibration can have a drift monitor
(`/calibrations/<id>` or `POST /api/v1/drift`). Shadow traffic that used the
calibration is compared with its held-out numbers; accuracy drops, confidence
shifts, or confidence running ahead of accuracy POST a `calibration.drift`
event to your webhook, signed `X-OTS-Signature: t=<unix>,v1=<HMAC-SHA256 of
"<t>.<body>">`, and `calibration.drift_resolved` when it clears.
- **Public leaderboard.** `npm run leaderboard` scores every available backend on
the in-repo gold suite and out-of-scope probes and writes
`src/lib/content/leaderboard.json` (rendered at `/leaderboard`). With
`LEADERBOARD_PUBLIC=1` it also scores Banking77, CLINC150 (with its
out-of-scope queries) and AG News, downloaded at run time into `.cache/`.
- **Benchmark Submissions.** `/submit-model` lets a third party point a live
HTTP endpoint or a precomputed answers file at the same gold suite and
scoring engine the built-in models use (accuracy, strict pass rate, ECE,
adaptive ECE, MCE, Brier, NLL, per-question-type breakdown), auth- and
quota-checked through the existing API-key system and SSRF-guarded before
any submitted URL is called. Every run also carries a real **execution
trace** — steps, outbound/tool-call count, wall-clock latency p50/p95,
tokens and cost when the endpoint reports them, and error/timeout counts —
and an opt-in **fault-injection** stress test (deterministically seeded
timeouts/5xx/malformed responses at configurable rates) that reports
**robustness**: accuracy under faults vs. the same model's clean-run
accuracy. Completed runs show up as extra "Community submitted" rows (with
their trace + robustness columns) on `/leaderboard`'s gold-suite tab, next
to OpenJev's own models. This is the feature behind our Geek Room Global AI
Hackathon entry, **Umpirix — Benchmark Arena for AI Agents** — see
[`hackathons/geekroom-2026/HACKATHON-SUBMISSION.md`](hackathons/geekroom-2026/HACKATHON-SUBMISSION.md)
for the full writeup, the problem-statement fit ("a benchmarking platform
for AI agents, not just models"), and a recorded demo.
### How Bifrost powers Umpirix
Umpirix's agent-under-test and judge LLM calls run through
[**Bifrost**](https://github.com/maximhq/bifrost), Maxim AI's open-source LLM
gateway, instead of hitting a provider directly. One env switch flips every
LLM-backed call in this repo onto it — the public leaderboard's `llm` row
(`scripts/leaderboard.mjs`), the `auto` escalation cascade, and the judge
stage of the router→gate→judge agent loop (`src/lib/jev/loop.ts`) — all
funnel through the SAME choke point, `src/lib/jev/llm.server.ts`, so flipping
the switch benchmarks the real call every one of those features already
makes, not a separate demo path.
**Install & run Bifrost locally** (the documented quickstart):
```bash
npx -y @maximhq/bifrost -port 8099 # gateway + dashboard on :8099
```
Configure a provider with a file-based `config.json` in Bifrost's app dir
(`-app-dir`), referencing a key from YOUR OWN environment — never Umpirix's:
```json
{
"providers": {
"anthropic": {
"keys": [{ "name": "anthropic-primary", "value": "env.ANTHROPIC_API_KEY", "models": ["*"], "weight": 1.0 }]
}
},
"client": { "enforce_auth_on_inference": false, "enable_logging": true }
}
```
Point Umpirix at it:
```bash
export BENCH_LLM_GATEWAY=bifrost
export BIFROST_BASE_URL=http://localhost:8099
export LLM_PROVIDER=anthropic # or groq / openai / xai / ollama — Bifrost's
export ANTHROPIC_API_KEY=... # provider slugs match this app's LlmProvider ids 1:1
```
**What's real, verified against a live local gateway (2026-11-11), not guessed
from docs:**
- **Multi-provider routing, OpenAI-compatible.** Every call goes to
`POST ${BIFROST_BASE_URL}/v1/chat/completions` with
`{"model":"<provider>/<model>", ...}` — Bifrost translates to whichever
real backend (anthropic/groq/openai/xai/ollama/…) that provider needs.
- **Real server-side fallback.** Setting `BIFROST_FALLBACK_MODEL` passes it
as Bifrost's own `fallbacks` array. Verified live: pointing the primary at
a model that does not exist still returned a correct answer, with
Bifrost's response reporting `routing_info.is_fallback: true` and the real
model that actually served it — Bifrost's failover, not a client-side
retry this app writes.
- **Per-call telemetry straight from Bifrost's response**, not recomputed:
`extra_fields.latency`, `routing_info.provider`/`model`, and the real
token usage, read by `telemetryFromResponse` (`src/lib/jev/bifrost.server.ts`)
into every `SystemOneResponse.bifrost` and surfaced as a **Gateway**
column/badge on `/leaderboard`'s gold-suite table and in the
`/submit-model` execution trace.
- **A real 93-case gold-suite run through the gateway**: 93/93 real calls,
accuracy 0.9794, Bifrost-measured latency p50/p95 1567/2324 ms, 106,543
input + 7,350 output tokens — all read back from Bifrost's own
`extra_fields`, not this app's timer (which independently measured
1580/2327 ms — a cross-check, not a guess). Full numbers, plus a faulted
pass (67/93 real calls after timeout/server_error faults short-circuited
the rest, robustness 0.621) and the fallback demo above, are committed at
[`hackathons/geekroom-2026/bifrost-run-results.json`](hackathons/geekroom-2026/bifrost-run-results.json)
and reproducible with
[`hackathons/geekroom-2026/verify-bifrost.mjs`](hackathons/geekroom-2026/verify-bifrost.mjs).
- **Not yet wired to Bifrost's own `/api/logs`/rate-limit/governance
features** — those need a dashboard admin account (`BIFROST_SETUP_TOKEN`),
which this integration deliberately skips for a local/CI gateway; every
number above comes from the inference response itself, which needs no
extra auth. Honest gap, not hidden.
- **Signed audit reports.** `GET /v1/reports/:calibrationId` (`?format=html` for a
printable copy) bundles a calibration's maps, raw vs calibrated ECE / Brier, the
recommended threshold, drift history and shadow agreement, signed with
HMAC-SHA256; `POST /v1/reports/verify` checks a report was issued by the server
and not edited.
Set `OPENJEV_SITE_URL` to your deployment's public origin; without it the app
publishes no sitemap and asks crawlers not to index it.
## Layout
```
src/ API server + web console (TanStack Start / Vite)
routes/api/v1/ POST /v1/systemone, batch, models, keys, usage, evals, ...
lib/jev/ the typed-decision engine (TS), backends, validation, cookbooks
engine/ native engine: C++ kernel, Rust HTTP server, Python binding, gold suite
migrations/ SQL migrations (PGlite in-memory by default, Postgres optional)
scripts/ CLI (`openjev serve`, `openjev mcp`, `openjev classify`), Laya sidecar/download/distill helpers
otsone/ otsone model toolkit (Python): train / calibrate / export / serve
otsone-1 and run Laya on MLX or ONNX (formerly laya-mlx)
sdk/js/ `@openjev/sdk` — full TypeScript SDK
sdk/ai-sdk/ `@openjev/ai-sdk` — Vercel AI SDK integration
sdk/js-lite/ `@openjev/lite` — zero-dependency client + decision modules
sdk/python/ Python SDK (import `openjev`)
adapters/python/ `openjev-adapters` (import `openjev_adapters`) — run the same API on top of OpenAI/Anthropic/...
skills/ agent skills (Claude Code, Codex, ...) for building with typed decisions
examples/ runnable examples (see examples/README.md)
docs/ weights provenance, screenshots
```
---
## SDKs
The HTTP API is the primary interface; any language can call it. The SDKs below live in
this repository. Registry names are reserved/claimed and verified free as of 2026-11-09
(`@openjev/sdk` and `@openjev/lite` are unclaimed on npm; the plain `openjev` PyPI name
belongs to an unrelated project, as do the `openjev-sdk` and `pyjev` fallbacks, so the
Python package publishes as `openjev-client` instead) — publish to the real registries is
pending owner npm/PyPI account auth, so install from a clone until then:
| SDK | Directory | Install from source | Registry name (pending publish) |
|-----|-----------|---------------------|----------------------------------|
| TypeScript (full) | [`sdk/js`](./sdk/js) | `cd sdk/js && npm install && npm run build`, then `npm install /path/to/openjev/sdk/js` | `@openjev/sdk` |
| TypeScript (zero-dependency) | [`sdk/js-lite`](./sdk/js-lite) | `cd sdk/js-lite && npm install && npm run build`, then install by path | `@openjev/lite` |
| Vercel AI SDK integration | [`sdk/ai-sdk`](./sdk/ai-sdk) | build `sdk/js` first, then as above | (bundled with `@openjev/sdk`) |
| Python | [`sdk/python`](./sdk/python) | `pip install ./sdk/python` | `openjev-client` |
| Python provider adapters | [`adapters/python`](./adapters/python) | `pip install './adapters/python[openai]'` | (bundled with `openjev-client`) |
Do not `pip install openjev` from PyPI: that name belongs to an unrelated project. Once
published, install the Python SDK with `pip install openjev-client` (the importable module
stays `openjev`).
---
## Examples
Runnable scripts in [`examples/`](./examples) — rank ad-copy hooks (Node.js) or triage
support tickets (Python) by batching Choice + Score + Noul per item and applying a policy in
code over the returned probabilities, plus the official TypeSafe SDKs pointed at OpenJev
(`official-sdk.py`, `official-sdk.mjs`). See [`examples/README.md`](./examples/README.md).
---
## Self-hosting
Docker (Node service + laya-onnx sidecar):
```bash
cp .env.example .env # optional — nothing is required to boot
docker compose up --build # → http://localhost:3000
```
See [SELF-HOSTING.md](./SELF-HOSTING.md) for the non-Docker Node server, Postgres, LLM
escalation and every environment variable ([.env.example](./.env.example)).
### Serving otsone-1 locally
```bash
cd otsone
uv sync --extra onnx --extra train
uv run otsone distill --backbone otsone-base --out models/otsone-base # train + export
OTSONE_BACKEND=onnx OTSONE_MODEL_PATH=./models/otsone-base uv run otsone-serve
```
Then point the server at it with `OTSONE_MODEL_URL` (see `.env.example`). Model weights are
never committed — see [docs/WEIGHTS.md](./docs/WEIGHTS.md).
---
## Development
| Component | Test command |
|-----------|--------------|
| server / console | `npm test && npm run gold && npm run typecheck` |
| native engine | `make -C engine` |
| `sdk/js` | `cd sdk/js && npm install && npm test` |
| `sdk/js-lite` | `cd sdk/js-lite && npm test` |
| `sdk/python` | `cd sdk/python && uv sync && uv run pytest` |
| `adapters/python` | `cd adapters/python && uv sync && uv run pytest` |
| `skills` | `cd skills && npm test` |
| `otsone` | `cd otsone && uv sync --extra dev && uv run pytest` |
See [`PUBLISHING.md`](./PUBLISHING.md) for how each package ships a release.
---
## Renamed from opentypesafe
This project was previously called **opentypesafe**. The HTTP API is unchanged
(`POST /v1/systemone`, `/v1/batch`, ... and `ots_live_` keys), and old names keep working.
Package names below are the names the packages are built under; none are published to a
registry yet.
| Old | New | Compatibility |
|-----|-----|---------------|
| npm `opentypesafe-sdk` | `@openjev/sdk` | `OpenTypeSafeClient` / `TypeSafeClient` still exported |
| npm `@caelum/opentypesafe` | `@openjev/lite` | `OpenTypeSafeClient` / `TypeSafeClient` still exported |
| Python `opentypesafe-sdk` (`import opentypesafe_sdk`) | `import openjev` | `opentypesafe_sdk` is kept as a deprecated alias module |
| Python `opentypesafe-adapter` (`import opentypesafe_adapter`) | `openjev-adapters` (`import openjev_adapters`) | `opentypesafe_adapter` is kept as a deprecated alias module |
| `OPENTYPESAFE_*` env vars | `OPENJEV_*` | old names are read as a fallback (then `TYPESAFE_*`) |
| model ids `opentypesafe-grok`, `opentypesafe-latest`, ... | `openjev-grok`, `openjev-latest`, ... | old ids still resolve |
| CLI `opentypesafe` | `openjev` | both bin names are installed |
---
## Origins & licenses
OpenJev began as an open, wire-compatible reimplementation of the TypeSafe AI System One
v1 API, and it is honest about where its pieces come from:
- The MIT-licensed `sdk/js`, `sdk/python`, `adapters/python` and `skills` directories are
derived from TypeSafe AI's SDKs; each keeps its original `LICENSE` and `NOTICE`.
- "Jev" is the name of TypeSafe AI's model family, and "System One" is the name of
TypeSafe AI's API. OpenJev is an independent project and is **not affiliated with,
sponsored by or endorsed by TypeSafe AI**; those names are used only to describe wire
compatibility, and the `jev-*` model ids exist only as compatibility aliases. OpenJev
does not ship or reproduce TypeSafe AI's models. Other unrelated open-source projects
also use the name "OpenJev".
- The `otsone-1` model is our own; the Laya models are Convai Innovations' (Apache-2.0).
## Licensing
- Root project, `engine/`, `otsone/`, `sdk/js-lite/`: **Apache-2.0** ([LICENSE](./LICENSE)).
- `sdk/js`, `sdk/python`, `adapters/python`, `skills`: **MIT**, derived from the
TypeSafe AI SDKs (each directory keeps its original `LICENSE` and `NOTICE`).
- `otsone/` includes code adapted from [Laya](https://github.com/NandhaKishorM/laya)
(Convai Innovations, Apache-2.0) — see [otsone/NOTICE](./otsone/NOTICE).
- Model weights are not distributed in this repository.
See [NOTICE](./NOTICE) for attributions.
---
## Roadmap
The live roadmap is rendered at `/roadmap` (source:
[`src/lib/content/roadmap.ts`](./src/lib/content/roadmap.ts)). Near-term priorities:
- Publish the SDKs to npm and PyPI (the Python distribution needs a new name; `openjev` is
taken on PyPI) and a prebuilt Docker image.
- Improve the default engine's accuracy on the public leaderboard, and add more public
datasets.
- Go client SDK, decision export (JSONL / CSV), OpenTelemetry trace export.
Suggestions are welcome as GitHub issues.
## Found this useful?
If OpenJev saved you a prompt-and-parse hack, a star helps the next person
wrestling with unreliable LLM output find it. If you know someone building
agents or routing/triage logic who'd rather get a typed answer back than parse
prose, a share or a mention helps more than the star — no worries if not, just
offering since it's free either way.
## Contributing
Contributions are welcome: bug reports, datasets for the leaderboard, engine
improvements, SDKs for more languages. Start with [CONTRIBUTING.md](./CONTRIBUTING.md);
this project follows the [Code of Conduct](./CODE_OF_CONDUCT.md). Please report security
issues privately as described in [SECURITY.md](./SECURITY.md), not in public issues.
TDQS
Scored across 4 tools
The three primitives map cleanly to distinct output types: single-choice classification, ordinal rating, and binary yes/no. The only overlap is decide, which deliberately aggregates the other three; its description makes the batch role clear, though an agent could occasionally reach for it when a single primitive would do.
All names are short, lowercase single tokens, which is consistent in form. However, the semantics are mixed: choice and score are nouns, decide is a verb, and noul is an opaque coinage that only the description decodes.
Four tools is exactly right for a small set of classification primitives, with one tool per output type plus a convenience aggregator. Nothing feels padded or missing.
Categorical, ordinal, and binary judgments plus a multi-question batch cover the core decision space well. Minor gaps remain: no multi-label classification and no batch operation across multiple texts, but agents can work around these by composing calls.