jev-harness
⚡ Jev Harness: The Token Optimizer & Decision Gate for AI Coding Agents
Stop burning 50,000 frontier tokens on missing packages, network flakiness, and circular doom loops.
jev-harnessis an ultra-fast, zero-dependency token optimizer, test failure triage gate, and semantic guardrail for AI coding agents (OpenCode, Command Code, Claude Code, Cursor, Antigravity IDE, Windsurf, Zed, and Pi). Powered by TypeSafe AI's Jev System One non-autoregressive decision model.🤖 Operating as an AI Agent? If you are working on this repository, read
AGENTS.md(Português) first. If you are integrating the tool into your own project, you do not need it — use the Universal AI Agent Integration Guide (2-minute setup, no API key required to start).What it is: a deterministic-first System 1.5 decision layer for the code-quality loop — test-failure triage, doom-loop breaking, completion veto, reasoning-effort governance and model routing — callable as a CLI, MCP server, git/CI hook and typed SDKs across three runtimes (Python, TypeScript, Rust). What it is not: a coding agent; a runtime supervisor (that is Foreman); a tool-call guardrail (jev-guard); a context sieve (Winnow); or a capability router (JevRouter). See Where it fits.
The Problem · How It Works · Where It Fits · Features · Quickstart · CLI · Astra-Jev effort governance · Integrations · SDKs (TS, Rust) · Git & CI guardrails · Economics & benchmarks · Architecture & roadmap · Contributing · Release · Changelog
🎯 The Problem
When an autonomous coding agent encounters a test failure or compiler error, the standard reaction is to dump 500 lines of raw traceback into an expensive frontier reasoning model (GPT-6 Astra, Claude Fable 5.1).
Failure Scenario | Without Jev Harness | With Jev Harness |
Missing dependency ( | 💸 ~50,000 LLM tokens (estimate) (~$0.50 - $2.50) + 15s delay to output | ⚡ Jev triage (measured: < 1 ms offline in-process, ~80–100 ms offline CLI, ~0.5–1 s live; ≈ $0.00002/call at ~470 input tokens) → Action: install package deterministically. 0 LLM tokens. |
Flaky transient error (network timeout, port busy, ECONNREFUSED) | 💸 LLM hallucinates architectural changes to "fix" an ephemeral glitch | ⚡ Jev detects flaky transient → Auto-retry worker once. 0 code changes. |
Circular refactoring (Doom Loop: attempting the same fix 3+ times) | 💸 200,000+ tokens burned in endless circular loops | 🛑 Jev Abort Gate triggers ( |
Trivial typo / formatting | 💸 Heavy reasoning frontier tier used for simple regex/typo | ⚡ Jev Route directs task to local script or Gemini 3.8 Flash. |
Related MCP server: aperion-shield
🏗️ How It Works: System 1 → System 1.5 → System 2
Daniel Kahneman's cognitive paradigm applied to agentic engineering, with this harness acting as the System 1.5 executive governance layer between System 1 perception and System 2 deliberation:
System 1 (Fast, Intuitive, Calibrated): TypeSafe Jev System One provides non-autoregressive, parallel, typed micro-decisions. Latency: 70–300 ms (measured: ~80–100 ms offline CLI, ~0.5–1.0 s live on the free tier). Pricing: $0.042 per 1M input tokens ($0 output tokens). It answers specific closed-world questions (
triage_category,severity_score,should_abort,target_tier,completion_status) without generative hallucinations.System 1.5 (Deterministic Connective Tissue & Executive Gate):
jev-harnessis the executive decision layer that coordinates System 1 and System 2 into a robust, bounded feedback loop (Josh Rosen's cognitive agent paradigm):Zero-trust state sanitization & focused perception: Redacts sensitive credentials, masks prompt injections, and extracts targeted assertion slices (≤15 lines) instead of flooding models with 500-line terminal logs.
Deterministic short-circuits & auto-recovery: Instantly diagnoses missing dependencies (
pip,npm,cargo) and transient network/port hiccups in < 500 µs locally without spending any LLM tokens.Uncertainty calibration & entropy envelopes: Computes confidence margins and normalized entropy to guard against borderline calls, automatically escalating ambiguous cases to System 2.
Reasoning-effort leasing & doom-loop breaking: Regulates cognitive effort tiers (
lowtoextra_high) for frontier models and trips an automatic circuit breaker (exit 1) when agents get stuck in repetitive repair loops.Cross-runtime parity: Implemented natively in Python, TypeScript, and Rust with zero external runtime dependencies and full offline fallback.
System 2 (Slow, Deliberative, Generative): Frontier reasoning models (GPT-6 Astra, Claude Fable 5.1, Claude Opus 5) write complex code, architect multi-file refactorings, and solve deep logic defects. System 1.5 ensures System 2 is only invoked when strictly necessary, cutting token spend by up to ~80–90%.
┌────────────────────────────────────────────────────────────────────────┐
│ AI Coding Agent Execution Loop │
│ (OpenCode / Claude Code / Cursor / Windsurf / Antigravity IDE) │
└───────────────────────────────────┬────────────────────────────────────┘
│
[Step / Test Execution]
│
▼
[Execution Output]
│
┌─────────────────────────┴─────────────────────────┐
▼ ▼
[✅ PASS: Continue] [❌ FAIL: Traceback]
│
════════════════════════════════════════════════════════════════════╪══════════════════════════════════
🧠 SYSTEM 1.5: EXECUTIVE DECISION & GOVERNANCE LAYER (jev-harness) │
────────────────────────────────────────────────────────────────────┼──────────────────────────────────
▼
┌───────────────────────────────┐
│ 1. Perception & Sanitization │
│ • Credential Redaction │
│ • Focused Traceback Slicing │
│ • Injection Screening Guard │
└───────────────┬───────────────┘
│
▼
┌───────────────────────────────┐
│ 2. Local Deterministic Fast │
│ Heuristics (< 500 µs) │
│ • Missing packages (regex) │
│ • Transient network / ports │
│ • Doom loop pattern match │
└───────────────┬───────────────┘
│
┌───────────────────┴───────────────────┐
▼ (Hit / Certain) ▼ (Ambiguous)
┌──────────────────────────────────────────────────────────────┐ ┌───────────────────────────────┐
│ 3. Decision Cache & Receipts │ │ ⚡ SYSTEM 1 (TypeSafe Jev) │
│ • Deduplication via .jev/cache.json │ │ • Non-autoregressive model │
│ • Append-only audit receipts (SHA-256) │ │ • Sub-second micro-decisions │
│ • Reasoning effort lease management │ │ • Typed score & distribution │
└──────────────────────────────┬───────────────────────────────┘ └───────────────┬───────────────┘
│ │
└───────────────────────────┬────────────────────────────┘
│
▼
┌───────────────────────────────┐
│ 4. Uncertainty & Policy Gate │
│ • Margin & Normalized Entropy│
│ • Dynamic effort modulation │
│ • Fail-open / fail-closed │
└──────────────┬────────────────┘
│
═══════════════════════════════════════════════════════════╪═══════════════════════════════════════════
┌──────────────────────────────────┴──────────────────────────────────┐
▼ ▼
[skip_llm = True / Actionable] [skip_llm = False / Escalated]
│ │
▼ ▼
Deterministic Shell Recovery 🧠 SYSTEM 2 (Frontier LLM)
• pip/npm/cargo install package (GPT-6 Astra / Claude Fable)
• Exponential backoff retry • Deep algorithmic debugging
• Abort circuit breaker (exit 1) • Multi-file architectural fix
│ • Bounded reasoning effort
▼ │
⚡ 0 Frontier Tokens Spent 💸 Cost cut by ~80%✨ Features
🛡️ Zero External Dependencies: Built entirely with Python's standard library (
urllib.request,dataclasses,json). Nopipbloat; measured offline CLI cold start ~80–100 ms, in-process gates in tens of microseconds.🔌 Universal MCP Server: Exposes Jev decision tools over stdio (
jev-mcp) for Cursor, Claude Desktop, Antigravity, Windsurf, Zed, and OpenCode.🚦 UNIX Philosophy Compliant: Standard exit codes (
0for safe/skip_llm,1for abort/logic defect,2for syntax error) allow clean pipe composition:pytest | jev-harness test-gate.🔄 Resilient Provider Fallback: retryable failures (
429,5xx, timeouts, network errors) retry with capped exponential backoff and honorRetry-After; after the attempts, the default fail-open policy degrades to the deterministic engine and marks the answer (is_mock=true+degraded_reason:auth_401,http_429,http_500,timeout,connection,invalid_response). A200the runtime cannot interpret — a wrong field type (score: "N/A"), an unknown answer type, a missing required field oranswers: []— takes the same path in all three runtimes: a marked degradation, never a crash, a silentNaNor a silently defaulted score.--fail-closedsurfaces errors instead (exit2, no traceback),--retries Ntunes attempts.👻 Shadow Mode:
--shadow(or"shadow": truein.jev.json) decides and reports[SHADOW] would exit Nwhile always exiting0, so the pipeline keeps running while you measure the gates on real traffic. CLI misuse (a missing--logfile, an invalid flag) still exits2.🧾 Audit trail & self-diagnosis:
jev-harness doctorchecks config, credentials (fingerprint only), model origin, limits, state permissions, receipts/cache and the git hook — every problem comes with the fix command.jev-harness receiptsreads the append-only decision trail (hashes + metadata, never raw logs;0600, TTL and size bounded,--no-receiptsto disable).🧪 Calibration corpus & replay gate:
jev-harness replay --corpus tests/corpusruns 160 labelled cases through every gate, prints a confusion matrix, precision/recall/F1 and ECE per gate, and fails CI on a regression againstdocs/REPLAY_REPORT.jsonor when an adversarial log is classified deterministically.⚡ Decision cache & debounce: identical live decisions are served from
.jev/cache.json(--no-cachebypasses it), and repeatednudge-gate/abort-checkevaluations inside the debounce window coalesce (debounced: true). Shadow runs and degraded answers are never cached; hit-rate shows up inmetrics.🤖 CI triage Action: a composite GitHub Action turns a failed step into a categorised annotation with the next deterministic action — offline by default (no key, no network), never blocks a green run, and blocking is opt-in via
fail-on.🎯 Uncertainty envelope: every gate result carries an additive
uncertaintyblock (margin,normalized_entropy,confidence,escalate_to_system2,escalation_reason) derived from the provider's real distribution — zeros and single-option questions guarded, both key conventions accepted, and a green run is never escalated. It never changesskip_llmor an exit code.✂️ Focused perception & state redaction: the triage gate sends the provider a
focused_slice(the assertion line and its neighbours, ≤15 lines) plus acausal_contextblock instead of only the raw log, and redacts credential-shaped material from the state itself — in all three runtimes.🧩 Structured recovery, never auto-executed: a missing dependency becomes data (
{action_type, package_name, package_manager, argv, is_safe_auto_run, rationale}) — no shell string, per-ecosystem name validation, andis_safe_auto_runrequires the package to be in your manifests and an explicit--allow-auto-recovery.🧠 Session memory & effort lease: the gates reuse this repository's recent decisions (an explicit
--historystill wins), and a decision opens a bounded effort lease that answers in sub-milliseconds with--use-lease— invalidated immediately by--tool-error(break-glass).🔌 Host plugins & interop: ready bundles for Claude Code (
plugins/claude-code, manifest + skill + MCP registration) and Codex/OpenCode (plugins/codex), plus a documented interop section with the dated ecosystem table and an offline link checker in CI.🌐 Multi-Provider Support: TypeSafe AI direct, OpenCode Zen, Command Code, Vercel AI Gateway, and OpenRouter (alpha access).
🧭 Where It Fits: The System 1.5 Decision Layer
jev-harness is one role in the emerging System 1.5 category: connecting a fast System One decision model (Jev) to System 2 frontier models through deterministic software. The category already has specialised tools — use each where it belongs (stars/snapshot: 2026-09-23):
Tool | Role | Works offline? |
jev-harness (this repo) | Quality gates: test triage, doom-loop breaking, completion veto, effort governance, model routing | ✅ Deterministic engine, no key needed |
Runs and supervises coding workers (steer / stop / retry / verify / finish) | ❌ Requires a key | |
Routes model / subagent / skill / MCP capabilities with policy and receipts | ❌ Requires a key | |
Filters what enters the agent's context window | ❌ Requires a key | |
Guardrails for tool calls (deny / ask / allow) and prompt-injection screening | ❌ Requires a key |
Our unique combination: the only tool in this set that (1) works fully offline with a deterministic engine, (2) ships three runtimes with semantic parity for gate verdicts (Python / TypeScript / Rust — known mock-probability divergence is tracked as E3.9), and (3) owns the test / commit / CI quality gate.
📚 Architecture & roadmap: System 1.5 plan · Ecosystem & opportunities · Implementation plan.
🚀 Quickstart
1. Installation Across Ecosystems
Available on all three major package registries with zero external runtime dependencies:
Ecosystem | Registry | Package / Command | Status |
Python |
| ||
TypeScript / Node |
| ||
Rust |
| ||
GitHub Releases | Prebuilt binaries & assets |
# Python (CLI + SDK)
pip install jev-harness
# or isolated global CLI
pipx install jev-harness
# TypeScript / Node.js (CLI + SDK)
npm install @ismaelsoilet/jev-harness
# or run directly via npx
npx @ismaelsoilet/jev-harness --version
# Rust (Crate + Standalone CLI)
cargo add jev-harness
# or install standalone binary 'jev'
cargo install jev-harness2. Configuration & Multi-Provider Support
Jev Harness supports multiple backend providers and auto-detects credentials:
Provider | Endpoint | Cost | Configuration |
Command Code (Free Deal) |
| $0.00 / Free |
|
OpenCode Zen (Free Tier) |
| $0.00 / Free |
|
TypeSafe AI (Direct) |
| $0.042 / 1M |
|
OpenRouter (Alpha) ⚠️ |
| $0.042 / 1M |
|
Vercel AI Gateway |
| $0.042 / 1M |
|
Autonomous Simulation | Local Heuristics (< 500µs) | $0.00 | Active by default if no key or offline |
📅 Provider verification date: 2026-09-22. Model IDs, free tiers and prices change weekly — agents and engineers should re-verify them (and record their own date) if more than 30 days have passed. Step-by-step key acquisition for every provider: Universal AI Agent Integration Guide.
🔒 Privacy: offline mode (
--mock, or no credentials) makes zero network calls. Live mode transmits the typed questions and the failure log (head 2,000 + tail 4,000 characters) to the provider endpoint. Since v0.2.0 the state itself is redacted in all three runtimes: credential-shaped material (API keys, JWTs, GitHub/AWS tokens, DB URLs, private keys) is masked before it leaves the process — in every gate and over MCP/SDK alike. That is shape-based hygiene, not a data-loss-prevention layer: structured customer data that does not look like a credential is still transmitted, so use--mockfor repositories with regulated data.
Credential resolution priority:
Environment variables (
TYPESAFE_API_KEY,CMD_API_KEY,COMMAND_CODE_API_KEY,OPENCODE_API_KEY,OPENROUTER_API_KEY, orAI_GATEWAY_API_KEY)Local repository
.jev.json,.env, or~/.commandcode/auth.jsonGlobal configuration
~/.config/jev/credentials.envResilient Fallback: active when no credentials are configured and on retryable provider failures (
429/5xx/timeouts/network) or401/403from any provider. Retries use capped backoff +Retry-After; the engine prints[JEV WARNING]to stderr and every degraded result is flaggedis_mock=truewith adegraded_reason. Use--fail-closedto surface those errors instead (CLI exit2, no traceback).
# Check current connection & provider status anytime
jev-harness statusRepository Configuration (.jev.json)
jev-harness init scaffolds a repository-local .jev.json. Honored keys:
Key | Type | Default | Effect |
| string | provider default | Overrides the model sent to the provider. The scaffold placeholder |
| float |
| Minimum confidence for |
| float |
| Minimum dead-end probability for |
| bool |
| Decide and report, but never change the exit code (see Shadow Mode). |
| string | — | Optional credentials. Environment variables take precedence. |
Values are clamped to [0, 1], and a corrupted file degrades to defaults instead of breaking CI. The same keys work identically in Python, TypeScript and Rust.
Pinning the model. The effective model resolves as explicit argument → JEV_MODEL env var → model in .jev.json → provider default (run jev-harness status to see both the model and where it came from). The alias jev-latest moves: the provider may change what it points to. Once your skip_llm_threshold is calibrated against a model version, pin it — for example "model": "jev-1.13.0" — so a provider release cannot silently change your decisions.
🛠️ CLI Usage
1. Test Failure Triage (test-gate)
Pipe error logs directly or pass a file:
# Pipe directly from your test runner
npm test | jev-harness test-gate
pytest | jev-harness test-gate
# Or analyze a saved log file
jev-harness test-gate --log error.log
# Or get machine-readable JSON
jev-harness test-gate --log error.log --json
# Observe decisions without blocking a pipeline (always exits 0)
jev-harness test-gate --shadow --log error.log
# Fail hard on provider outages instead of degrading to the offline engine
jev-harness test-gate --fail-closed --log error.log
# Bound how long retryable provider failures are retried (default: 3 attempts)
jev-harness test-gate --retries 1 --log error.logOutput Example:
--- JEV TEST TRIAGE VERDICT ---
Category: ENV_MISSING
Confidence: 92.0%
Skip LLM Call: YES (Save Tokens!)
Skip Probability: 96.0%
Severity Score: 1.0 / 4.0
Recommendation: AUTO-ACTION: Install missing dependency or check environment configuration (Do NOT call LLM).
--------------------------------2. Guard Against Doom Loops & Dead-Ends (abort-check)
Verify that a proposed plan isn't repeating a failed path:
jev-harness abort-check \
--plan "Retry rewriting the entire database schema without backup" \
--history "Attempt 1 failed with timeout. Attempt 2 failed with circular foreign key error."Returns exit code 1 if abort is recommended, enabling automated CI stops.
3. Model Tier Routing (route)
Pick the cheapest model capable of solving the task:
jev-harness route --task "Fix typo in docstring and reformat with black"
# -> TIER: DETERMINISTIC | Model: Direct Python/Bash Script (0 LLM Tokens)
jev-harness route --task "Refactor distributed actor supervision tree across 14 modules"
# -> TIER: HEAVY_SYSTEM2 | Model: Claude Fable 5.1 / GPT-6 Astra (~$10.00 in / $50.00 out)4. Step Completion Verification (verify)
Verify evidence against criteria with calibrated confidence:
jev-harness verify \
--criteria "Must export format_date function and pass all 10 unit tests" \
--output "All 10 unit tests passed in 0.02s. format_date exported in index.ts."5. Dynamic Reasoning Effort Governance (reasoning-effort / astra-jev)
Dynamically modulate reasoning effort per-generation (inspired by Vechen @miu21590) to eliminate latency and save thousands of tokens on mechanical tool steps:
# Evaluate immediate step for DeepSeek (e.g. DeepSeek V4.1-Flash / V4-Pro)
jev-harness reasoning-effort \
--context "git status e verificar arquivos alterados no commit recente" \
--target-provider deepseek
# Output:
# Effort: LOW | Dialect: {"extra_body": {"thinking": {"type": "enabled"}}, "reasoning_effort": "low"}
# Latency eliminated: ~200s internal CoT reduced to 1.5s!
# Evaluate architectural task for Anthropic (Claude Fable 5.1 / Claude Opus 5)
jev-harness reasoning-effort \
--context "Architect distributed actor supervision tree with raft consensus" \
--target-provider anthropic --json
# Safeguard check for direct models (returns empty params and warnings for non-reasoning models)
jev-harness reasoning-effort \
--context "Run bash command" \
--target-provider openai \
--model gpt-5.6-luna6. Continuation Nudge Gate (nudge-gate / nudge)
Inspired by CommandCodeAI/cmd-mod-jev-nudge, nudge-gate combines gated workflow phases (research, ask, plan, execute, verify, complete) with calibrated Noul probabilities (nudge, waiting, progress) to evaluate whether an autonomous agent paused prematurely with unfinished work or unverified changes (should_nudge = true, exit code 0), while automatically vetoing nudges when waiting on user input (waiting >= 0.5 or phase == "ask"), when the previous nudge produced no progress (progress < 0.5), or when all tests pass (phase == "complete"):
# Evaluate if an agent stopped after editing code without running tests
jev-harness nudge-gate \
--transcript "Assistant: Edited src/auth.py. Now I need to run pytest to verify." \
--json
# -> should_nudge: true | workflow_phase: "verify" | exit code 0
# Evaluate when waiting on user choice (vetoed automatically)
jev-harness nudge-gate \
--transcript "Assistant: Which AWS region should I deploy to? Would you like me to proceed?"
# -> should_nudge: false | workflow_phase: "ask" | exit code 17. ROI & Token Savings Telemetry (metrics)
Inspect cumulative tokens saved, dollars saved, and doom loops intercepted:
# View active telemetry
jev-harness metrics
# Reset session telemetry counters
jev-harness metrics --resetOutput Example:
============================================================
JEV HARNESS TELEMETRY & ROI
============================================================
Total Triage Interceptions: 14 calls
LLM Frontier Calls Skipped: 11 calls (78.6%)
Abort Guard Stops Triggered: 2 doom loops killed
Deterministic Routes: 6 tasks
Reasoning Effort Modulations: 8 steps (6 low, 2 high)
Estimated Tokens Saved: 422,200 tokens (heuristic estimate)
Estimated Frontier Dollars Saved: $6.12 USD (heuristic estimate)
Assumption Model: 26,200 tokens/$0.31 per intercepted triage; 80,000 tokens/$1.20 per aborted doom loop
============================================================📊 These figures are a planning estimate, not metered usage. The per-event assumptions are fixed constants (26,200 tokens/$0.31 per intercepted triage, 80,000 tokens/$1.20 per aborted loop).
--jsonexposesestimates_are_heuristic: trueso downstream tooling can label them correctly.
8. Self-diagnosis, audit trail and calibration (doctor / receipts / replay)
# Is my installation healthy? (never prints secrets; --live spends ONE request)
jev-harness doctor
jev-harness doctor --live --json
# What did this repository decide? (append-only, hashes + metadata only)
jev-harness receipts --tail 10
jev-harness receipts --json
# How accurate are the gates? (confusion matrix, P/R/F1, ECE per gate; fails on regression)
jev-harness replay --corpus tests/corpus7. One-Command Agent Setup (init)
Automatically scaffold MCP configurations for your active agent or IDE:
# Setup for Cursor
jev-harness init --cursor
# Setup for Antigravity IDE
jev-harness init --antigravity
# Setup git pre-commit hook (detects npm/pytest/cargo; never overwrites an existing hook)
jev-harness init --git
# Setup all supported tools at once
jev-harness init --all⚡ Astra-Jev: Dynamic Reasoning Effort Governance (2026 Frontier Models)
Inspired by Vechen's (@miu21590) groundbreaking work on Astra-Codex and the Astra-Ares framework, Astra-Jev introduces autonomous, per-generation reasoning effort modulation governed by TypeSafe Jev System One.
Instead of locking an entire multi-turn coding session into heavy, slow reasoning (or risking bugs by running exclusively in low reasoning), Astra-Jev evaluates the cognitive demand of the immediate next generation in < 500µs locally (in-process; live latency depends on the provider — measured ~0.5–1.0 s on the free tier).
┌────────────────────────────────────────────────────────┐
│ Autonomous Agent Loop │
└──────────────────────────┬─────────────────────────────┘
│
Proposed Next Action
("git status", "read file", or "architect kernel")
│
▼
┌───────────────────────────────┐
│ Astra-Jev Gate │
│ (Jev System One Micro-Eval) │
└───────────────┬───────────────┘
│
┌────────────────────────────────┼────────────────────────────────┐
▼ ▼ ▼
[Trivial / Mechanical] [Standard Feature] [Deep Architecture]
Cognitive Depth: LOW Cognitive Depth: MED Cognitive Depth: HIGH
│ │ │
▼ ▼ ▼
Compile Provider Dialect Compile Provider Dialect Compile Provider Dialect
(e.g. enable_thinking: false) (e.g. reasoning_effort: med) (e.g. thinking: adaptive max)
│ │ │
▼ ▼ ▼
⚡ 1.5s response (~0 CoT) 🎯 Balanced ~4,000 CoT 🧠 Deep 32,000 CoT Analysis
Western: Saves ~$0.80 USD Western: Normal pricing Western: Maximum reasoning
Chinese: Saves ~240s wait Chinese: Normal thinking Chinese: Deep explorationDual ROI: Why Modulate Reasoning Effort in 2026?
The value of dynamic reasoning modulation fundamentally depends on the provider architecture:
Provider Ecosystem | Problem Solved | Without Astra-Jev | With Astra-Jev |
Western Frontier(GPT-6 Astra, Claude Fable 5.1) | Dollar Cost($10/1M in, $50/1M out) | Agent burns ~8,000 reasoning tokens ($0.40 - $1.20) just to inspect | Injects |
Chinese Frontier(DeepSeek V4.1-Flash, Qwen 3.8 Max, Kimi-k3, MiMo) | Latency & GPU Starvation(Tokens are cheap, but internal CoT takes 3–5 minutes) | Agent enters 200–300 second internal thinking loop before running a trivial bash command | Disables thinking CoT or sets |
🛡️ Critical Safeguards Built into Astra-Jev
Direct Single-Pass Model Safeguard: Models that do not support internal reasoning (e.g.
gpt-5.6-luna,gemini-3.8-live,claude-3.5-haiku) will return fatal HTTP 400 Bad Request if reasoning parameters are injected. Astra-Jev automatically detects non-reasoning targets, setsis_reasoning_supported = False, and returns clean empty payloads{}.Preservation of
reasoning_content(DeepSeek multi-turn): In DeepSeek V4.1-Flash/Pro APIs, strippingreasoning_contentacross multi-turn tool calling can corrupt tool execution. Astra-Jev enforces dialect compliance to preserve thinking structures across turn transitions.Prompt Cache (KV Cache) Trade-off Advisory: Toggling reasoning parameters back-and-forth mid-session can invalidate prefix cache on long contexts (>100k tokens). Astra-Jev provides
cache_safe_recommendationadvisories:For pure mechanical actions, use Jev's
skip_llm=trueto execute directly without calling the LLM at all.Keep reasoning effort stable across related sub-steps of a single complex implementation.
🤖 Universal Agent & IDE Integrations
📖 Looking for a turnkey setup for any project? Read the Universal AI Agent Integration Guide (Português) with copy-paste configs for Claude Code, Codex, Pi, Oh My Pi, CommandCode, Cursor, and CI/CD.
1. Claude Code (claude CLI by Anthropic)
Connect Jev to Claude Code in 5 seconds via native MCP:
# Register Jev Harness MCP into Claude Code
claude mcp add jev-harness -- npx -y @ismaelsoilet/jev-harness mcp
# Or using Python:
claude mcp add jev-harness -- jev-mcpAdd to your project's CLAUDE.md:
## Jev Harness Token Governance
- Before querying LLMs on test/build failures, call `jev_triage_test_failure`. If `skip_llm=true`, execute the deterministic fix directly.
- If a task retries twice without progress, call `jev_abort_check` to break doom loops.
- For mechanical bash commands or file lookups, call `jev_modulate_reasoning_effort` with effort="low".2. OpenAI Codex / Astra-Codex
Dynamically modulate GPT-6 Astra reasoning effort per generation step inside Codex without invalidating prefix cache (as featured on X):
# In Codex automation scripts or step pre-hooks:
jev-harness reasoning-effort \
--context "$TASK_STEP_DESCRIPTION" \
--target-provider openai --jsonInject the resulting reasoning_effort: "low" | "medium" | "high" into the root API payload. The harness never rewrites your message history, so it does not invalidate the provider's prefix cache by design.
3. Pi & Oh My Pi (pi / oh-my-pi)
Equip Mario Zechner's minimalist terminal agent (pi) and oh-my-pi workflows:
# In your Pi task or terminal prompt:
npm test 2>&1 | npx @ismaelsoilet/jev-harness test-gate
pytest 2>&1 | jev-harness test-gateIf the exit code is 0 (skip_llm=true), Pi applies the deterministic package installation or retry command without querying expensive models.
4. CommandCode
In .commandcode/config.json or CLI pre-command triggers:
{
"mcpServers": {
"jev-harness": {
"command": "npx",
"args": ["-y", "@ismaelsoilet/jev-harness", "mcp"]
}
}
}5. Cursor IDE (.cursor/mcp.json)
Add to .cursor/mcp.json (or run jev-harness init --cursor in your repo):
{
"mcpServers": {
"jev-harness": {
"command": "npx",
"args": ["-y", "@ismaelsoilet/jev-harness", "mcp"]
}
}
}6. Claude Desktop (claude_desktop_config.json)
Add to claude_desktop_config.json:
{
"mcpServers": {
"jev-harness": {
"command": "npx",
"args": ["-y", "@ismaelsoilet/jev-harness", "mcp"]
}
}
}7. Google Antigravity IDE (mcp_config.json & hooks.json)
Connect as an MCP Server:
{
"mcpServers": {
"jev-harness": {
"command": "jev-mcp",
"args": []
}
}
}Or hook into the execution lifecycle in ~/.gemini/config/hooks.json:
{
"jev-guard": {
"PreInvocation": [
{
"type": "command",
"command": "echo '{\"injectSteps\": [{\"ephemeralMessage\": \"[JEV ACTIVE] Triage test errors with jev-harness test-gate before calling LLMs. If skip_llm=true, fix deterministically.\"}]}'"
}
]
}
}8. OpenCode, Windsurf & Zed
OpenCode: Add Jev triage gate to
.opencode/config.json.Windsurf: Add to
~/.codeium/windsurf/mcp_config.json.Zed: Add to
~/.config/zed/settings.jsonundercontext_servers.
🐍 Python SDK
You can also use jev-harness directly in Python scripts and agent orchestration frameworks (LangChain, LlamaIndex, CrewAI, AutoGen):
from jev_harness import (
JevClient,
triage_test_failure,
should_abort_trajectory,
route_model_tier,
verify_step_completion,
modulate_reasoning_effort,
)
client = JevClient()
# 1. Triage test traceback
res = triage_test_failure("ModuleNotFoundError: No module named 'scipy'", client=client)
if res.skip_llm:
print(f"Safe to fix deterministically: {res.action_recommendation}")
# 2. Check trajectory before spending tokens
abort_decision = should_abort_trajectory(
proposed_step="Tentar novamente a mesma abordagem",
recent_attempts_summary="Tentativa 1 falhou com timeout",
client=client,
)
if abort_decision.should_abort:
print("Trajectory aborted! Re-align with user.")
# 3. Dynamic Reasoning Effort Modulation (Astra-Jev)
effort_res = modulate_reasoning_effort(
context="git status e inspecionar diff de arquivos alterados",
provider="deepseek",
model="deepseek-v4.1-flash",
client=client,
)
print(f"Effort: {effort_res.effort}") # low
print(f"Provider Params to inject: {effort_res.provider_params}") # {'extra_body': {'thinking': {'type': 'enabled'}}, 'reasoning_effort': 'low'}🟦 TypeScript / JavaScript SDK & CLI
For Node.js, Bun, Deno, Vite, Tauri, and Next.js applications:
# Install via npm
npm install @ismaelsoilet/jev-harness
# Or via bun
bun add @ismaelsoilet/jev-harnessProgrammatic Usage
import {
triageTestFailure,
shouldAbortTrajectory,
routeModelTier,
verifyStepCompletion,
modulateReasoningEffort,
JevClient,
} from "@ismaelsoilet/jev-harness";
// 1. Triage test failure in < 2ms locally (or sub-second remote)
const triage = await triageTestFailure(rawErrorOutput);
if (triage.skipLlm) {
console.log("Safe to fix deterministically! LLM call skipped.");
console.log("Recommended Action:", triage.actionRecommendation);
}
// 2. Prevent circular doom loops before spending frontier tokens
const abortCheck = await shouldAbortTrajectory(
"Repeat previous refactoring step",
"Step failed with: TypeError: undefined is not a function"
);
if (abortCheck.shouldAbort) {
console.error("Agent trapped in dead-end loop! Aborting.");
}
// 3. Dynamic Reasoning Effort Modulation (Astra-Jev)
const effortRes = await modulateReasoningEffort(
"git status and check changed files",
"anthropic"
);
console.log("Effort:", effortRes.effort); // low
console.log("Params:", effortRes.providerParams); // { thinking: { type: 'adaptive' }, output_config: { effort: 'low' } }TypeScript CLI
# Run test triage via npx
npx @ismaelsoilet/jev-harness triage "Cannot find module 'lodash'"
# Trajectory abort check
npx @ismaelsoilet/jev-harness abort-check --plan "Try identical prompt again"
# Model router
npx @ismaelsoilet/jev-harness route "Architect distributed consensus protocol"
# Dynamic reasoning effort modulation (Astra-Jev)
npx @ismaelsoilet/jev-harness reasoning-effort --context "git status" --target-provider deepseek --json🦀 Rust Crate & Standalone CLI
Ultra-low latency (< 500µs local, zero-overhead) for systems programming, Tauri backends, and terminal tools without Python or Node.js dependencies:
[dependencies]
jev-harness = "0.2.0"
tokio = { version = "1", features = ["full"] }Programmatic Usage
use jev_harness::gates::{triage_test_failure, should_abort_trajectory, modulate_reasoning_effort};
#[tokio::main]
async fn main() {
// 1. Triage test traceback in < 500µs
let triage = triage_test_failure("error[E0463]: can't find crate for 'serde'", None).await.unwrap();
if triage.skip_llm {
println!("Safe to fix deterministically: {}", triage.action_recommendation);
}
// 2. Trajectory guard against dead ends
let abort = should_abort_trajectory("Repeat same step", "Attempt 1 failed", None).await.unwrap();
if abort.should_abort {
eprintln!("Doomed loop detected: {}", abort.reasoning_summary);
}
// 3. Dynamic Reasoning Effort Modulation (Astra-Jev)
let effort_res = modulate_reasoning_effort("git status", "openai", None, None).await.unwrap();
println!("Effort: {}", effort_res.effort); // low
println!("Params: {:?}", effort_res.provider_params); // {"reasoning_effort": "low"}
}Standalone CLI Binary (jev / jev-harness)
# Install via Cargo
cargo install jev-harness
# Or use directly in shell pipelines
cargo test 2>&1 | jev test-gate
jev route --task "Architect enterprise distributed consensus"
jev reasoning-effort --context "git status" --target-provider deepseek --json📦 Git & CI/CD Guardrails
Pre-commit Hook (.pre-commit-config.yaml)
The hook needs your test command as an argument (a hook repository cannot guess your runner):
repos:
- repo: https://github.com/ismaelsoilet/jev-harness
rev: v0.2.0
hooks:
- id: jev-test-gate
args: ["pytest -q"] # or "npm test", "cargo test --quiet", ...Generated Git Hook (jev-harness init --git)
Detects your runner (npm/pytest/cargo), writes a failure-only hook and never overwrites an existing one (it saves pre-commit.jev instead). It is virtualenv-aware: when a .venv/venv exists the hook uses its binaries (e.g. ./.venv/bin/python, ./.venv/bin/jev-harness), so it works whether or not the environment is activated.
jev-harness init --git
jev-harness init --git --test-cmd "make test-fast" # override the detected commandinit also scaffolds .jev.json, .env.jev.example (all providers), .agents/skills/jev-harness/SKILL.md and, with --cursor, .cursor/mcp.json. It never overwrites existing files or foreign hooks; re-running regenerates only the hook it generated itself (recognised by its marker, including pre-v0.1.13 variants).
Husky Hook (.husky/pre-commit)
Let the runner decide; ask Jev only for the triage of a failure:
if ! OUT=$(npm test 2>&1); then printf '%s\n' "$OUT" | jev-harness test-gate; exit 1; fiGreen runs are detected deterministically (
category: "no_failure", exit0, zero API calls), sonpm test 2>&1 | jev-harness test-gateis also safe — but the failure-only form above is the most explicit and never depends on run-summary parsing.⚠️
.git/hooks/is not version-controlled. A generated hook lives only on your machine; for teams, commit a hook script (or use thepre-commitframework withid: jev-test-gate) so every developer gets the same gate.
📊 Economics & Benchmarks (September 2026 Frontier)
Token & Cost Economics
Metric | 2026 Frontier Reasoning (GPT-6 Astra, Claude Fable 5.1) | Fast Agentic Tier (Gemini 3.8 Flash) | TypeSafe Jev System One ( |
Input Pricing | $10.00 / 1M tokens | $0.75 / 1M tokens | $0.042 / 1M tokens (~238x cheaper) |
Output Pricing | $50.00 / 1M tokens | $3.75 / 1M tokens | $0.00 (Free - Non-autoregressive) |
Latency | 10,000ms – 30,000ms | 1,500ms – 4,000ms | Provider claim: ~100 ms typical (70 ms floor). Measured harness E2E: ~0.5–1.0 s live (free tier), < 1 ms offline in-process |
Output Structure | Free-form prose & streaming tokens | Structured JSON tool calls | Strictly typed: Choice, Score, Noul |
Determinism | Stochastic reasoning | Stochastic generation | Calibrated probabilities — not infallible (see the model's documented jaggedness) |
Tri-Runtime Offline Latency (measured 2026-09-23, in-process mock)
When --mock (or no credentials) is active, every gate runs locally with zero network. Budget: p99 < 500µs — asserted in CI for Rust, measured for all three runtimes on 2026-09-23 (N = 1000 for triage, N = 500 for abort/effort, standard Linux host):
Runtime | Decision Gate | p50 | p95 | p99 | Mean |
Rust ( |
| 22.0 µs | 38.7 µs | 58.5 µs | 26.1 µs |
| 14.6 µs | 30.3 µs | 43.1 µs | 16.8 µs | |
| 15.5 µs | 29.9 µs | 40.5 µs | 17.6 µs | |
pure | 1.9 µs | 1.9 µs | 2.8 µs | 1.9 µs | |
TypeScript ( |
| 39.5 µs | 143.2 µs | 340.3 µs | 57.2 µs |
| 37.1 µs | 106.0 µs | 234.4 µs | 47.3 µs | |
| 21.9 µs | 77.2 µs | 338.4 µs | 34.1 µs | |
Python ( |
| 124.9 µs | 208.1 µs | 295.5 µs | 140.4 µs |
| 112.4 µs | 181.3 µs | 229.2 µs | 125.0 µs | |
| 78.1 µs | 131.3 µs | 162.0 µs | 86.8 µs |
⚡ Reproduce / zero-overhead: Rust asserts its budget in
packages/rust/tests/gates_test.rs(cargo test --test gates_test -- --nocapture); Python and TypeScript values are a point-in-time sample (2026-09-23, this host, in-process forced-mock client), not a CI assertion. Re-measure on your hardware before citing; the budget, not the exact microsecond, is the contract. Piping test runners through the gates still adds overhead far below human perception.
🗺️ Architecture & Roadmap
Document | What it answers |
Where the tool sits between System 1 (Jev) and System 2; what is verified today (v0.2.0) and what is missing | |
How we compare with Foreman, JevRouter, Winnow and jev-guard; 21 prioritised opportunities; the "can we be 1.5?" verdict | |
Epics, acceptance criteria, tests and sequencing (H1–H3) | |
Copy-paste setup for MCP, CLI, hooks and CI in any project |
🤝 Contributing & Submissions
Contributions are welcome!
Submitting to awesome-jev and awesome-jev-use-cases.
Open an Issue or Pull Request on GitHub.
# Development setup
git clone https://github.com/ismaelsoilet/jev-harness.git
cd jev-harness
python -m unittest discover -s tests🚀 Multi-Registry Release & Synchronization (PyPI, npm, Crates.io)
Do Registries Automatically Update on git push?
No. PyPI, npm, and Crates.io are immutable, independently versioned package registries. Pushing commits to GitHub updates only the Git repository, not the registry packages or their online documentation.
To update the packages and documentation across all 3 registries:
1. Unified Local Release Script (scripts/release.sh)
Use the automated multi-runtime script to check, bump versions, and publish:
# 1. Run full test battery (Python, TS, Rust - 569 tests)
./scripts/release.sh --check
# 2. Synchronously bump version in pyproject.toml, package.json, and Cargo.toml
./scripts/release.sh --bump 0.2.0
# 3. Publish to a specific registry or all at once:
./scripts/release.sh --publish rust # Publishes to crates.io
./scripts/release.sh --publish npm # Publishes to npm (@ismaelsoilet/jev-harness)
./scripts/release.sh --publish python # Builds wheel/sdist for PyPI
# 4. Create git tag and push to GitHub
./scripts/release.sh --git-tag 0.2.02. Automated GitHub Actions CD (.github/workflows/release.yml)
You can also trigger releases via GitHub Actions:
Automatic: Pushing any tag matching
v*.*.*(e.g.git push origin v0.1.6) triggers therelease.ymlworkflow, which tests all runtimes and automatically publishes to PyPI, npm, and Crates.io.Manual: Go to GitHub Actions → Release & Publish → Run workflow, specify the version, and click run.
(Requires PYPI_API_TOKEN and CARGO_REGISTRY_TOKEN in GitHub Repository Secrets; npm uses OpenID Connect (OIDC) Trusted Publishing with cryptographic Sigstore provenance without static tokens).
🌟 What's New in v0.2.0
🦀 Rust live parity fixed (breaking for direct crate users): live
Scoreanswers were silently dropped because the parser expected an integer score and a list legend while the provider returns a float and a level map.ScoreAnswer.scoreis nowf64,legendaccepts a map or a list, andprobabilitiesare parsed —severity,viability,rigorandcomplexitynow match Python/TypeScript in live mode. Two other public signatures changed in 0.2.0:JevClient::retry_delay_msnow takesOption<f64>(fractionalRetry-After) andJevClient::parse_api_responsereturns an error instead of silently dropping an unparseable answer. The crate is pre-1.0, so this ships as a minor release.🔁 Provider resilience: retry with capped, jitter-free exponential backoff and
Retry-Aftersupport (429/5xx/timeouts, fractional seconds included); after the attempts, the default fail-open policy degrades to the deterministic offline engine and marks the answer (is_mock=true+degraded_reason:auth_401,http_429,http_500,timeout,connection,invalid_response).--fail-closedsurfaces the error instead (exit2, no traceback);--retries Ntunes attempts.🧩 Malformed provider payloads are a first-class failure: a
200with type-mismatched fields (score: "N/A",answers: [],nullnumerics), an unknown answertypeor a missing required field used to crash Python with a raw traceback, produce a silentNaNin TypeScript and silently drop the answer in Rust — three different semantics for the same input, and a gate quietly falling back to its default score. All three now treat it asinvalid_response, retry it like a bad status, degrade under fail-open and raise under fail-closed, anddegraded_reasonis finally visible in--json, in the MCP payloads and in the humanMode:line.📦 Payload guard:
stateand questions are validated against the provider limits (128k state chars / 256k total, ~32k/64k tokens) before any network call, counting code points consistently across the three runtimes.📌 Model pinning and origin: the effective model resolves as explicit argument →
JEV_MODEL→"model"in.jev.json→ provider default, andstatusnow reports where it came from (Model origin: repository .jev.json), warning thatjev-latestis a moving alias. Pin a version once your thresholds are calibrated.🐛 No more traceback on long literal input: a task, state or
--logvalue longer than the OS path limit used to crashroute/verify/effortwith[Errno 36] File name too long(exit1with a traceback); such a value is now treated as literal text (or reported as "log file not found"), and an oversized live payload exits2with a clear message.👻 Shadow mode:
--shadow(or"shadow": truein.jev.json) decides and reports[SHADOW] would exit Non stderr while always exiting0— in all three runtimes, including when the provider fails, where it reports[SHADOW] would exit 2instead of breaking the pipeline. CLI misuse still exits2;test-gate --jsonexposesshadowandwould_exit.🧪 569-Test Battery: 393 Python + 89 TypeScript + 87 Rust, including a shared live-payload fixture, real HTTP/TCP retry servers, malformed-payload probes, and shadow/payload-limit/model-pinning parity.
🧾 Trust, audit and self-diagnosis:
doctor(OK/AVISO/FALHA + the fix command,--jsonfor agents),receipts(append-only audit trail with a stable input hash, hashes only,0600, TTL/size bounded),.jev/git-ignored by the repo and byinit, measuredusage/costsplit from the heuristic estimates inmetrics, and a decision cache with a reported hit-rate (--no-cacheto bypass).🧪 Measured calibration instead of assumed accuracy: a 160-case labelled corpus (
tests/corpus, 84 hand-labelled) plusreplay, which prints the confusion matrix, precision/recall/F1 and ECE per gate and fails CI on a regression or on an adversarial log being classified deterministically. The first baseline and its open findings are published indocs/REPLAY_REPORT.md.🛡️ Untrusted logs are treated as untrusted: a deterministic prompt-injection detector escalates (never skips the LLM) when the failure log addresses the judge, in all three runtimes — required by the adversarial corpus gate.
🤖 CI triage Action (
examples/github-action): annotates a failed job with the category and the deterministic action, offline by default and never blocking a green run.🎯 Decision quality made explicit:
uncertaintyper result (shape + escalation),recoveryas structured data with an allowlist-backed safety flag, a focused slice instead of a raw log for the provider, state-level secret redaction, session memory in the gates and a bounded effort lease with a caller break-glass. All additive: no exit code orskip_llmchanged.🔀 Tri-runtime parity is now enforced, not claimed:
tests/fixtures/corpus_parity.jsonlocks 160 corpus cases × 6 gates across Python, TypeScript and Rust. Building it exposed and fixed real divergences, including a non-deterministicHashMapiteration order in the Rust mock.📦 Distribution: host plugin bundles (Claude Code, Codex/OpenCode), an interop section with sourced dates, and an offline documentation link checker wired into CI.
🧭 System 1.5 documentation: architecture, ecosystem comparison and implementation plan are linked from the README (
SYSTEM_1_5_*.md).🔒 Privacy/ops unchanged: offline mode still makes zero network calls; degraded live answers are always labeled, never silent.
🌟 What's New in v0.1.14
🐛 A mistyped
--logpath is no longer triaged as if it were the log text.jev-harness test-gate --log /missing/filepreviously produced a fabricatedENV_MISSING/skip_llm=trueverdict with exit0; it now exits2with a clear hint (use a positional argument,--sample, or stdin). Fixed in all three runtimes.🪝 No more false sense of protection after
init --git. When a foreignpre-commithook is preserved, the CLI no longer prints a plain success message: it states that the gate is not active until you merge.git/hooks/pre-commit.jev.📖 Guide polish from a fresh-agent run: the MCP smoke test shows the project-virtualenv absolute path,
initdocuments exactly what it writes and that it must run inside a git repository, and exit code2now has a concrete example.🧪 211-Test Battery: 100% pass rate across 211 tests (117 Python, 49 Rust, 45 TypeScript).
🌟 What's New in v0.1.13
🔌 MCP integration actually works out of the box: the guide now lists the real tool names and their arguments (
jev_triage_test_failure,jev_abort_check,jev_route_task,jev_verify_completion,jev_modulate_reasoning_effort,jev_should_nudge_continuation), a 20-second smoke test, a validtools/callexample, a virtualenv note for client configs and an OpenCode snippet. The previous names (jev_should_abort_trajectory,jev_route_model_tier,jev_verify_step_completion,jev_get_telemetry) did not exist.🪝
init --gitis virtualenv-aware: the generated hook uses the project's environment binaries (./.venv/bin/python,./.venv/bin/jev-harness) when present, so a green suite is no longer blocked when the virtualenv is not activated, and--test-cmd "<command>"overrides the detected runner.🛡️
initnever clobbers: existing agent skills are preserved (like.jev.jsonand.env.jev.example), and only the hook it generated itself (marker, including pre-v0.1.13 variants) is regenerated.📋 MCP/CLI output parity: the Python MCP server now returns
action_recommendationalongsiderecommendation(andsummaryalongsidereasoning_summary), matching the TypeScript runtime.🧭 Clearer onboarding: the Quickstart states that no API key is required (offline mode is free and makes zero network calls),
statusguidance is fully in English,.env.jev.examplelists every provider, and the README scopesAGENTS.mdto people working on the repository itself.🧪 211-Test Battery: 100% pass rate across 211 tests (117 Python, 49 Rust, 45 TypeScript). A fresh, context-free AI agent reproduced the full integration twice from the published docs; every gap it found is fixed here.
🌟 What's New in v0.1.12
✅ Green runs never block or escalate: a strict, deterministic success detector recognizes passing summaries from pytest, vitest, jest, cargo, go, mocha, rspec and unittest, returning
category: "no_failure"with exit0and zero API calls. Real failures always veto the shortcut (1 failed,FAILED, tracebacks, panics, dependency/transient errors). This fixes false failures in pre-commit/husky recipes for JS and Rust projects.🪝 Working pre-commit integration:
jev-test-gatenow takes your test command asargs(the runner decides, Jev advises) through a console entry point that works from any consumer directory (a shell wrapper remains available for non-pre-commit users), andjev-harness init --gitgenerates a runner-aware hook (npm/pytest/cargo) that usespython3, never overwrites an existing hook, and records the detected command.🔐 State files hardened:
~/.config/jevis created0700andsession.json/ the lock file are written0600(POSIX), so error snippets are no longer world-readable.📖 Documentation: provider access and API-key acquisition for every backend (TypeSafe console, OpenCode Zen, Command Code, OpenRouter alpha, Vercel AI Gateway) with a verification date and re-check instructions for agents, plus an explicit privacy matrix (what leaves the machine in live vs offline mode).
🧪 207-Test Battery: 100% pass rate across 207 tests (113 Python, 49 Rust, 45 TypeScript).
🌟 What's New in v0.1.11
🐛 Fixed a v0.1.10 classification regression: a bare
RuntimeError:/ValueError:/TypeError:line no longer masks a concrete dependency or transient root cause. Logs such asRuntimeError: ... Caused by: ModuleNotFoundErrorandRuntimeError: ... Timeoutare triaged asenv_missing/flaky_transientagain (skip_llm=true), while real logic exceptions without an env/flaky root cause still escalate asdeep_logic.🌐 Busy ports are flaky:
Address already in use/EADDRINUSE/port already in use(EN, PT-BR, ES) now classify asflaky_transient, matching the documented behaviour.⚙️
.jev.jsonis honored end-to-end:model,skip_llm_thresholdandabort_thresholdtake effect across Python, TypeScript and Rust (previously the file was scaffolded but silently ignored).🔐 No more auth CI crashes: HTTP
401/403from any provider degrades to offline simulation with a stderr warning andis_mock=true, instead of raising a traceback. Other failures (e.g. HTTP500) still surface as errors.📊 Honest ROI metrics: the savings counters are labeled as heuristic estimates, the assumption model is printed, and
--jsonexposesestimates_are_heuristic.📖 OpenRouter documented as alpha: it requires approved alpha access; the endpoint and
typesafe/jev-1.13model are not publicly listed, so it is no longer presented as a turnkey provider.🧩 Contract parity: Standardized
workflow_phase(research,ask,plan,execute,verify,complete) across CLI, SDK, and MCP outputs fornudge-gate.🚦 Release gate hardened:
release.ymlnow requires the full CI matrix (Linux/macOS/Windows, Python 3.9-3.13, Node 18-22, Rust) through a reusable workflow gate before publishing to PyPI, npm or crates.io — a red CI can no longer ship a release.🧹 Zero clippy warnings across the Rust workspace.
🧩 Tri-runtime heuristic parity: the TypeScript engine now scores an explicit assertion exactly like Python and Rust, so the rules/04 precedence snippet (
FAIL+ cross-lineExpected:/Received:containing a module name) isdeep_logic/skip_llm=falseon every runtime. Assertions spanning multiple lines are detected, andPort 8080 is already in use-style messages areflaky_transient.🧪 197-Test Battery: 100% pass rate across 197 tests (107 Python, 47 Rust, 43 TypeScript) at v0.1.11; superseded by the 211-test battery in v0.1.12.
🌟 What's New in v0.1.10
🛡️ Native Rust MCP Server (
packages/rust/src/mcp.rs): High-performance JSON-RPC 2.0 stdio MCP server for the Rust runtime (jev mcp/jev-harness mcp), providing full feature parity with Python and TypeScript across all 6 decision gates.⚡ Atomic Concurrency File Locking (
fcntl.flock): Robust transactional file locking insession.pyguaranteeing zero metric corruption and 0.0% counter data loss under heavy concurrent agent executions.🔄 OpenCode Zen Live Auth Fallback: Automatic graceful fallback to offline heuristic simulation on HTTP 401/403 when community dummy/zen keys are used, eliminating unhandled crashes in production.
🛠️ CLI Subcommands Unification (
init&metrics): Full cross-runtime availability ofinit(repo scaffolding & agent adapter generation) andmetrics(session ROI and token telemetry) across Python, TS, and Rust.📐 Rigid JSON Schema & Contract Parity: Ensured dual availability of
action_recommendation+recommendationandreasoning_summary+summaryacross all CLI--jsonoutputs and MCP tool invocations.🧪 149-Test Battery: 100% test pass rate across 149 tests (86 Python, 34 Rust, 29 TypeScript) with sub-100µs latency in Rust.
🌟 What's New in v0.1.9
🆓 Command Code Free Provider Integration: Zero-cost live inference via Command Code ($0.00/M Deal - model
typesafe/jev) with auto-auth detection from~/.commandcode/auth.json(CMD_API_KEY).🚪 6th Semantic Decision Gate (Continuation Nudge): Evaluates whether an autonomous agent paused prematurely with unfinished work or unverified changes, injecting targeted continuation nudges while vetoing nudges when waiting on user permission.
🔌 CLI & MCP Support: Subcommand
nudge-gate(alias:nudge) and MCP tooljev_should_nudge_continuation.
🌟 What's New in v0.1.8
🏛️ Astra-Ares v0.2.1 Protocol Parity: Native endpoints for OpenRouter (
https://openrouter.ai/api/alpha/decisions, modeltypesafe/jev-1.13withprovider: { only: ["typesafe"], allow_fallbacks: false }), Vercel AI Gateway (https://ai-gateway.vercel.sh/v1/evaluate, modeltypesafe-ai/jevwithproviderOptions: { gateway: { only: ["typesafe-ai"] } }supportingVERCEL_API_KEY,AI_GATEWAY_API_KEY,VERCEL_AI_GATEWAY_API_KEY), TypeSafe AI direct (https://api.typesafe.ai/v1/systemone), and OpenCode Zen (https://opencode.ai/zen/v1/systemone).⚡ 8-Level Reasoning Scale & Provider Dialects: Support for
none,minimal,low,medium,high,xhigh,max,ultravia--supported-efforts/supportedEfforts. Proper CoT disable mapping (none/minimal) across DeepSeek (extra_body.thinking.type: disabled), Qwen (enable_thinking: false), Anthropic (thinking.type: disabled), Kimi (extra_body.thinking: false), and MiMo (thinking.type: disabled).🔄 Multi-Generation Effort Leasing: Multi-step stability leasing (
lease_steps/leaseSteps: 1, 2, 5, 10 generations) with safe clamping (max_lease_steps >= 1), allocating 5 for mechanical tool calls, 1 for errors/tracebacks, and 2 for standard tasks.🔒 Zero-Trust Secret Redaction: Automatic masking (
[REDACTED]) ofBearer ...,sk-...,vck_..., and active API keys in all HTTP error messages and diagnostics across Python, TypeScript, and Rust.🌐 Polyglot & Multilingual Semantic Parity: Uniform classification across 9 programming languages (Python, TypeScript/Node, Rust, Go, Java, C#, Ruby, C++) and 3 natural languages (EN, PT-BR, ES).
🛡️ Red-Team Defense Hardening: Collision resistance in
verify_step_completion(never verifying real failures despite positive tokens) andshould_abort_trajectory(never aborting forward progress), plus prompt injection mitigation inmodulate_reasoning_effort.🔌 CLI & MCP Tooling Evolution:
--supported-effortsand--max-lease-stepssupported across all 3 CLI binaries and native MCP server schemas withlease_stepsin JSON responses.🧪 Expanded 138-Test Battery: 100% test pass rate across 138 tests (79 Python, 31 Rust, 28 TypeScript) with latency p99 < 90µs in Rust.
🌟 What's New in v0.1.7
📊 Empirical Tri-Runtime Latency Benchmarks & Verified < 500µs Guarantee: Added comprehensive empirical latency benchmark tables ($p50$, $p95$, $p99 < 500\mu\text{s}$) across Python, TypeScript, and Rust, verified with 1,000-iteration automated test suites.
⚡ Rust Engine Optimization: Regex caching with
std::sync::LazyLockand zero-allocation fallback client references in gates, dropping Rust triage $p99$ to 37.5µs and pure System One simulation to 1.3µs.🛠️ TypeScript CLI & Native MCP Full Parity: Hardened Unix semantic exit code handling (code
2on missing command or invalid arguments) and added dual camelCase + snake_case JSON field support across CLI commands and native MCP tools (skip_llm,should_abort,provider_params,is_reasoning_supported).🚀 In-Memory Heuristic Acceleration: Added
record_session: bool = Falseflag to decouple memory-only gate calls from filesystem I/O, achieving sub-150µs $p99$ latency in Python while preserving full telemetry on CLI executions.🧪 Complete 123-Test Battery: 100% test pass rate across 123 tests (73 Python, 26 Rust, 24 TypeScript) with zero compiler warnings.
🌟 What's New in v0.1.6
🛡️ Expanded Direct Model Safeguards: Automatically identifies non-reasoning direct execution models (
gpt-4o,gpt-4o-mini,gemini-2.5-flash,gemini-2.0-flash,claude-3-5-haiku,qwen-2.5-coder,llama-3.3, etc.), injecting{}to prevent fatal HTTP 400 Bad Request parameter rejections across all 3 runtimes.🧠 Active Session Context Tokens & Cache Risk Warning: Added
--session-context-tokensparameter to CLI and MCP tool. Emits proactiveHIGH CACHE RISKadvisory when session context exceeds 30,000 tokens to protect Prompt Cache (KV Cache) prefix hit rates.💻 TypeScript CLI
reasoning-effort& Zero-Dependency Native MCP Server: Added full native CLI command support (npx @ismaelsoilet/jev-harness reasoning-effort) and zero-dependency stdio MCP server (npx @ismaelsoilet/jev-harness mcp) matching Python and Rust.🧪 Hardened Adversarial Heuristics & 122-Test Battery: Hardened regex matching for Jest and Pytest logs where
AssertionErrorcontains module names (guaranteeingdeep_logicandskip_llm=false), preventing premature aborts on forward-progress implementation steps, and reaching 100% pass across 122 tests (73 Python, 25 Rust, 24 TypeScript).🌐 OpenRouter & Provider Enhancements: Updated OpenRouter fallback model to
google/gemini-2.5-flashwith direct provider configuration options.
🌟 What's New in v0.1.5
⚡ Astra-Jev Dynamic Reasoning Effort Governance: Pioneered from Vechen (@miu21590) with native Jev System One semantic governance. Modulates per-generation reasoning effort in <500µs local (remote latency depends on the provider; provider-claimed 70ms floor).
🌐 2026 Frontier Models & Dialect Compiling: Full provider parameters dialect mapping across OpenAI (
reasoning_effort), Anthropic (thinking.type: adaptive,output_config.effort), Gemini (thinking_level), DeepSeek (extra_body.thinking: enabled, preservingreasoning_content), Qwen (enable_thinking,thinking_budget), Kimi, and Xiaomi MiMo.🛡️ Zero-Failure Model Safeguards: Automatically detects direct single-pass models (
gpt-5.6-luna,gemini-3.8-live) to prevent fatal HTTP 400 Bad Request rejections.🧠 Prompt Cache (KV Cache) Advisories: Includes cache-safety recommendations to prevent prefix cache invalidation across 100k+ token sessions.
🔌 Universal MCP Tool & CLI Subcommand: Added
jev_modulate_reasoning_effortMCP tool andjev-harness reasoning-effort/astra-jevCLI command with telemetry tracking.🧪 Comprehensive 94-Test Battery: 100% test pass rate across Python (62 tests), Rust (18 tests), and TypeScript (14 tests) with zero external runtime dependencies.
🌟 What's New in v0.1.4
🎨 100% Visual & Badge Standardization: Standardized all badges across PyPI, npm, and Crates.io with official logos, official colors, docs.rs and Sigstore provenance links.
🔐 Zero-Token npm Trusted Publishing: Fully migrated npm release workflow to OpenID Connect (OIDC) Trusted Publishing with Sigstore signed provenance statements. No expiring tokens required.
⚡ Cross-Runtime CLI Unification: Standardized subcommands (
test-gate,triage,abort-check,abort,route,verify,status) and argument parsing (positional or flags) across Python, TypeScript (@ismaelsoilet/jev-harness), and Rust (jev).🧪 Comprehensive 82-Test Battery: 100% test pass rate across Python (54), TypeScript (13), and Rust (15), covering adversarial inputs, Portuguese & English tracebacks, and UTF-8 truncations.
📦 Tri-Registry Synchronization: Crates.io, npm, and PyPI synchronized with automated GitHub Actions CD release.
🛡️ Cross-Platform Test & Runtime Hardening: Resilient HTTP/network handling across Python 3.9 through 3.13, Node 18 through 22, and Rust stable.
🙏 Acknowledgements
Astra-Ares by Vechen (@miu21590): Inspiration for per-generation reasoning effort modulation, multi-generation effort leasing (
lease_steps), provider dialect mapping, and zero-trust secret redaction in error messages.TypeSafe AI: Creators of the Jev System One decision architecture.
📄 License
Distributed under the MIT License. See LICENSE for more information.
Available Tools
6 toolsjev_abort_checkA
Guards against doom loops, dead-ends, circular retries, and destructive refactors. Evaluates proposed plan against recent attempt history before burning tokens.
| Name | Required | Description | Default |
|---|---|---|---|
| proposed_step | Yes | The next proposed plan, code modification, or architectural direction. | |
| recent_attempts_summary | No | Summary of previous failed attempts, errors encountered, or circular patterns. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the burden of behavioral disclosure. It mentions 'before burning tokens' and 'Guards against' which implies a check operation, likely non-destructive. However, it doesn't specify what the tool returns (e.g., a go/no-go decision) or any side effects. The description adds some context but lacks critical behavioral details like whether it modifies anything or requires historical data.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, both front-loaded with key information. The first sentence lists the problems it guards against, and the second explains the core evaluation action. It's efficient with no fluff, every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity of the tool (no output schema, no annotations, abstract purpose), the description is incomplete. It doesn't explain what the tool returns or how an agent should interpret the result. It mentions 'before burning tokens' but not whether it returns a boolean, a verdict, or a modified plan. With no output schema and sparse annotations, a more complete description is needed for reliable invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Although schema coverage is 100%, the description adds crucial context for the parameters: it explains that 'proposed_step' is the plan to evaluate and 'recent_attempts_summary' is the history used for evaluation. This goes beyond the schema's basic descriptions by framing them in the context of avoiding doom loops, making the parameters' purpose clearer.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: to guard against problematic loops and evaluate plans against attempt history. However, it does not explicitly name the verb 'check' but uses 'Guards against' and 'Evaluates', which is specific enough. The resource is the proposed plan and attempt history, though it's a bit abstract but distinct from siblings.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use it: before burning tokens on a proposed step, especially in scenarios with recent failures. It suggests a preventive context but doesn't explicitly exclude alternatives or mention when not to use it. Given the sibling tools, it's clear that this is a pre-execution guard, but no explicit routing is provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
jev_modulate_reasoning_effortA
Dynamically modulates reasoning effort (low, medium, high, etc.) and stability lease steps for the immediate generation step. Maps exact parameters for OpenAI (GPT-6 Astra/o3), DeepSeek (V4.1-Flash/R1), Qwen (3.8 Max), Anthropic (Claude Fable 5.1), and Gemini (3.8 Thinking). Eliminates reasoning token waste and cuts multi-minute delays on mechanical tool calls.
| Name | Required | Description | Default |
|---|---|---|---|
| model | No | Optional model identifier to check for direct non-reasoning compatibility. | |
| context | Yes | The command, prompt, or next step to evaluate. | |
| provider | No | Target provider (openai, deepseek, qwen, anthropic, gemini, kimi, mimo). Default: openai. | |
| max_lease_steps | No | Optional upper bound for generation stability lease steps (default: 10). | |
| supported_efforts | No | Optional list of supported effort levels (e.g. ['none', 'minimal', 'low', 'medium', 'high', 'xhigh', 'max', 'ultra']). | |
| session_context_tokens | No | Optional active prompt tokens in session context to evaluate prompt cache risk. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must fully disclose behavior. It states that it 'dynamically modulates' and 'maps exact parameters' for multiple providers, and mentions the outcome of reducing waste and delays. However, it does not explain the mechanism, potential side effects, or whether the change is temporary or persistent. The description is informative but lacks depth on what happens during execution.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, highly concise, and front-loaded with the primary action. It wastes no words and efficiently conveys the tool's core function, provider coverage, and a key benefit. Structure is optimal.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has six parameters, full schema coverage, no output schema, and no annotations, the description covers the main purpose and some context (mechanical calls, provider mapping). However, it omits details about when to use this tool vs. siblings, how the tool interacts with the generation pipeline, and what the expected result or return value is. It is adequate but leaves gaps that an agent might need to resolve.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 100% coverage, so all six parameters have individual descriptions. The tool description adds context about provider mapping and effort levels but does not elaborate on any specific parameter beyond what the schema already provides. With full schema coverage, the baseline of 3 is appropriate; the description adds minimal extra value for parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: it dynamically modulates reasoning effort and stability lease steps for the immediate generation step. It uses specific verbs and a specific resource, and the focus on reasoning effort distinctly separates it from sibling tools like triage, routing, or verification, even without naming them.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for mechanical tool calls to avoid token waste and delays, which gives some context. However, it does not explicitly state when to use this tool versus alternatives, nor does it mention any exclusion criteria or prerequisites. The guidance is inferred rather than direct.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
jev_route_taskA
Routes programming task to the minimal sufficient model tier (deterministic script, lightweight fast flash model, or heavy frontier reasoning model) to optimize cost and latency.
| Name | Required | Description | Default |
|---|---|---|---|
| task_description | Yes | Clear description of the task, bug to fix, or feature to implement. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
There are no annotations, so the description carries the behavioral burden. It discloses the decision categories (deterministic script, fast flash model, heavy frontier reasoning) and the cost/latency objective, but it does not say what the tool returns, whether it executes the task, or what side effects or errors may occur.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single front-loaded sentence with zero filler. The core function, target tier types, and objective are all conveyed compactly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a one-parameter tool with no annotations and no output schema, the description is reasonably complete for invocation, but it leaves the return value and its usage relationship to sibling tools unstated. An agent would know what to pass but not clearly what to expect back.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%: task_description is already described as the task, bug, or feature. The description adds the scope that the input should be a 'programming task,' which is useful, but no format, length, or example is provided beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description names a specific verb ('Routes'), a specific resource ('programming task'), and a concrete target ('minimal sufficient model tier') with three enumerated tiers. This clearly distinguishes it from related siblings like jev_modulate_reasoning_effort, which would change reasoning effort rather than select a model tier.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit when-to-use guidance or alternatives are given. The goal 'to optimize cost and latency' implies a use case, but the agent is not told when to choose this over jev_modulate_reasoning_effort or other sibling tools, nor what circumstances make routing unnecessary.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
jev_should_nudge_continuationA
Evaluates whether an autonomous agent paused prematurely with unfinished work or unverified changes (Workflow phases: research, ask, plan, execute, verify, complete + CommandCode Jev Nudge protocol). Vetoes nudges when waiting on user permission/input or when the previous nudge produced no progress.
| Name | Required | Description | Default |
|---|---|---|---|
| threshold | No | Optional probability threshold for nudge/waiting/progress (default: 0.5). | |
| transcript_tail | Yes | Recent agent transcript tail or turn output. | |
| previous_nudge_summary | No | Optional summary of the previous nudge to check if real progress was made. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden of behavioral disclosure. It discloses the evaluation and veto behavior, but does not state side effects (e.g., whether it mutates state), required permissions, or the exact return format. It adds useful context about workflow phases but leaves key behavioral details unspecified.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loaded with the primary purpose, and contains no filler. Every phrase adds value, including the workflow phases and veto conditions.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a decision tool with three parameters and no output schema, the description covers the main evaluation logic and veto conditions. It does not explicitly state the return type (e.g., boolean or score), but 'evaluates whether' implies a decision. The workflow phase list provides useful context, making it sufficiently complete for an agent to call it correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all parameters. The description adds minimal parameter-specific meaning, though it references the concept of a previous nudge, which aligns with previous_nudge_summary. This meets the baseline for a schema-heavy description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool evaluates whether an agent paused prematurely, with specific reference to workflow phases and a nudge protocol. It also mentions veto conditions, making the purpose distinct from sibling tools like jev_abort_check or jev_verify_completion.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context on when the tool applies (when an agent might be paused prematurely) and specifies conditions that veto a nudge (waiting on user input or no progress). However, it does not explicitly contrast with sibling tools or state when NOT to use it, though the conditions imply usage boundaries.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
jev_triage_test_failureA
Triages test traceback, compile error, or runtime failure using Jev System One (70-300ms, zero-generation). Returns root cause category, skip_llm flag (true if resolvable deterministically without frontier LLM), and immediate action recommendation.
| Name | Required | Description | Default |
|---|---|---|---|
| failure_log | Yes | Raw test failure output, stack trace, or compiler error log. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations are absent, so the description carries the full burden of behavioral disclosure, and it delivers: it reveals a 70-300ms latency bound, states the tool is zero-generation (no frontier LLM call), and explains the semantic of the skip_llm flag (true when resolvable deterministically). It stops short of declaring safety/side-effect behavior or error handling, but for a diagnostic triage tool this is strong coverage.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, roughly 50 words, with the primary verb and resource front-loaded in the first sentence and the return contract in the second. Every piece — input types, mechanism, latency, and output semantics — earns its place, and nothing is repeated from structured fields.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema and no annotations, the description correctly compensates by explaining what the tool returns (root cause category, skip_llm flag with meaning, action recommendation) and its execution characteristics. The tool is simple (one parameter, fully documented), so the only missing context is limited to untested edges like malformed input handling, which is minor.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, with failure_log already described as 'Raw test failure output, stack trace, or compiler error log', so the schema does the heavy lifting. The tool description echoes these input types in its purpose sentence, reinforcing alignment but adding no new format, constraint, or preprocessing information. Per the baseline rule for high schema coverage, 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb ('Triages') and a specific resource domain ('test traceback, compile error, or runtime failure'), which lays out exactly what the tool consumes. The output fields it names (root cause category, skip_llm flag, action recommendation) further pin down the tool's role. None of the siblings (abort_check, route_task, verify_completion, etc.) perform triage of failure logs, so an agent can distinguish this tool without opening the schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives clear when-to-use context by enumerating the specific inputs it handles: test tracebacks, compile errors, and runtime failures. It does not name explicit alternatives or state explicit exclusions, but the input domain is stated explicitly enough that routing against the listed siblings is straightforward.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
jev_verify_completionA
Calibrates step completion against acceptance criteria using typed rubric scoring. Checks if evidence is sufficient to declare done without launching expensive extra review loops.
| Name | Required | Description | Default |
|---|---|---|---|
| produced_output | Yes | The evidence, test results, code diff, or output produced. | |
| acceptance_criteria | Yes | Explicit requirements, constraints, or definition of done. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It discloses that the tool performs a calibration without launching expensive extra review loops, giving insight into its efficiency but not detailing side effects or return structure. However, since it's likely a read-only evaluation, the lack of further detail is acceptable.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences with zero wasted words, front-loading the core purpose and adding a performance-related benefit. It is concise and structured well for LLM consumption.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a two-parameter verification tool with 100% schema coverage and no output schema, the description is sufficient. It clarifies the tool's role and efficiency without needing to explain return values. However, it could mention typical return formats, but that may be a minor gap.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema description coverage is 100%, so both parameters are already described clearly. The description adds context by mentioning 'evidence' and 'acceptance criteria' but does not add significant new semantics beyond what the schema provides. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's function: calibrating step completion against acceptance criteria using typed rubric scoring, and it distinguishes itself from vague alternatives by mentioning 'typed rubric scoring' and the specific purpose. This is a specific verb+resource combo that sets it apart from sibling tools like jev_triage_test_failure.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use it: when you need to verify completion without launching expensive extra review loops. It clearly states the tool avoids unnecessary additional processes, but it does not explicitly exclude any sibling tools or state when not to use them. Clear context is provided, but explicit alternatives are missing.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
6 tool updates
v0.2.0- First observed
jev_abort_check - First observed
jev_modulate_reasoning_effort - First observed
jev_route_task - First observed
jev_should_nudge_continuation - First observed
jev_triage_test_failure - First observed
jev_verify_completion
TDQS
Scored across 6 tools
Each tool addresses a distinct decision point in the agent lifecycle: failure triage, abort protection, task routing, completion verification, reasoning effort modulation, and continuation nudging. There is no meaningful overlap; even the related pairs (abort vs. nudge, route vs. modulate) focus on different moments and inputs.
All tools share the 'jev_' prefix and use snake_case with a verb-like leading word, e.g., triage_test_failure, route_task, verify_completion. The only slight deviation is 'jev_should_nudge_continuation', which uses a modal verb phrase rather than a direct imperative, but it remains clear and predictable.
Six tools is a tight, purposeful set for a decision-harness server. Each tool covers a distinct optimization decision without redundancy or bloat, making the count well-scoped for the domain.
The harness covers the main meta-decision points: triage failures, prevent wasted work, route to cost-effective models, verify completion, modulate effort, and decide on nudges. A minor gap might be a tool to record outcomes or update state, but the core lifecycle appears well covered.
Maintenance
Related MCP Connectors
Pre-execution governance for AI agents. Deterministic PASS/FAIL/REVIEW verdicts, replayable proof.
Deterministic runtime safety for AI agents: scan PII, gate tool actions, verify LLM output.
The WAF for agents. Pattern-based + heuristic firewall scans prompts, RAG documents, tool argume...
AgentGuard — 20-tool AI safety MCP: policy preflight, risk scoring, audit logging, rate limits.
Related MCP Servers
- AlicenseNot gradedqualityDmaintenanceRuntime safety guardrails for AI coding agents. Checks file access, validates shell commands, and scores your repo's AI safety — all via MCP.4 npm8MIT

aperion-shieldofficial
FlicenseAqualityAmaintenanceLocal guardrail proxy for AI coding agents. Wraps any MCP server (stdio or HTTP/SSE) and blocks destructive tool calls before they execute, with TOFU catalog pinning against rug pulls and tool-poisoning/result-injection scanning. Single Rust binary, Apache-2.0.1411-- AlicenseAqualityAmaintenanceRuntime governance and budget guardrails for Claude Code, Cursor, and autonomous AI agents. Enforces per-session spend caps, verifier safety gates, and runaway loop prevention.24265 npm191Apache 2.0
- AlicenseNot gradedqualityAmaintenanceA runtime gate for coding agents. Blocks the tool calls that wreck a repo (force-push main, rm -rf, secret exfiltration, CI wipe) and lets normal build and commit work through. Machine-checked git-branch core (z3); the rest is high-precision heuristics. Tested on 3,790 real CI commands, 0 false blocks.1MIT