decide

Your agent reasons. Jev sorts.

One MCP tool for bulk decisions over files, logs and JSONL.

Get started · Benchmarks · Tool reference · MIT

Give your agent a path and a rubric. decide sends each record to Jev, writes decisions locally, and returns counts and paths. Your agent reviews the exceptions and processes the rest with code.

Files + rubric → Jev → accepted decisions + review queue → your agent

Keep bulk input out of the expensive agent's context. Tune review to the quality your task needs; there is no universal “review 5%” setting.

Get started

You need uv and a TypeSafe key. No clone or Git required: this command installs the small release wheel, without the benchmark archive.

uvx --python 3.11 --from https://github.com/alsoleg89/decide/releases/download/v0.1.1/decide_mcp-0.1.1-py3-none-any.whl decide-mcp

Use it as your MCP client's launch command. Set TYPESAFE_API_KEY and DECIDE_ROOT to an absolute data-directory path in the server environment. Missing or relative roots are rejected before any files are read. Codex setup · Claude setup · Verify the wheel

Then ask your agent:

Use decide on feedback.jsonl to find feature requests. Pass the path without reading the full file first. Use yes/no criteria, a global cutoff of 0 and a cutoff of 0.9 for no. Review every queued case, save one final label per ID, and verify complete coverage.

{
  "question": "Does this review request a new or changed capability?",
  "criteria": {"yes": "Requests a new or changed capability", "no": "Does not"},
  "source": {"kind": "jsonl", "paths": ["feedback.jsonl"]},
  "confidence_threshold": 0,
  "confidence_thresholds": {"no": 0.9}
}

Each JSONL row is {"id":"review-1","content":"Please add CSV export."}. The example uses the UX study's routing. Validate the rubric and cutoffs on your own labeled sample; a confidence score is not a probability of correctness.

Related MCP server: Jev MCP Server

What we measured

Real UX feedback and developer issue triage, with complete output files and all model turns counted. Cost includes Jev, agent reading, tools, retained history, writing and the final answer. These are guided API loops with 25-record batches, not native Codex/Claude sessions or optimized baselines; larger standalone-model batches have not been compared.

Luna used reasoning none. Both Luna policies failed the frozen all-metrics quality gate. The mini UX accuracy gain has a paired 95% interval of +4 to +8 percentage points; statistics and limitations are published alongside every prediction, mistake and cost calculation.

Review buys a different error tradeoff. In the mini UX study it raised Jev-only feature recall from 67.5% to 84.7%, while lowering accuracy from 91.4% to 90.2%. More review is not automatically better. Review effect · Losing tasks and baselines

The broader workflow study includes four UX labels and issues from five projects. Results vary by task; sorting labels is not UX synthesis, prioritization or incident diagnosis.

Full audit trail

• results.jsonl: every decision, confidence, probabilities, usage and error.

• review.jsonl: complete original records needing review, including provider failures.

• request.json and summary.json: rubric, routing settings and run totals.

Files live in DECIDE_ROOT/.decide/<run-id>/. The default tool response has no input previews. Oversized items go intact to review; corrupt provider responses do not cancel the other records. decide.py is the entire server.

Selected content is sent to TypeSafe. Scope your input paths accordingly. Limits, timeouts and retry costs

Try your own workload

Label a sample, run the evaluator, and compare errors, recall and complete cost. Share a small, reproducible issue when a task works well or fails; both belong in the benchmark suite.

Download v0.1.1 · Local verification · Development