# jev-local

Run [Kev](https://github.com/jaredpalmer/kev) decision models on an Apple Silicon Mac and call them from Claude through MCP.

Each call gives the model a text and a question. It returns a calibrated answer instead of generated text:

- **choose**: pick one of several options
- **yes_no**: answer a yes/no question
- **score**: place the text on an ordered scale

## Install

You need an Apple Silicon Mac and [uv](https://docs.astral.sh/uv/). Kev-4B needs about 16 GB of memory while loading.

```bash
git clone https://github.com/phykn/jev-local.git && cd jev-local
git clone https://github.com/jaredpalmer/kev.git vendor/kev && git -C vendor/kev checkout kev-1.0
uv sync --python 3.13
```

## Use

```bash
.venv/bin/kevctl start kev-4b                                     # uses only weights already in weight/
claude mcp add kev --scope user -- "$PWD/.venv/bin/kev-mcp"
```

Start a new Claude Code session and ask, for example: "Use kev to decide whether this ticket is urgent: ...".

- **As a Mac app:** run `scripts/build-app.sh` once, then open `JEV-LOCAL.app`, pick a model and press Start. Quitting the app stops the server.
- **Stop the model:** `kevctl stop kev-4b`
- **Check what is running:** `kevctl status`

kevctl does not download weights. To download a missing model, run it once with `HF_HUB_OFFLINE=0 .venv/bin/kevctl start <model>`.

Every answer includes a `confidence` value. When `low_confidence` is true, check the answer before acting on it.

## Models

The models are listed in `models.toml`. `kevctl`, the MCP server and the evaluation script all read this file.

To add a model, such as your own fine-tuned checkpoint, add an entry:

```toml
[models."my-4b"]
backend = "kev-http"
run = "checkpoints/my-4b"   # local checkpoint directory, or a Hugging Face id@revision
port = 8012
memory_gb = 16
min_confidence = 0.5        # answers below this confidence are marked low_confidence
```

## Layout

- `src/jev_local/`: the Python package. `registry.py` reads `models.toml`, `backends.py` calls a running Kev server, `ctl.py` is `kevctl` and `mcp_server.py` is `kev-mcp`.
- `app/`: the Mac app in Swift. `scripts/build-app.sh` builds it into `JEV-LOCAL.app`.
- `scripts/evaluate.py` and `evals/`: evaluation. `tests/`: pytest.

## Evaluate and test

```bash
.venv/bin/python scripts/evaluate.py --data evals/ko/smoke.jsonl   # accuracy, Brier score, coverage at 5% error
.venv/bin/pytest
```

`scripts/evaluate.py` runs every model that is currently running.

## License

MIT. Kev and its weights are licensed separately under Apache-2.0.