lbb_model_choice_manage
Model choice, writes. start tests a candidate model on the uses in jobs (without jobs, every use it can do that has checked calls): the server answers those checked calls again with the candidate and scores it and the model in use against…
Instructions
Model choice, writes. start tests a candidate model on the uses in jobs (without jobs, every use it can do that has checked calls): the server answers those checked calls again with the candidate and scores it and the model in use against the same ground truth. It changes no model and spends at most the trial budget ($3.00 by default) of model and judge cost; an open trial of the same use and candidate comes back as it is. Pick candidates from lbb_model_choice action=options (provider anthropic with an effort, or typesafe jev-latest). A trial runs by itself: poll lbb_model_choice action=trial until its status is no longer running. stop ends a trial; it keeps what it compared. switch makes the trial's use run its candidate on this graph from the next call on; only a trial that qualifies and compared with the model in use switches (ask to a Claude model, route to a Jev model). Show the user the trial's report (score, difference and interval, cost per call, time) and switch only what the user confirmed. revert puts the use back on LBB's model.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| job | No | revert: the use to put back on LBB's model. | |
| days | No | start: the days the trial takes new checks (default 14). | |
| jobs | No | start: the uses to test it on. | |
| graph | No | Graph to target; defaults to the connection's graph | |
| model | No | start: the candidate model id, e.g. claude-haiku-5-5 or jev-latest. | |
| action | Yes | ||
| detail | No | Response detail level. Defaults to compact. | |
| effort | No | start: the effort of a Claude candidate. | |
| target | No | start: the calls to compare (default 40). | |
| provider | No | start: the candidate's provider. | |
| trial_id | No | stop / switch: the trial id. |