Rank eval results
modelfinder_rank_eval_resultsRank models from a finished eval run by cost per passing call and recommend a cost-effective model that meets the pass threshold. Returns the ranked cascade and dominated models dropped.
Instructions
Rank the models in a finished eval run (the results.json object written by eval_models.py) and recommend the cheapest one that met the pass threshold, ranked by cost per passing call. Use after an eval run; to test a single reply use modelfinder_check_output, and to turn the ranking into a fallback chain use modelfinder_generate_routing. Read-only and local: no network calls, no files written. Returns the recommendation, the ranked list, the cost/reliability cascade and models dropped as dominated (a model is dropped when it costs at least as much per passing call as a model earlier in the cascade and is not more reliable; ties on cost or reliability count as dominated).
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| results | Yes | The full results.json object written by eval_models.py. | |
| response_format | No | markdown (default) for a readable summary, json for the full machine-readable result. | markdown |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
| ranked | Yes | ||
| cascade | Yes | ||
| dominated | Yes | ||
| recommendation | Yes |