Cheapest LLM that passes each task: cost per successful task and pass rate on public datasets.
Scored across 2 tools
The two tools form a canonical list/get pair: list_verdicts enumerates published verdicts and get_verdict retrieves the full detail for a single slug. There is no overlap in purpose, and get_verdict even tells the agent where the slug comes from.
Both tools follow the same verb_noun convention (list_verdicts, get_verdict) with snake_case and a shared domain noun. Fully predictable and readable.
Two tools is on the thin side, but the server's scope is a read-only browser over published verdicts, where list plus detail is the natural shape. Nothing extraneous is included, though the surface is minimal enough that one more query tool would not feel bloated.
The read-only lifecycle is fully covered: enumerate verdicts (with task-type filtering), then drill into cost, pass rate, candidates, and artifact links. Minor gaps exist for model-centric questions, e.g. looking up all tasks where a given model wins, which the agent would have to assemble by paging through list_verdicts.
The full verdict for one task: the cheapest model that clears the bar, its pass rate and cost per successful task in US dollars, the baseline model it is compared with, every candidate's status, plus links to the verdict page, the run file (JSON) and the script that re-runs it. Get the slug from list_verdicts.
| Name | Required | Description | Default |
|---|---|---|---|
| slug | Yes | Verdict slug from list_verdicts, for example sms-spam. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already cover the safety profile (readOnly, idempotent, non-destructive, closed-world), so the description's job is to add payload context, which it does well by naming the returned fields, the JSON run file and the re-run script. It does not say what happens for an unknown slug (error vs empty result), which is the remaining behavioral gap.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
One dense sentence plus one actionable directive, front-loaded with the core identity of the resource and followed by the return payload and the prerequisite. The enumerated clause list is long but each item carries distinct information; nothing is filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description must describe the return value and it does so comprehensively, naming the headline result, cost, baseline, candidate statuses and artifacts. Only error/not-found behavior is unaddressed, which is minor for a single-parameter read tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% and the schema already specifies that slug comes from list_verdicts with the example 'sms-spam'. The description repeats the same provenance hint without adding format, slug-shape or validation detail beyond the schema, so the baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb+resource ('The full verdict for one task') and then enumerates exactly what that verdict contains: cheapest passing model, pass rate, USD cost per successful task, baseline model, per-candidate status, and links to the verdict page, run JSON and re-run script. This is clearly a single-record fetch, distinct from the sibling list_verdicts.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The closing line 'Get the slug from list_verdicts' establishes the prerequisite dependency and implicitly routes the agent to the sibling when it lacks a slug. It does not state the inverse condition (use list_verdicts when you want to browse rather than retrieve one), so it stops short of full when/when-not guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Which LLM is the cheapest that still passes? Lists every published Clearbar verdict: the task, the pass bar, the cheapest passing model, its cost per successful task (retries and failures counted) and the page to check it. Optional filter by task type.
| Name | Required | Description | Default |
|---|---|---|---|
| task | No | Only verdicts of this task type, for example classification, extraction or tool-call. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, idempotentHint, and closed-world behavior, so the safety profile is covered. The description adds genuinely useful behavioral detail beyond that: that verdicts are "published" (a scoping/curation constraint), and that cost is measured per *successful* task with retries and failures counted — a non-obvious accounting rule an agent could not infer from structured fields.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with the decision question ("Which LLM is the cheapest that still passes?") followed by a compact enumeration of returned fields in one sentence. No filler, though the cost-accounting parenthetical is dense enough to slow the first read.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description carries the burden of describing return values and does so field-by-field, and annotations cover the safety profile of this read-only, idempotent call. The only remaining gap is routing guidance to the get_verdict sibling for single-task lookups.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% and the single parameter is fully documented with examples (classification, extraction, tool-call). The description restates it as "Optional filter by task type," adding only that omission means all types — consistent with the schema rather than additive to it, so baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
"Lists every published Clearbar verdict" gives a concrete verb and resource, and the description enumerates exactly what each row contains (task, pass bar, cheapest passing model, cost per successful task, verification page). The plural/"every" framing implicitly opposes it to the single-item sibling get_verdict, but that sibling is never named, so the separation is inferred rather than stated.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The only usage signal is "Optional filter by task type," which implies the browse-all use case. There is no explicit guidance on when to call this versus get_verdict for a specific task, and no stated preconditions or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
get_verdictlist_verdictsAI inference price index: token prices, cost per capability, open vs closed, hosts, cost per task.
Benchmark your own prompt on 100+ LLMs with real API calls; pick the cheapest model that passes.
Real-time, drift-verified LLM pricing across 31 providers with cost+quality model routing.
Pay-per-call ($0.01 USDC) model-routing for AI coding agents: which LLM to call, cost vs. quality.
Add one secure layer between your agents and this server.