Checks if a cheaper model agrees with your GPT, Claude or Gemini outputs, and what routing saves.
Scored across 2 tools
The two tools serve clearly separate purposes: one is a read-only reference lookup of models and pricing, the other executes an audit computation. There is no plausible way to confuse listing model metadata with running a cost/agreement audit.
Both names follow a clean verb_noun snake_case pattern (list_models, run_cutover_audit). The convention is predictable and readable.
The purpose is narrow enough that two tools are defensible, but the surface feels thin — there is no tool to inspect samples, configure the audit, or apply routing. It sits at the borderline of minimal rather than well-scoped.
Core audit workflow is covered (see models, run audit), but obvious gaps remain: no way to view or manage the sample set, retrieve past audit results, or apply/configure the routing decision the audit recommends. Agents can read a report but cannot close the loop.
Current models with $/1M token prices and effort levels, usable as current.model.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
readOnlyHint=true already declares the safety profile, so the description only needs to add substance, and it does disclose the two payload categories (per-1M-token prices, effort levels) and that entries are consumable as current.model. It does not describe the return shape (array vs map), whether prices are live or cached, or whether any auth is required; with no output schema, that is a modest gap.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single clause with zero filler; the most actionable detail ('usable as current.model') is placed last for emphasis and every word carries information. Nothing is repeated from the schema or annotations.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a zero-parameter, read-only listing tool with no output schema and an unrelated sibling, the description covers intent, payload, and consumption well enough to call it correctly. Only the absence of any return-shape or freshness note keeps it from being fully complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool takes zero parameters, so there is nothing for the description to disambiguate; the baseline applies. The mention of '$/1M token' units and 'effort levels' adds useful semantic context about what domain values mean even though there are no inputs.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Names the resource precisely ('Current models') and the payload it carries ('$/1M token prices and effort levels'), which is enough for an agent to know this is a model catalog with pricing. It is a noun phrase rather than a verb+resource sentence, and it never explicitly says 'list', though the plural and the title make the operation unambiguous. The sibling (run_cutover_audit) is unrelated, so no differentiation is needed.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The clause 'usable as current.model' implicitly tells the agent this is how you discover valid values for a configuration field before setting it, which is a real usage cue. There is no explicit when-to-use or when-not statement, and no mention of prerequisites or alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Measures how often a cheaper model (Sage 2) agrees with your current LLM on your samples, the confidence threshold for routing, and the monthly cost now vs a hybrid. Returns the full report as JSON.
| Name | Required | Description | Default |
|---|---|---|---|
| mode | Yes | yesno: a yes/no decision per text. extract: fields pulled out of each text. | |
| usage | No | ||
| fields | No | extract only. Up to 12 fields: {"name": {"type": "string|number|integer|boolean|date|money", "definition": "..."}} | |
| target | No | Agreement you need on the calls the cheap model keeps. | |
| current | Yes | Your current model: {"model": "<id from GET /api/models>", "effort": "<level>"} or custom prices {"in": USD per 1M input tokens, "out": USD per 1M output tokens}. | |
| samples | Yes | Real inputs with the answer your CURRENT model gave. Array of objects or a JSONL/CSV string. yesno: {"text","label":"yes|no"}; extract: {"text","expected":{...}}. Each text ≤ 4000 chars. Min 5. Max 10 anonymous, 25 free on the website, 500 yesno / 200 extract with a pass. | |
| question | No | yesno only. The yes/no question your current model answers, e.g. "Is this ticket about billing?" |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotations already establish the safety profile: readOnlyHint=true, openWorldHint=true, and idempotentHint=false. The description adds that the operation produces a full JSON report and quantifies agreement, threshold, and cost, which is useful context beyond the annotations. It does not disclose turnaround time, sample-size throttling behavior, or why the call is not idempotent, which leaves a few behavioral gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences with no filler, and the main capability is front-loaded before the return-format note. The first sentence is dense, packing model agreement, threshold, and cost into one clause, but every phrase earns its place. Slightly harder to scan than it could be.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
This is a moderately complex tool with 7 parameters, nested objects, and no output schema. Because there is no output schema, the description should say more about the report structure than 'returns the full report as JSON'. It covers purpose and headline outputs but omits what the report contains and any limits on sample counts or runtime.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 86%, so the schema already documents mode, fields, target, current, and samples in detail, including the enum semantics. The description echoes the concepts (samples, confidence threshold, cost) without adding syntax or format detail beyond the schema. A baseline of 3 is appropriate when the schema already provides most of the meaning.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb and resource: it measures agreement between a cheaper model (Sage 2) and the caller's current LLM, derives a routing confidence threshold, and compares monthly cost. That is far more than a restatement of the name. It does not explicitly contrast with the sibling list_models, but that tool is clearly unrelated, so the only gap is the missing sibling routing.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The framing implies the scenario (evaluating a cutover from the current model to a cheaper one), and it mentions 'your samples' and 'your current LLM', which signals prerequisites. However, it never states when to choose this tool over alternatives, what to do first (for example, list_models to obtain a valid model id), or what conditions make an audit inappropriate. Usage is inferable rather than stated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
list_modelsrun_cutover_auditBenchmark your own prompt on 100+ LLMs with real API calls; pick the cheapest model that passes.
Real-time, drift-verified LLM pricing across 31 providers with cost+quality model routing.
Sourced AI-model pricing and capability data — compare and route to the cheapest capable model.
AI model routing on your own vendor keys: pick the best model per prompt, or route and run it.
Add one secure layer between your agents and this server.
