Skip to main content
Glama

compare_models

Read-onlyIdempotent

Compare LLM models side-by-side for a given token budget, rank them by estimated cost or time, and review quality tiers to choose the right model for a task.

Instructions

Compare all LLM models side-by-side for a given token budget.

Ranks models by estimated cost or time. Shows quality tier for each model. Use when choosing which model to use for a task.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "tokens": "<tokens>"
    },
    "name": "compare_models"
  }
}

Input Schema

NameRequiredDescriptionDefault
tokensYesTotal number of tokens to estimate across all models.
sort_byNoSort models by cost (default) or estimated time.cost
tool_callsNoNumber of tool calls expected.
reasoning_depthNoExpected depth of chain-of-thought reasoning.moderate

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed1 schema field changedv0.8.0
    • removedInput schema / additionalProperties
      Removed value: -false
  2. First observedv0.1.0

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare this a safe, idempotent, read-only, closed-world operation, so the safety profile is fully covered elsewhere. The description does add that figures are *estimated* cost/time rather than measured, which is meaningful context, but it says nothing about how many models are returned, how quality tier is derived, or how reasoning_depth/tool_calls shift the numbers.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Four short sentences, front-loaded with the core action, with no filler or hedging. The only soft spot is mild overlap between "Compare all LLM models side-by-side" and "Ranks models by...", which restate the same idea twice.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema, so the description is the only place return values can be explained; it says ranks, cost, time, and quality tier, but never defines what a "quality tier" is or what happens to reasoning_depth and tool_calls in the estimate. For a read-only comparison tool with simple params this is workable, but it leaves real gaps an agent would hit at invocation time.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With schema description coverage at 100%, all four parameters (tokens, sort_by, tool_calls, reasoning_depth) are already fully documented in the schema. The description only echoes the sort_by behavior ("by estimated cost or time") and never mentions tool_calls or reasoning_depth, so it adds no meaning beyond the structured fields. Baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific verb and resource ("Compare all LLM models side-by-side") plus the scoping input (a token budget), so the agent knows exactly what the tool produces. It stops short of differentiating itself from near-neighbors like token_cost_estimate or token_time_bridge, which it partly overlaps with, so it is clear but not sibling-aware.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives an explicit triggering condition: "Use when choosing which model to use for a task." That is a clear when-to-use context, but it offers no exclusions or named alternatives for closely related cost/time estimation tools, leaving the agent to infer boundaries.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.