Skip to main content
Glama

tokmeter_leaderboard

Read-only

Rank AI models and providers by cost, cost-efficiency, tokens, cache and reasoning rates, or output volume to compare value, spot heavy usage, and filter by time scope or project.

Instructions

Rank models and providers by various metrics: total cost, cost-efficiency (cost per 1M tokens), total tokens, cache efficiency, reasoning usage, and output volume. Use this to find the best value models or identify the heaviest hitters.

Input Schema

NameRequiredDescriptionDefault
limitNoTop N to show (default: 10)
scopeNoTime scope: today, week (last 7d), month (calendar month), or all (default)
entityNoRank models or providers (default: models)
projectNoFilter by project name/path substring
rank_byNoRanking metric (default: cost)

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observedv1.19.3

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

readOnlyHint=true and openWorldHint=false already establish this as a safe, closed-world read, so the description's burden is light. It adds nothing about ordering semantics, which matters here: cost_efficiency presumably sorts ascending while cost sorts descending, and that is never stated. No mention of what a row contains either.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the capability and followed by the use case; nothing is padded. The metric enumeration is dense but earns its place by previewing the rank_by options.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

All parameters are optional with full schema coverage and annotations carry the safety profile, so little is strictly required. However, with no output schema, the description never says what a leaderboard entry returns (rank, entity name, metric value) or how ties/limits behave, leaving the return shape undocumented.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so all five parameters including three enums are fully documented in structured fields, giving a baseline of 3. The description does add a gloss on cost_efficiency ('cost per 1M tokens') that the schema lacks, but it omits 'records' from its metric list and adds nothing about scope, entity, or project filtering.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb+resource ('Rank models and providers') and enumerates the ranking metrics, which maps closely to the rank_by enum. It does not, however, distinguish itself from overlapping siblings such as tokmeter_efficiency, tokmeter_cache_efficiency, tokmeter_model_advisor, or the plain tokmeter_models/tokmeter_providers lists.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

'Use this to find the best value models or identify the heaviest hitters' gives a use context, but no when-not guidance and no named alternatives despite several siblings covering the same territory (efficiency, cache efficiency, model advisor). The agent must infer the boundary itself.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.