Skip to main content
Glama

The Aggregate — LLM benchmark aggregate

Aggregate leaderboard

get_leaderboard
Read-onlyIdempotent

Top of the cross-benchmark aggregate ranking: every model placed on one rating scale by an IRT model fit over public benchmark leaderboards (call about_the_aggregate for the current coverage counts). One row per model by default, fused across reasoning-effort settings. Supports paging via limit/offset.

Input Schema

NameRequiredDescriptionDefault
limitNoRows to return (1-100, default 25).
offsetNoRows to skip from the top (default 0).
include_variantsNoRank each reasoning-effort variant separately (e.g. "Claude Opus 4.6 (High)") instead of one fused row per model. Default false.

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed1 schema field changed
    • addedInput schema / properties / include_variants
      Added value: +{
      +  "description": "Rank each reasoning-effort variant separately (e.g. \"Claude Opus 4.6 (High)\") instead of one fused row per model. Default false.",
      +  "type": "boolean"
      +}
  2. First observed

TDQS

A3.9/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, destructiveHint=false and a closed world, so the safety profile is covered. The description adds genuinely useful behavior beyond that: default output is one row per model fused across reasoning-effort settings, plus paging support. It does not describe ordering stability, tie-handling, or scale units, so it remains adequate rather than rich.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, front-loaded with what the ranking is before the sibling pointer and the paging note. Tight and readable, though the closing paging sentence largely duplicates what the schema already states.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description carries the burden of conveying the return shape, and it does so reasonably: one row per model, fused across reasoning-effort settings, pageable. It omits the exact fields per row and the ranking direction, but for a read-only listing tool it is close to complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, with each of limit, offset and include_variants fully documented in the schema, so the baseline is 3. The description restates paging via limit/offset and the fused-by-default behavior tied to include_variants, adding no format or constraint detail the schema lacks.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Top of the cross-benchmark aggregate ranking', 'one rating scale by an IRT model fit over public benchmark leaderboards') and distinguishes itself from siblings like get_benchmark and get_model by describing a fused cross-benchmark aggregate. An agent can identify exactly what this tool returns without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives clear context for use and routes one adjacent need explicitly ('call about_the_aggregate for the current coverage counts'). However, it never states when to prefer this over siblings like compare_models or get_benchmark, so the alternative-selection guidance is partial rather than complete.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources