Skip to main content
Glama

The Aggregate — LLM benchmark aggregate

Server Details

LLM rankings from public benchmarks, model comparisons and source-linked results. Updated daily.

If you are the author of this connector, you can claim ownership. Claimed connector authors can inspect health checks, view analytics, and manage their listing.
Status
Healthy
Uptime
100.0% over the last 24 hours
Last Tested
Transport
Streamable HTTP · MCP 2025-06-18
URL
ChatGPT
–
Claude
–

TDQS

A3.8/5.0

Scored across 8 tools

Disambiguation4/5

Each tool targets a distinct resource/action: model lookup, benchmark lookup, leaderboard listing, comparison, prediction duel, and methodology. The only soft overlap is get_leaderboard vs search_models, which both return rank+rating rows, but the name-based query requirement on search_models and paging on get_leaderboard make the boundary readable. about_the_aggregate is clearly a one-off meta tool.

Naming Consistency4/5

Seven of eight tools follow a clean verb_noun snake_case pattern (get_model, search_benchmarks, compare_models, get_leaderboard). The single outlier, about_the_aggregate, breaks the verb-first convention but is unambiguous in intent.

Tool Count5/5

Eight tools is well within the ideal range and every tool maps to a distinct query a user of a benchmark aggregate would make. No redundant or filler tools are present.

Completeness4/5

Coverage spans discovery (search), depth (get_model, get_benchmark), ranking (get_leaderboard), comparison, methodology, and the prediction duel — a solid surface for the domain. Minor gaps: no way to browse/enumerate all benchmarks without a name query, and no historical or time-series view of ratings despite the stated update cadence.

Available Tools

8 tools
about_the_aggregateAbout The AggregateA
Read-onlyIdempotent
Inspect

What this data is: the published rating method, current coverage counts, update cadence, and how to cite it.

Inputs: no named parameters.

compare_modelsCompare modelsA
Read-onlyIdempotent
Inspect

Head-to-head between 2-4 models: aggregate ranks, rating gap with a significance note based on the standard errors, and notable benchmarks they share.

Inputs: models (required).

get_benchmarkBenchmark detailA
Read-onlyIdempotent
Inspect

One benchmark in depth: what it measures, the original source leaderboard URL, IRT stats (difficulty, noise, model coverage), skill weights, and the current top models on it.

Inputs: benchmark (required), top.

get_leaderboardAggregate leaderboardA
Read-onlyIdempotent
Inspect

Top of the cross-benchmark aggregate ranking: published Aggregate Ratings combine public benchmark evidence on one scale (call about_the_aggregate for the current coverage and method). One row per model by default, fused across reasoning-effort settings. Supports paging via limit/offset.

Inputs: limit, offset, include_variants.

get_modelModel profileA
Read-onlyIdempotent
Inspect

One model in depth: aggregate rank, Aggregate Rating with standard error, provider, what it is, cost per task where known, and its most notable benchmark results (with percentiles).

Inputs: model (required).

get_prediction_duelPrediction duel standingsB
Read-onlyIdempotent
Inspect

Guesswork — the public prediction duel: every day frontier LLMs and The Aggregate's own IRT model predict newly scraped benchmark scores before seeing them, and the errors are scored. Returns the current monthly standings, wins and losses included.

Inputs: no named parameters.

search_benchmarksSearch benchmarksA
Read-onlyIdempotent
Inspect

Find benchmarks in the aggregate by (partial) name. Returns model coverage, difficulty on the rating scale, and the benchmark page URL.

Inputs: query (required), limit.

search_modelsSearch modelsA
Read-onlyIdempotent
Inspect

Find ranked models by (partial) name or provider. Returns rank, Aggregate Rating and the model page URL. One row per model by default, fused across reasoning-effort settings.

Inputs: query (required), limit, include_variants.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    Related MCP Connectors

    Related MCP Servers

    • A
      license
      Not graded
      quality
      C
      maintenance
      Enables users to ask which LLM fits a task and get deterministic ranked recommendations, leaderboards, side-by-side comparisons, API prices, and model details from daily benchmark scores and prices, with no API key required.
      MIT
    • A
      license
      A
      quality
      B
      maintenance
      Live LLM API pricing: current token prices, model comparisons, cheapest-model lookups, and The LLM Price Index for 150+ models across 20+ providers, re-verified daily. No API key required.
      4
      1
      MIT
    Try in Browser

    Glama MCP Gateway

    Add one secure layer between your agents and this server.

    Resources