The Aggregate — LLM benchmark aggregate
Server Details
LLM rankings from public benchmarks, model comparisons and source-linked results. Updated daily.
- Status
- Healthy
- Uptime
- 100.0% over the last 24 hours
- Last Tested
- Transport
- Streamable HTTP · MCP 2025-06-18
- URL
- ChatGPT
- –
- Claude
- –
TDQS
Scored across 8 tools
Each tool targets a distinct resource/action: model lookup, benchmark lookup, leaderboard listing, comparison, prediction duel, and methodology. The only soft overlap is get_leaderboard vs search_models, which both return rank+rating rows, but the name-based query requirement on search_models and paging on get_leaderboard make the boundary readable. about_the_aggregate is clearly a one-off meta tool.
Seven of eight tools follow a clean verb_noun snake_case pattern (get_model, search_benchmarks, compare_models, get_leaderboard). The single outlier, about_the_aggregate, breaks the verb-first convention but is unambiguous in intent.
Eight tools is well within the ideal range and every tool maps to a distinct query a user of a benchmark aggregate would make. No redundant or filler tools are present.
Coverage spans discovery (search), depth (get_model, get_benchmark), ranking (get_leaderboard), comparison, methodology, and the prediction duel — a solid surface for the domain. Minor gaps: no way to browse/enumerate all benchmarks without a name query, and no historical or time-series view of ratings despite the stated update cadence.
Available Tools
8 toolsabout_the_aggregateAbout The AggregateARead-onlyIdempotentInspect
What this data is: the published rating method, current coverage counts, update cadence, and how to cite it.
Inputs: no named parameters.
compare_modelsCompare modelsARead-onlyIdempotentInspect
Head-to-head between 2-4 models: aggregate ranks, rating gap with a significance note based on the standard errors, and notable benchmarks they share.
Inputs: models (required).
get_benchmarkBenchmark detailARead-onlyIdempotentInspect
One benchmark in depth: what it measures, the original source leaderboard URL, IRT stats (difficulty, noise, model coverage), skill weights, and the current top models on it.
Inputs: benchmark (required), top.
get_leaderboardAggregate leaderboardARead-onlyIdempotentInspect
Top of the cross-benchmark aggregate ranking: published Aggregate Ratings combine public benchmark evidence on one scale (call about_the_aggregate for the current coverage and method). One row per model by default, fused across reasoning-effort settings. Supports paging via limit/offset.
Inputs: limit, offset, include_variants.
get_modelModel profileARead-onlyIdempotentInspect
One model in depth: aggregate rank, Aggregate Rating with standard error, provider, what it is, cost per task where known, and its most notable benchmark results (with percentiles).
Inputs: model (required).
get_prediction_duelPrediction duel standingsBRead-onlyIdempotentInspect
Guesswork — the public prediction duel: every day frontier LLMs and The Aggregate's own IRT model predict newly scraped benchmark scores before seeing them, and the errors are scored. Returns the current monthly standings, wins and losses included.
Inputs: no named parameters.
search_benchmarksSearch benchmarksARead-onlyIdempotentInspect
Find benchmarks in the aggregate by (partial) name. Returns model coverage, difficulty on the rating scale, and the benchmark page URL.
Inputs: query (required), limit.
search_modelsSearch modelsARead-onlyIdempotentInspect
Find ranked models by (partial) name or provider. Returns rank, Aggregate Rating and the model page URL. One row per model by default, fused across reasoning-effort settings.
Inputs: query (required), limit, include_variants.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
Related MCP Connectors
Daily LLM leaderboard from independent benchmarks: ranks, scores, prices, speed and comparisons.
Current AI model rankings, API prices, benchmark results and history from BenchLM.
Source-backed AI model pricing, rankings, history, and benchmark data.
Compare LLM API prices, search models and providers, and access reviewed benchmark results.
Related MCP Servers
- AlicenseAqualityDmaintenanceGive your AI assistant real-time LLM/VLM knowledge. Pricing, benchmarks, and recommendations — updated every hour, not every training cycle.490 npm2MIT
- AlicenseNot gradedqualityCmaintenanceEnables users to ask which LLM fits a task and get deterministic ranked recommendations, leaderboards, side-by-side comparisons, API prices, and model details from daily benchmark scores and prices, with no API key required.MIT
- AlicenseAqualityBmaintenanceLLM benchmark rankings, score history, drift status, incidents and provider status from AI Stupid Level, plus route previews and answers through its OpenAI-compatible Smart Router.11MIT
- AlicenseAqualityBmaintenanceLive LLM API pricing: current token prices, model comparisons, cheapest-model lookups, and The LLM Price Index for 150+ models across 20+ providers, re-verified daily. No API key required.41MIT
Glama MCP Gateway
Add one secure layer between your agents and this server.