The Aggregate — LLM benchmark aggregate

Server Details

LLM rankings from public benchmarks, model comparisons and source-linked results. Updated daily.

If you are the author of this connector, you can claim ownership by verifying the domain or GitHub account it belongs to. Claimed connector authors can inspect health checks, view analytics, and manage their listing.

Status: Healthy
Uptime: 100.0% over 55 days
Last Tested: 2026-10-11 14:32
Transport: Streamable HTTP · MCP 2025-06-18
URL

TDQS

A3.9/5.0

Scored 2026-09-12 01:10 across 8 tools

Disambiguation4/5

Tools mostly target distinct entities and actions: model search/get/compare, benchmark search/get, aggregate leaderboard, prediction duel, and metadata. There is minor overlap between get_leaderboard and search_models (both surface model rankings), but descriptions distinguish top aggregate ranking from name/provider lookup.

Naming Consistency4/5

Names follow a mostly consistent verb_noun pattern with get_*, search_*, and compare_* prefixes, all in snake_case. about_the_aggregate is a minor deviation from the verb_noun convention but is a reasonable one-off for metadata.

Tool Count5/5

Eight tools are well-scoped for a read-only benchmark aggregate, covering essential query needs without bloat. Each tool earns its place and the count is comfortably within the ideal 3-15 range.

Completeness4/5

The surface covers core read-only workflows: discovery (search), detail (get), comparison, aggregate leaderboard, metadata, and prediction standings. Minor gaps include no explicit list-all benchmarks/providers or bulk export, though search tools likely suffice for navigation.

Available Tools

8 tools

about_the_aggregateAbout The Aggregate

Read-onlyIdempotent

Inspect

What this data is: how the IRT fusion works, current coverage counts, update cadence, and how to cite it.

ParametersJSON Schema

Name	Required	Description	Default
No parameters

compare_modelsCompare models

Read-onlyIdempotent

Inspect

Head-to-head between 2-4 models: aggregate ranks, Elo gap with a significance note based on the standard errors, and notable benchmarks they share.

ParametersJSON Schema

Name	Required	Description	Default
`models`	Yes	Two to four model names or slugs.

get_benchmarkBenchmark detail

Read-onlyIdempotent

Inspect

One benchmark in depth: what it measures, the original source leaderboard URL, IRT stats (difficulty, noise, model coverage), skill weights, and the current top models on it.

ParametersJSON Schema

Name	Required	Description	Default
`top`	No	How many top models to list (1-50, default 10).
`benchmark`	Yes	Benchmark name or slug, e.g. "Aider polyglot".

get_leaderboardAggregate leaderboard

Read-onlyIdempotent

Inspect

Top of the cross-benchmark aggregate ranking: every model placed on one Elo scale by an IRT model fit over public benchmark leaderboards (call about_the_aggregate for the current coverage counts). One row per model by default, fused across reasoning-effort settings. Supports paging via limit/offset.

ParametersJSON Schema

Name	Required	Description
`limit`	No	Rows to return (1-100, default 25).
`offset`	No	Rows to skip from the top (default 0).
`include_variants`	No	Rank each reasoning-effort variant separately (e.g. "Claude Opus 4.6 (High)") instead of one fused row per model. Default false.

get_modelModel profile

Read-onlyIdempotent

Inspect

One model in depth: aggregate rank, Elo with standard error, provider, what it is, cost per task where known, and its most notable benchmark results (with percentiles).

ParametersJSON Schema

Name	Required	Description	Default
`model`	Yes	Model name or slug, e.g. "Claude Opus 4.5" or "gpt-5-5".

get_prediction_duelPrediction duel standings

Read-onlyIdempotent

Inspect

Guesswork — the public prediction duel: every day frontier LLMs and The Aggregate's own IRT model predict newly scraped benchmark scores before seeing them, and the errors are scored. Returns the current monthly standings, wins and losses included.

ParametersJSON Schema

Name	Required	Description	Default
No parameters

search_benchmarksSearch benchmarks

Read-onlyIdempotent

Inspect

Find benchmarks in the aggregate by (partial) name. Returns model coverage, difficulty on the Elo scale, and the benchmark page URL.

ParametersJSON Schema

Name	Required	Description	Default
`limit`	No	Max results (1-25, default 10).
`query`	Yes	Benchmark name fragment, e.g. "swe-bench" or "arena".

search_modelsSearch models

Read-onlyIdempotent

Inspect

Find ranked models by (partial) name or provider. Returns rank, Elo and the model page URL. One row per model by default, fused across reasoning-effort settings.

ParametersJSON Schema

Name	Required	Description
`limit`	No	Max results (1-25, default 10).
`query`	Yes	Model or provider name fragment, e.g. "opus" or "deepseek".
`include_variants`	No	Return each reasoning-effort variant separately (e.g. "Claude Opus 4.6 (High)") instead of one fused row per model. Default false.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

2 tool updatesAug 17, 2026

Changedget_leaderboard1 field changed

addedInput schema / properties / include_variants

Added value: +{
+  "description": "Rank each reasoning-effort variant separately (e.g. \"Claude Opus 4.6 (High)\") instead of one fused row per model. Default false.",
+  "type": "boolean"
+}

Changedsearch_models1 field changed

addedInput schema / properties / include_variants

Added value: +{
+  "description": "Return each reasoning-effort variant separately (e.g. \"Claude Opus 4.6 (High)\") instead of one fused row per model. Default false.",
+  "type": "boolean"
+}

8 tool updatesJul 25, 2026
- First observedabout_the_aggregate
- First observedcompare_models
- First observedget_benchmark
- First observedget_leaderboard
- First observedget_model
- First observedget_prediction_duel
- First observedsearch_benchmarks
- First observedsearch_models

Related MCP Connectors

AI Pricing Hub A
io.github.gishh
Source-backed AI model pricing, rankings, history, and benchmark data.
AI & Machine Learning Research & Data
112
XFMS — Model Source A
dev.xpansion
Pick the right LLM for any task. Ranked shortlist with rationale across 8 evaluators.
AI & Machine Learning Research & Data
52
Cheapest LLM API A
io.github.tehtelabs
Compare LLM API prices, search models and providers, and access reviewed benchmark results.
AI & Machine Learning Search
71
Harpd Agent Data
com.harpd
Live rankings and pricing for 10,000+ AI products and LLMs. Free tier; paid data via x402.
AI & Machine Learning Open Data Research & Data
1

Related MCP Servers

LLM-Adviser-Mcp
RAG Systems Search Developer Tools
Daichi-Kudo
A
license
A
quality
D
maintenance
Give your AI assistant real-time LLM/VLM knowledge. Pricing, benchmarks, and recommendations — updated every hour, not every training cycle.
Updated 4 months ago (2026-06-18 04:55 UTC)
4
85 npm
2
MIT
Model Price Watch
AI & Machine Learning Developer Tools
romanshumy
A
license
A
quality
A
maintenance
Live LLM API pricing: current token prices, model comparisons, cheapest-model lookups, and The LLM Price Index for 150+ models across 20+ providers, re-verified daily. No API key required.
Updated a day ago (2026-10-10 17:05 UTC)
4
1
MIT
XFMS
AI & Machine Learning Research & Data
VisionAIrySE
A
license
A
quality
F
maintenance
Enables users to select the optimal LLM for their specific task by aggregating benchmark data and user intent, returning a ranked shortlist with plain-English rationale.
Updated 5 months ago (2026-05-23 23:02 UTC)
5
MIT
FreeModel MCP
AI & Machine Learning Developer Tools
yummy342
A
license
A
quality
C
maintenance
Routes tasks to the optimal AI model based on task type and benchmark scores across 25+ platforms. Automatically selects the best model for coding, reasoning, writing, and more using public benchmark data.
Updated 11 days ago (2026-09-30 12:56 UTC)
5
414 npm
1
MIT