Skip to main content
Glama

The Aggregate — LLM benchmark aggregate

Prediction duel standings

get_prediction_duel
Read-onlyIdempotent

Guesswork — the public prediction duel: every day frontier LLMs and The Aggregate's own IRT model predict newly scraped benchmark scores before seeing them, and the errors are scored. Returns the current monthly standings, wins and losses included.

Input Schema

NameRequiredDescriptionDefault

No arguments

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observed

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, non-destructive and openWorldHint=false, so the safety profile is covered. The description adds useful scope context (daily cadence, predictions made before scores are seen, errors scored) and states the return payload includes wins and losses, but says nothing about freshness or how monthly standings reset.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, with the resource identified up front via 'the public prediction duel'. The middle clause about LLMs and the IRT model is somewhat wordy but does real explanatory work for a novel concept, so it mostly earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a parameterless read-only tool with no output schema, the description supplies enough: it names the entity, describes the underlying mechanism, and previews the returned content (monthly standings, wins, losses). Only the absence of any refresh/period detail keeps it from a 5.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool takes zero parameters, so the baseline of 4 applies. There is nothing for the description to clarify beyond confirming the call needs no arguments.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb+resource: returns current monthly standings for the 'Guesswork' prediction duel, including wins and losses. It is clearly distinguishable in substance from the data-lookup siblings, though it never names or contrasts with get_leaderboard, which is the closest relative.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explains what the duel is but gives no when-to-use guidance, no prerequisites, and no mention of alternatives such as get_leaderboard or compare_models. The agent must infer the call context entirely.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources