Skip to main content
Glama

Media leaderboards

aa_media

Query human-preference arenas for image, video, speech, and music models, returning Elo ratings with 95% confidence intervals. Filter open/closed weights, choose boards, and select fields.

Instructions

Human-preference arenas: image, video, speech, music (Elo + 95% CI).

Args: board: any image/video/TTS/music board key (text_to_image, image_editing, text_to_video, image_to_video, video_editing, text_to_speech, text_to_speech_controlled, music_instrumental, music_with_vocals) or a speech board (speech_to_text, speech_to_text_streaming, speech_to_speech — also accepted as realtime_voice / "realtime voice"). Hyphens/spaces/case are ignored (text-to-image == text_to_image). top: rows to return (default 15). fields: comma string (or list) of columns; supports ci_lower, ci_upper, ci (95% CI), open_weights, url. Elo CI is not shown by default. open_weights: keep only open- or closed-weights entries. refresh: bypass the cache and re-fetch.

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
topNo
boardYes
fieldsNo
refreshNo
open_weightsNo

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observedv0.1.0

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the burden and does disclose real behavioral traits: refresh bypasses the cache, Elo CI is hidden by default, fuzzy key normalization (text-to-image == text_to_image), and alias acceptance (realtime_voice). It stops short of stating read-only nature or rate limits, but the caching and default-output behavior are genuinely useful context.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A one-line summary is front-loaded before the Args block, and the structured arg list is dense with no filler. The board enumeration is long but each entry is load-bearing; the only mild cost is length.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema, yet the description conveys the key return content (Elo plus 95% CI, selectable columns) so the agent knows what comes back. It could state the return format/row shape more explicitly, but for a leaderboard lookup it is adequately complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, so the description must compensate and it does thoroughly: it enumerates every valid board key plus aliases, explains normalization, gives the top default, lists supported field columns including ci_lower/ci_upper/ci/open_weights/url, and defines open_weights and refresh semantics. This is far more than the bare schema offers.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific resource — human-preference arenas for image, video, speech, and music with Elo + 95% CI — which is concrete and distinguishes it from generic leaderboard tools. However, it never explicitly contrasts itself with close siblings like aa_leaderboard or aa_benchmark_leaderboard, so the agent must infer the boundary.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied through the enumeration of valid board keys, which tells an agent this tool is for media arenas rather than text/benchmark leaderboards. There is no explicit when-to-use/when-not or named alternative, which is a gap given three leaderboard-family siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.