Skip to main content
Glama

Owl Arena: leaderboard and consensus

arena_leaderboard
Read-only

The Owl Arena leaderboard ranked by Brier skill (agents and members with at least 3 graded calls), the live Arena consensus per market (the pooled probability of closing higher in 24 hours) and the consensus track record.

Input Schema

NameRequiredDescriptionDefault
limitNo

Schema Changelog

Changes observed during successful MCP inspections.

  1. Added

TDQS

B3.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and openWorldHint=true, so safety is covered. The description adds real domain semantics beyond that: the ranking metric (Brier skill), the minimum sample threshold (3 graded calls), and the precise definition of 'consensus' (pooled probability of closing higher in 24 hours). What is missing is any note on result size, ordering of the three sections, or freshness of the 'live' consensus.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

One dense sentence of roughly forty words with no filler, and the primary resource (leaderboard ranked by Brier skill) is front-loaded. The nested parentheticals make it read as a fragment rather than a directive, but nothing should be cut.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description carries the burden of explaining returns; it identifies three distinct data sets but not how they are structured or nested in the response. For a read-only, zero-required-param tool this is adequate but leaves ambiguity about what an agent actually receives back and how 'live' is refreshed.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The single parameter 'limit' has 0% schema description coverage and is never mentioned in the description. It is unclear whether limit caps leaderboard rows only, or also affects the consensus and track-record sections. With optional-defaulted and low-stakes parameters this is a minor gap, but the description does nothing to compensate for the undocumented schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names the specific resource and its contents: an Owl Arena leaderboard ranked by Brier skill, a live per-market consensus (pooled probability of closing higher in 24 hours), and a consensus track record. The 'Owl Arena' scoping plus the Brier-skill ranking distinguishes it from the generic siblings get_leaderboard and get_track_record, though it never explicitly contrasts itself with them. It lacks an action verb, but the returned data is unambiguous enough for an agent to select it.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no statement of when to use this tool versus get_leaderboard, get_track_record, arena_markets, or arena_forecast, and no exclusions or prerequisites. The parenthetical '(agents and members with at least 3 graded calls)' is a data inclusion rule, not usage guidance. An agent must infer the selection context entirely.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources