Skip to main content
Glama

Catalog.Leaderboard

catalog.leaderboard
Read-onlyIdempotent

Public benchmark leaderboard: pipeline rankings per dataset.

Rankings are per-dataset under the canonical within_session protocol (see protocol). Within each dataset, rows are sorted desc by meanAccuracyPct (95% CI in ciLoPct/ciHiPct). Use pipelineId as the template id hint for catalog.template when building a pipeline. updated marks each dataset's most recent run; packFingerprint identifies the exact dataset pack the scores came from.

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault

No arguments

Output Schema

TableJSON Schema
NameRequiredDescriptionDefault

No arguments

Schema Changelog

Changes observed during successful MCP inspections.

  1. Added

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already establish the safe read-only/idempotent profile, so the bar is lower; the description adds real context beyond that: the ``within_session`` protocol, that ``rows`` are sorted descending by ``meanAccuracyPct`` with CI bounds, and what ``updated``/``packFingerprint`` signify.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

It is front-loaded with the core purpose, then layers protocol/sorting/field meaning efficiently. The density of backtick-referenced fields is slightly heavy, but each sentence earns its place with actionable information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, return-value explanation is not strictly required, yet the description goes further by naming key fields and their meaning. For a zero-parameter read tool this is quite complete, though it omits framing like freshness or scope limits of the leaderboard.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool takes zero parameters, so there is nothing parameter-level to document; the baseline for a no-param tool is 4. The description's field references (``pipelineId``, ``meanAccuracyPct``, etc.) are return-value semantics rather than input parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The opening clause states a specific verb+resource: a public benchmark leaderboard returning pipeline rankings per dataset. This is clearly distinguishable from siblings like catalog.datasets, catalog.templates, and execution.results, which don't describe benchmark rankings.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives an explicit downstream usage link ('Use ``pipelineId`` as the template id hint for ``catalog.template`` when building a pipeline'), which tells the agent a concrete follow-up action. It does not state when NOT to use this tool or name alternatives, so it falls short of the top band.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.