Skip to main content
Glama

BenchLeader

List benchmarks

list_benchmarks
Read-onlyIdempotent

The benchmarks BenchLeader reads: what each measures, publisher, category, whether it feeds the Index (and its weight), how many models it covers, status and caveats. Filter by category or a search query. Paged.

Input Schema

NameRequiredDescriptionDefault
limitNoHow many benchmarks to return (at most 100).
queryNoSearch benchmark names, ids, aliases and publishers ("swe-bench", "arc", "epoch").
offsetNoRows to skip, for paging (use next_offset from the previous page).
categoryNoOnly this category, e.g. coding, agentic, reasoning, math, knowledge.
in_index_onlyNoOnly benchmarks that feed the BenchLeader Index.

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observed

TDQS

A3.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, non-destructive, closed-world, so the safety profile is covered. The description adds genuinely useful context beyond them: the exact per-row content (weight, model coverage, status, caveats) and that results are paged.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with what the tool returns, followed by two short operational sentences. Minor redundancy between 'Filter by category or a search query' and the schema descriptions, but no wasted bulk.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description compensates well by enumerating the fields each benchmark record contains. Combined with full parameter documentation, an agent has enough to call it correctly; only upstream routing versus singular get_benchmark is unstated.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so all five parameters are already documented, including query examples and offset paging. The description only restates category/query filtering and paging, adding no syntax or semantics beyond the schema — baseline 3.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('The benchmarks BenchLeader reads') and enumerates the fields each row carries, so an agent knows exactly what comes back. It is clearly the plural/list counterpart to get_benchmark, but it never names that sibling to make the distinction explicit.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

'Filter by category or a search query. Paged.' implies how to narrow results, but gives no explicit when-to-use guidance, no exclusion against get_benchmark for a single benchmark, and no note on when to page versus filter.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources