Skip to main content
Glama

HypePaper

sota_papers

State of the art for a research topic, a dataset, or a free-text dataset query. Use first for any 'what is SOTA on X', 'best method for X' or 'who leads benchmark X' question. Per benchmark lane (dataset x metric x condition) returns the leader, runners-up and entrants since since (default 365 days) with whether each beat the incumbent. Every row carries the value, its provenance (self-reported or reported as a baseline in another paper's table, match method), the paper (id, arXiv id, date, venue, citations, hype, code URL), up to 5 authors and, on paid plans, affiliations. Single-source lanes are flagged: one paper's table is not a leaderboard. Rankings require matching experimental protocols; rows carry conditions and source evidence. Pass the user's own words as topic when no topic name is known: scope.interpreted_as then names the topic and benchmarks it was read as (say so), and lists the newest matching papers with ranked_in_lanes — a paper without a lane row has no verified standing. [Free · 6 credits]

Input Schema

NameRequiredDescriptionDefault
limitNoLanes to return
queryNoFree text matched against dataset names when no topic/dataset is known; read like a free-text topic when no dataset matches
sinceNoISO date; entrants on/after it are listed. Default 365 days ago
topicNoTopic name or UUID; lanes come from its whole subtree. Free text that names no topic (e.g. '360 metric depth') is read as the topic it is about, on the benchmarks its matching papers report; scope.interpreted_as says how, with the newest matching papers
datasetNoDataset name or key, optionally with a protocol, e.g. 'NAVSIM navtest'
per_laneNoRows per lane (leader + runners-up); recent_entrants gets the same cap
conditionNoExact protocol key returned by dataset_leaderboard or sota_papers

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observed

TDQS

A4.5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are supplied, so the description carries the full burden and does so: it discloses the credit cost, the default 365-day entrant window, the single-source-lane caveat that 'one paper's table is not a leaderboard', the requirement that rankings need matched experimental protocols, the paid-plan gating of affiliations, and the scope.interpreted_as fallback behavior. That is unusually complete disclosure of limits and edge cases for a costly retrieval call.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The opening is well front-loaded and the guidance sentence lands early, but the body becomes a dense multi-clause run-on listing every returned field, which is hard to scan and duplicates some schema detail. Most sentences do earn their place, but the density and the repeated explanation of free-text topic handling cost it clarity.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 7-parameter tool with no output schema and no annotations, the description supplies the return shape (lane rows, provenance, paper metadata, author cap, plan-gated affiliations) and the caveats needed to interpret it. An agent has everything required to call it correctly and to explain the results to a user.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is already 100%, so the baseline is 3. The description still adds genuine meaning beyond the schema: it explains that the user's own words should be passed as `topic`, that free text resolving to no topic is reinterpreted onto benchmarks its matching papers report, and that scope.interpreted_as names the reading so the agent can relay it. The lane/per_lane relationship and condition semantics are mostly left to the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb+resource combination up front: it returns state-of-the-art standings per benchmark lane (dataset x metric x condition) for a topic, dataset, or free-text query. It goes further than the name by enumerating what each lane row contains (leader, runners-up, entrants, provenance, paper metadata, authors), so an agent knows the shape of the answer before opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives an explicit trigger phrase set — 'what is SOTA on X', 'best method for X', 'who leads benchmark X' — and instructs 'use first', which is a real routing rule against siblings like dataset_leaderboard or new_sota. It stops short of stating when NOT to use it or naming the alternative tool an agent should pick for drill-down queries, which keeps it out of 5 territory.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources