Skip to main content
Glama

sota_changelog

Benchmark lanes whose leader changed since a date, corpus-wide or within a topic's subtree: for each change the previous holder, the new leader (paper, method, venue), the delta and how long the old record stood. Use for 'what changed in SOTA on X since June' or a periodic catch-up. Examines the 150 most recently active lanes in scope; a change whose jump is outside the SOTA feed's believability band is not reported. [Free · 3 credits]

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "since": "<since>"
    },
    "name": "sota_changelog"
  }
}

Input Schema

NameRequiredDescriptionDefault
limitNo
sinceYesISO date the window starts at (YYYY-MM-DD)
topicNoTopic name or UUID to scope to
min_entrantsNoIgnore lanes with fewer entities than this

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observed

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden, and it does disclose two non-obvious behaviors: only the 150 most recently active lanes in scope are examined, and changes outside the SOTA feed's believability band are silently dropped. It also states the credit cost. However, it never explains what the believability band operationally is, nor ordering or whether results can be paged beyond 'limit', leaving real coverage limits partly opaque.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the core predicate in the first clause, then return shape, then usage, then coverage caveats and cost. Every sentence carries information, but the first sentence is a long multi-clause run that packs four output fields plus two scoping modes and could be broken for faster scanning.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No output schema and no annotations exist, so the description must describe returns itself — and it does, listing each change's fields. The only residual gap is operational detail around result ordering and how limit interacts with the 150-lane scan, which an agent could still need in order to interpret counts.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 75% and the schema already describes since (ISO date), topic (name or UUID) and min_entrants ('ignore lanes with fewer entities than this'). The description reinforces the date-window and subtree-scoping semantics but adds nothing about limit, min_entrants thresholds, or default behavior, so 3 is the correct baseline when the schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific resource and filter: 'Benchmark lanes whose leader changed since a date, corpus-wide or within a topic's subtree,' and enumerates what each result contains (previous holder, new leader with paper/method/venue, delta, tenure of old record). That is far more than a restatement of the name. It stops short of naming a sibling (e.g. benchmark_timeline or sota_check) as the alternative, so it is clear but not explicitly differentiated.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It supplies concrete invocation contexts — 'what changed in SOTA on X since June' or a periodic catch-up — which tells the agent the right situation to reach for this tool. It gives no exclusions or named alternatives for the neighboring case (e.g. a full historical timeline), so the guidance is clear but one-sided.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources