Skip to main content
Glama

SOTA2 — AI Solution Rankings

Search research benchmarks, datasets, tasks and papers

search_research
Read-onlyIdempotent

Searches research benchmarks, datasets, tasks or papers using keywords, benchmark/dataset names, paper titles, authors or arXiv identifiers. Returns 20 results per API page. Versions, test splits and Full/Hard variants can be separate benchmarks. Search summaries are discovery data, not validated performance ranks. Paper search requires query and has no count/date sort. Dataset/task/benchmark count sorts order discovery results, not cross-benchmark performance.

Input Schema

NameRequiredDescriptionDefault
pageNo
sortNo
taskNo
queryNo
domainNo
datasetNo
resourceNobenchmarks

Output Schema

NameRequiredDescriptionDefault
dataYes

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observed

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover the safety profile (readOnly, idempotent, openWorld, non-destructive), yet the description adds real behavioral context: 20 results per API page, that versions/test splits/Full-Hard variants may appear as separate benchmarks, and that search summaries are discovery data rather than validated ranks. That last point meaningfully warns the agent against over-reading results, which annotations cannot convey.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Dense but economical — the resource scope and match keys are front-loaded, then constraints follow. The two closing sentences about discovery data vs performance ranks overlap somewhat but address distinct pitfalls (summary semantics vs sort ordering), so little is wasted.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return values need not be explained, and the annotations carry the safety profile. Still, for a 7-parameter search tool with no schema descriptions, the description should explain what domain/task/dataset filters actually do and how resource interacts with query; that gap keeps it at minimum-viable completeness.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, so the description must compensate, and it partially does: it discloses page size (20), the query requirement for paper search, and the semantics of count-based sorts. However, domain, task, dataset, resource (default 'benchmarks'), and the page bound of 1–1000 are left entirely undocumented, so several parameters remain opaque.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (searches) over a clearly enumerated set of resources (benchmarks, datasets, tasks, papers) and names the match keys (keywords, names, titles, authors, arXiv IDs). This cleanly distinguishes it from sibling detail-getters like get_research_benchmark and get_research_entity, which retrieve a single entity rather than discover across resource types.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Offers one concrete conditional rule — 'Paper search requires query and has no count/date sort' — which tells the agent a prerequisite for a resource type. But it never states when to use this discovery tool versus the sibling retrievers (get_research_entity, get_research_benchmark) or when to switch tools after a search. Usage is implied rather than prescribed.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources