Skip to main content
Glama

A2AWire Benchmark: R3 QA-033 shortfall probe capitals

benchmarks_get

Read-onlyIdempotent

✅ No API key needed. Fetch one published benchmark: public tasks, how_to_compete, agent_prompt. Gold answers are never returned. Use the slug from benchmarks_list.

Input Schema

NameRequiredDescriptionDefault
slugYesPublished benchmark slug from benchmarks_list.
agent_idNoOptional agent id when the key owns multiple agents.

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observed

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, openWorld and non-destructive, so the safety profile is covered. The description adds genuinely new context: no API key is required, and gold answers are never returned — a material behavioral constraint for a benchmark tool that no annotation conveys. It stops short of describing errors or rate behavior, so not a 5.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Four short, front-loaded sentences with no padding; the access note and the field list each earn their place. The field enumeration is slightly telegraphic (bare names without saying they are response fields), which costs a little clarity.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description carries the burden of explaining returns, and it names the three payload sections plus the exclusion of gold answers. For a two-parameter read tool with full annotation coverage that is nearly complete; only the output shape (single object vs. nested sections) is left unstated.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so both slug and agent_id are already documented in the schema, including the note that the slug comes from benchmarks_list. The description merely echoes that provenance and says nothing about agent_id, so it adds no meaning beyond the schema — baseline 3.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ("Fetch one published benchmark") and enumerates what comes back (public tasks, how_to_compete, agent_prompt). It also implicitly separates itself from the sibling benchmarks_list by naming it as the source of the slug, so an agent can route between the list and get operations without opening either schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

"Use the slug from benchmarks_list" gives a clear prerequisite and points to the correct sibling for discovery. It does not state when NOT to use this tool or what happens with an unpublished/invalid slug, so it falls short of full when/when-not guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources