Skip to main content
Glama

AP Exception Review — Meridian Parts (run-037bea)

benchmark_corpus

Read-onlyIdempotent

Get this benchmark's corpus for FREE (API key required): the protocol doc + case-packet documents of the exact data version the benchmark is built on -- no purchase, no faucet, no wallet (SPEC benchfree). Study it, then benchmark_start_run; wall clock starts at run start. On a /mcp/benchmarks/{slug} session the slug defaults to the routed benchmark. Gold answers and private task payloads are never returned. REST twin: GET /api/v1/benchmarks/{slug}/corpus.

Tool call template

This tool declares no required parameters. Add optional parameters from the input schema as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {},
    "name": "benchmark_corpus"
  }
}

Input Schema

NameRequiredDescriptionDefault
slugNoPublished benchmark slug from benchmarks_list. Optional on a /mcp/benchmarks/{slug}/http session (defaults to that benchmark).
agent_idNoOptional agent id when the key owns multiple agents.

Schema Changelog

Changes observed during successful MCP inspections.

  1. Added

TDQS

A4.4/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare this a read-only, idempotent, non-destructive, closed-world operation, and the description adds meaningful context beyond that: no purchase, faucet, or wallet is needed, gold answers and private task payloads are never returned, and the timing clock does not start until a run begins. It also names the REST equivalent endpoint for parity.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loads the core action and the key constraint (free, no wallet), then layers the sequencing and return-value caveats. The parenthetical spec reference and REST-twin note are slightly extraneous but short, and no sentence is wasted.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description carries the burden of describing the return value and does so: protocol doc plus case-packet documents, explicitly excluding gold answers and private payloads. It also covers prerequisites, timing, and the session-default behavior, so an agent has everything needed to call it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so both slug and agent_id are already fully documented, including the routed-session default. The description restates that the slug defaults to the routed benchmark on a /mcp/benchmarks/{slug} session but adds no new syntax or format detail, making the baseline 3 appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource: fetch this benchmark's corpus, defined as the protocol doc plus the case-packet documents for the exact data version the benchmark is built on. It is clearly distinguishable from siblings like benchmark_start_run and benchmarks_get, which it references by name as the next step.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives explicit sequencing guidance: study the corpus, then call benchmark_start_run, with the note that the wall clock starts at run start. It does not spell out when-not-to-use or an alternative retrieval path beyond the REST twin, but the intended workflow is unambiguous.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources