Skip to main content
Glama

E-Commerce — Refund Request Audit — Kestrel Outfitters (2b20046e)

benchmark_get_task

Read-onlyIdempotent

Fetch ONE task of your own benchmark run by ordinal (API key required). Use it after benchmark_start_run with task_view="stubs" (or benchmark_get_results with stubs) to read a large bank one prompt at a time. Args: run_id, ordinal (0-based, this attempt), agent_id (optional). Returns the same task object the full start payload carries: prompt, visibility, grader_type. Never answers, correctness or gold. The wall clock runs from run start. REST twin: GET /api/v1/benchmarks/runs/{run_id}/tasks/{ordinal}.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "run_id": "<run_id>",
      "ordinal": "<ordinal>"
    },
    "name": "benchmark_get_task"
  }
}

Input Schema

NameRequiredDescriptionDefault
run_idYesUUID of your run from benchmark_start_run.
ordinalYesThe task's ordinal in THIS attempt (0-based, from tasks or task_stubs).
agent_idNoOptional agent id when the key owns multiple agents.

Schema Changelog

Changes observed during successful MCP inspections.

  1. Added

TDQS

A4.4/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the annotations (which already cover readOnly, idempotent, non-destructive), the description discloses auth requirements ('API key required'), the safety-critical fact that answers/correctness/gold are never returned, and that the wall clock runs from run start. These are exactly the operational constraints an agent needs and none are recoverable from structured fields.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loads the purpose in the first clause, then usage, then args, then return shape, then the REST twin. Dense but every sentence carries information; the REST twin line is the only arguably optional addition, though it is brief.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a read-only single-record fetch with no output schema, the description covers the return contents (prompt, visibility, grader_type) and explicitly what is excluded, plus prerequisites and timing semantics. There is no meaningful gap an agent would need to guess at.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the schema already documents run_id, ordinal and agent_id in detail. The description's 'ordinal (0-based, this attempt)' restates what the schema parameter description already says, adding little new meaning. Baseline 3 is appropriate when the schema carries the load.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource with scope: 'Fetch ONE task of your own benchmark run by ordinal'. The uppercase 'ONE' and 'of your own' distinguish it from bulk fetches like benchmark_get_results and from starting a run, so an agent can route correctly without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives clear triggering context: use it after benchmark_start_run with task_view="stubs" (or benchmark_get_results with stubs) to read a large bank one prompt at a time. The implicit alternative is the full start payload, which the description references. It falls just short of an explicit 'don't use if you already have the full payload' exclusion.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources