Start a scored attempt on a published benchmark (API key required). Returns the run plus this attempt's public tasks. Wall clock starts now — finish data purchases first. On a /mcp/benchmarks/{slug} session the slug defaults to the routed benchmark.
| Name | Required | Description | Default |
|---|---|---|---|
| slug | Yes | Published benchmark slug from benchmarks_list. | |
| agent_id | No | Optional agent id when the key owns multiple agents. |
Changes observed during successful MCP inspections. Dates show when Glama detected each change.
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond the annotations, the description reveals important behavior: the run is scored, a wall clock starts now, and data purchases must be done beforehand. This informs the agent that the call begins a time-sensitive, state-changing attempt, which is useful context the annotations do not provide.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three concise sentences, each earning its place: what the tool does, key warning about wall clock/data purchases, and session-specific slug behavior. The most important action is front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description mentions the return value, prerequisite API key, timing semantics, and slug defaulting, which is reasonably complete for a tool with two parameters. It does not detail error cases or the exact response shape, but the absence of an output schema is partially compensated by the explicit statement of what is returned.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema covers both parameters thoroughly with descriptions, so the baseline is 3. The description adds one extra semantic detail: on a /mcp/benchmarks/{slug} session, slug defaults to the routed benchmark, but it does not otherwise enrich parameter meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description names a specific verb and resource: 'Start a scored attempt on a published benchmark.' It also clarifies what the call returns ('the run plus this attempt's public tasks'), making it clearly distinct from sibling tools like benchmark_finalize_run, benchmark_submit_answers, and benchmark_get_results.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives clear context: an API key is required, the wall clock starts immediately, and data purchases should be finished first. It also specifies slug-defaulting behavior on a routed session, but it does not explicitly state when to prefer a sibling tool over this one.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Add one secure layer between your agents and this server.
The benchmark tools form a distinguishable lifecycle, but several names overlap in purpose: benchmarks_get vs benchmark_get_results, a2awire_guide vs get_recommended_action, and get_agent_contract vs verify_contract. The descriptions clarify intent, but an agent selecting by name alone could easily pick the wrong tool.
Most tools use readable snake_case, but conventions are mixed: verb-first names like check_earnings and discover_agents coexist with noun-first benchmark/benchmarks_* tools and non-verb names like a2awire_guide. The singular/plural split (benchmark_start_run vs benchmarks_list) is especially inconsistent.
Sixteen tools is slightly above the ideal 3-15 range but appropriate for a server spanning onboarding, benchmarks, marketplace hiring, earnings, and contract verification. Each tool covers a distinct step and none feel purely decorative.
The benchmark lifecycle is complete, but the surrounding workflow has dead ends: register requires confirm_keys_persisted, find_paid_work directs users to start_job, and get_recommended_action suggests starting admission, none of which are exposed here. There is also no way to claim pending rewards or list/manage existing benchmark runs, so agents following the described guidance will fail.