Skip to main content
Glama

eschaton_fast_batch

Runs test matrices across multiple code submissions in parallel and short-circuits on first failure to evaluate expected outputs in batch.

Instructions

Parallel early-exit batch matrix execution: Runs test matrices across submissions concurrently with sub-millisecond short-circuiting on failure.

Tool call template

This tool declares no required parameters. Add optional parameters from the input schema as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {},
    "name": "eschaton_fast_batch"
  }
}

Input Schema

NameRequiredDescriptionDefault
testsNo
language_idNo
source_codeNo
submissionsNoList of code submissions to evaluate

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observedv1.0.6

TDQS

C2.9/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden, and it does disclose two real traits: concurrent execution and early-exit short-circuiting on first failure. However, it omits critical behavior for a code-execution tool — sandboxing, resource limits, what counts as 'failure', and whether partial results are returned.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single sentence with the core action front-loaded before the colon and no filler. It is slightly jargon-heavy ('matrix execution', 'sub-millisecond short-circuiting') but every clause carries information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with no annotations, no output schema, and a confusing four-parameter schema at 25% coverage, the description is far too thin. It should explain the two submission input modes, what is returned on early exit, and how results map to submissions.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is only 25%, and the description names none of the four parameters. It does not clarify the ambiguous dual input shape (top-level tests/source_code/language_id versus the submissions array) or the language_id default of 54, leaving the agent to guess which input mode applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a concrete verb and resource: runs test matrices across submissions concurrently. The agent can tell it is a batch evaluation executor, though it never explains what a 'matrix' means or how it differs from the similarly named nephis_fast_batch sibling.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance on when to use this tool versus eschaton_execute_suite, nephis_fast_batch, or the other eval tools, nor any prerequisites or exclusions. The only implicit usage cue is the performance framing ('fast', 'parallel'), which is a benefit statement rather than a selection rule.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Deploy Server

Other Tools