Skip to main content
Glama

eschaton_execute_suite

Run code against any of 10 Eschaton engine suites using custom testcase matrices, including stdin, expected output, files, schemas, and special judge checkers.

Instructions

Execute code across any of the 10 Eschaton engine suites (Omni, Apex, Void, Complexity, Sanitizer, MemoryProfile, FileIO, DBMS, SpecialJudge) with custom testcase matrices.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "language_id": "<language_id>",
      "source_code": "<source_code>"
    },
    "name": "eschaton_execute_suite"
  }
}

Input Schema

NameRequiredDescriptionDefault
suiteNoExecution suiteOmni
verboseNoVerbose mode (0=early-exit short-circuit, 1=full diagnostics for Void suite)
databaseNoDatabase engine for DBMS suite
test_casesNoList of testcases with stdin, expected_output, and optional schema_sql or files
language_idYesLanguage ID (e.g. 54 for C++, 71 for Python 3, 202 for C++20, 204 for Python 3.11)
source_codeYesSource code to execute
checker_codeNoCustom validator checker source code for SpecialJudge suite
checker_language_idNoChecker language ID (default 202)

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observedv1.0.6

TDQS

C2.8/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full behavioral burden, and it discloses almost nothing beyond the existence of suites. It says nothing about sandboxing, timeouts, resource limits, permissions, or what execution produces — all critical for a tool that runs arbitrary submitted source code.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

One front-loaded sentence with no filler, which is good, but enumerating nine suite names duplicates the suite enum verbatim and the sentence is internally inconsistent, claiming '10' suites while the enum and parenthetical list both contain nine.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For an 8-parameter arbitrary-code-execution tool with no annotations and no output schema, the description leaves too much unexplained: no mention of what is returned, how test-case results or checker output are reported, or how suite selection changes behavior. The miscount of suites also undermines trust in the enumeration.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3; each parameter, including enums for suite, verbose, and database, is already documented in the schema. The description adds only the vague phrase 'custom testcase matrices' and does not explain interactions such as verbose being Void-suite-specific or database applying only to DBMS.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

Specific verb ('Execute code') plus resource ('Eschaton engine suites') and a stated capability ('custom testcase matrices'). It is clear what the tool does, but it never distinguishes itself from the sibling-specific tools (eschaton_dbms_eval, eschaton_complexity_profile, eschaton_special_judge) that appear to do subsets of the same job.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No when-to-use or when-not-to-use guidance at all. With six sibling tools that look like per-suite specializations, an agent cannot tell whether this general tool should be preferred over, or is a fallback to, eschaton_special_judge or eschaton_fast_batch.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Deploy Server

Other Tools