Skip to main content
Glama

run_interleaved

Idempotent

Compare two or more versions as matched arms in rotating task blocks so each arm sees every task at nearly the same time, controlling for latency or cost drift; blocks until all arms finish.

Instructions

Run two or more versions as matched arms in rotating task blocks, so each arm sees every task at nearly the same time. Use it for parent-versus-candidate measurements when latency or cost may drift; use run_launch for one arm or a detached run. Blocks until all arms finish. Writes run records, results and traces, and costs whatever the scenario's tasks cost. Repeating the call with identical arms and parameters resumes and skips committed slots; a new plan needs fresh run ids. Returns each arm's run status, keyed by run id.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "config": "<config>",
      "arms": "<arms>"
    },
    "name": "run_interleaved"
  }
}

Input Schema

NameRequiredDescriptionDefault
armsYesMap of new run id to version id, at least two entries, for example {"parent-1": "<sha256>", "child-1": "<sha256>"}. Run ids are 1 to 80 letters, digits, `_` or `-`, starting with a letter or digit.
seedNoInteger that rotates which arm starts first in each block. Default 0.
splitNoTask split: `diagnosis` to explore and read failures, `selection` for gate evidence. Default `selection`. Confirmation and final splits are never available.selection
configYesPath of the scenario YAML file on the server host, absolute or relative to the server's working directory, for example `scenario.yaml`. It selects the workspace; a server started with --config rejects any other path.
repeatsNoTimes each task runs, at least 1. Default 1.
workersNoParallel task workers, 1 to 256. Default 4.

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed10 schema fields changedv0.1.9
    • addedInput schema / properties / arms / description
      Added value: +"Map of new run id to version id, at least two entries, for example {\"parent-1\": \"<sha256>\", \"child-1\": \"<sha256>\"}. Run ids are 1 to 80 letters, digits, `_` or `-`, starting with a letter or digit."
    • addedInput schema / properties / config / description
      Added value: +"Path of the scenario YAML file on the server host, absolute or relative to the server's working directory, for example `scenario.yaml`. It selects the workspace; a server started with --config rejects any other path."
    • addedInput schema / properties / repeats / default
      Added value: +1
    • addedInput schema / properties / repeats / description
      Added value: +"Times each task runs, at least 1. Default 1."
    • addedInput schema / properties / seed / default
      Added value: +0
    • addedInput schema / properties / seed / description
      Added value: +"Integer that rotates which arm starts first in each block. Default 0."
    • addedInput schema / properties / split / default
      Added value: +"selection"
    • addedInput schema / properties / split / description
      Added value: +"Task split: `diagnosis` to explore and read failures, `selection` for gate evidence. Default `selection`. Confirmation and final splits are never available."
    • addedInput schema / properties / workers / default
      Added value: +4
    • addedInput schema / properties / workers / description
      Added value: +"Parallel task workers, 1 to 256. Default 4."
  2. First observedv0.1.7

TDQS

A4.9/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations cover the safety profile (writes, open-world, non-destructive, idempotent), but the description adds what they cannot: it blocks until all arms finish, writes run records, results and traces, incurs the scenario's task cost, and resumes by skipping committed slots on repeat calls. It also explains that a new plan requires fresh run ids, which is exactly the resumption caveat behind idempotentHint=true.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Six tight sentences, front-loaded with purpose, then usage, then side effects and resumption. No filler, and each sentence carries a distinct fact an agent would otherwise have to discover.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema, so the description compensates by stating the return shape ('each arm's run status, keyed by run id'), along with blocking behavior, cost exposure and repeat-call semantics. Nothing an agent needs to invoke this correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description adds meaning the schema does not spell out: the arms map keys are run ids whose identity drives resume-versus-new-plan behavior. It does not add anything for seed, split, repeats or workers, which the schema already handles well.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Run two or more versions as matched arms in rotating task blocks') and immediately explains the mechanism that distinguishes it from a plain launch. An agent can tell it apart from run_launch without opening either schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives an explicit selection rule: use this for parent-versus-candidate measurements where latency or cost may drift, and use run_launch for one arm or a detached run. The condition and the named alternative are both present.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.