Skip to main content
Glama
Yan-Vi

easy-spec-mcp

by Yan-Vi
OverviewSchemaSourceRelated ServersDiscussionsAdmin
JavaScript
Local

live_run_suite

Run a test suite live in parallel browser tabs, retrying failures, and get a runId for immediate polling of progress.

Instructions

Runs a suite (or a single suite entry) live, in parallel, across workerCount concurrently-opened dedicated tabs (a global variable, see set_variable/get_variables -- defaults to 1), trying each failing run up to attempts TOTAL times (another global variable, defaults to 1) on the same worker tab before giving up. A run that fails at least once but eventually passes is reported as "flaky", never "passed" or "failed". Pass exactly one of path (every entry placed at or under that suite folder, each expanded to its own saved run(s) -- see add_to_suite/list_suites; entries are NOT deduplicated by scenario, two placements of the same scenario with different saved params both genuinely run -- "" or omitted means the WHOLE suites tree) or scenarioIds (an explicit list bypassing suite membership entirely, each run with that scenario's own default dataset). Returns immediately with a runId; poll live_get_suite_run (or live_list_suite_runs) to check progress/outcome -- this does NOT wait for the run to finish. Each individual scenario attempt is also an ordinary scenario run, independently visible via live_list_scenario_runs/live_get_scenario_run. Requires the side panel open and connected.

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
pathNo
projectNoPath to the project root (same folder the side panel connects to). Defaults to the EASYSPEC_PROJECT environment variable if omitted.
sessionNoWhich connected side panel to target, by its connection id (see live_status) -- only needed when more than one side panel is connected to the same project.
scenarioIdsNo

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observedv1.1.4

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and does well: it discloses parallel tab usage, retry count, flaky-run reporting, async return behavior, run visibility, and the side panel requirement. It does not detail tab lifecycle or cancellation, but the most important behavioral aspects are covered.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is dense and front-loaded with the core action, and every sentence adds useful information. It is somewhat long due to parenthetical clarifications, but this is a complex tool with retries, path semantics, and async behavior that benefit from explanation.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity and absent output schema, the description is complete enough: it explains asynchronous behavior, retries, flaky status, run visibility, path/scenario selection, and the side panel prerequisite. No critical operational detail appears to be missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema leaves path and scenarioIds undescribed, but the description provides rich meaning for both: path expands suite entries, empty path means the whole tree, entries are not deduplicated, and scenarioIds bypasses suite membership with default datasets. The project and session parameters are already documented in the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's specific purpose: running a suite or single suite entry live, with parallel execution and retries. It also distinguishes the tool from sibling live replay/start tools by describing the returned runId and the visibility of individual scenario attempts.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives clear usage context: requires the side panel to be open and connected, pass exactly one of path or scenarioIds, and poll for results because the call returns immediately. It does not explicitly contrast this tool with alternatives like live_start_run or live_replay_flow, but the suite-specific focus makes the intended use clear.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Deploy Server

Other Tools