Skip to main content
Glama

run_test

Executes a saved test on a new device or specified session, returning a run ID for get_run. Use it to verify app behavior on live cloud devices.

Instructions

Run a saved test. With session_id it runs on that session (the device you are driving). Without it, VibeView always starts a new device of the test app's platform, runs the test and stops the device (it does not use a session you hold from dev_start; pass session_id to run on a session you hold). Returns a run id for get_run.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "test": "<test>"
    },
    "name": "run_test"
  }
}

Input Schema

NameRequiredDescriptionDefault
testYesTest id (from create_test or the Tests page).
ai_useNosmart (default): replays recorded and learned paths, and uses AI only where the screen changed or a step has no path yet; no_ai: replays only, no AI and no usage credit (a step with no path is not checked); ai_only: the AI does every step.
session_idNoRun on this session instead of a new device.
device_modelNoDevice model for a new device (from list_device_models); omitted: any free one. A model of another platform (same form factor) runs the test's app for that platform: the app linked in its app family, else the app with the same bundle id / package. The reply names the platform and app the run went to.
start_from_current_screenNoWith session_id: no reset before the test.

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observedv0.1.0

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only say readOnlyHint=false; the description adds the substantive side effects: a new device is started, the test is run, and the device is then stopped. It also discloses the return contract (a run id for get_run), which the absent output schema does not. It does not say whether the call blocks or returns before the run completes, or what happens on a failing test, so it stops short of fully transparent.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the one-line purpose, then the two branches in order, with the return value last. The nested parenthetical about dev_start is slightly awkward but carries the single most important disambiguation, so it earns its space.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a five-parameter mutation tool with no output schema, the description covers the surprising behavior (automatic device lifecycle), the session branching, and the return value. Residual gaps are about execution semantics — synchronous vs asynchronous, failure behavior, cost — rather than about how to call it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description adds real meaning: session_id is characterized as "the device you are driving" and its absence is explained as a provisioning-and-teardown cycle rather than a default. The schema already explains ai_use and device_model well, and the description correctly does not duplicate them.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ("Run a saved test") and immediately distinguishes the two execution modes: with session_id it targets the device you are driving, without it a new device is provisioned. It also names a sibling concern explicitly ("it does not use a session you hold from dev_start"), so an agent can tell the two apart without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives the decision rule directly: pass session_id to run on a session you already hold, omit it to get a disposable new device. It also pre-empts the most likely mis-assumption — that an existing dev_start session is reused — and states the remedy. The only unmentioned alternative is run_group, which is not required to make this tool selectable.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.