Skip to main content
Glama
Sign In

Testsy

by dev.testsy
OverviewDiscussionsAdmin

Run a test

run_test

Start a run of one test. Runs take minutes: poll get_run until final is true. watchUrl opens the run live without signing in. App tests run the newest build of their app unless buildId names another (see list_builds). watchUrl is a view-only link anyone can open (no sign-in) to watch the run live and read its result for a while after it ends; give it to the person waiting for the result. Absent when the workspace turned watch links off. REST: send an Idempotency-Key header so a retried request does not start a second run.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "testId": "<testId>"
    },
    "name": "run_test"
  }
}

Input Schema

NameRequiredDescriptionDefault
urlNoWebsites: start at this address instead of the test's saved one — a preview or staging deploy of the same site. The run then explores (cold).
modeNoauto (default) replays the learned path when it is fresh, else explores; cold always explores; warm only replays.
deviceNoApp tests: this run on a phone or tablet, whatever the test is set to (the test does not change; see list_devices)
testIdYesTest id (t_…)
buildIdNoApps: run this build (ab_…)
osVersionNoApp tests: this run on this OS version, for example latest, iOS 26 or Android 14 (the test does not change; see list_devices)
orientationNoApp tests: this run started in this orientation, default, portrait or landscape (the test does not change)

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observed

TDQS

A4.5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only tell the agent this is a non-read-only, non-idempotent, non-destructive write; the description adds the materially important behavior: runs take minutes and are asynchronous, watch links are view-only and anonymous, watchUrl is absent when the workspace disables them, and REST callers should send Idempotency-Key to avoid duplicate runs. Those are exactly the traits an agent would otherwise get wrong.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The operational facts are front-loaded (start, then poll, then share the watch link). It loses a little to redundancy: the first sentence and the third both explain that watchUrl is a no-sign-in live view, so the same fact is stated twice in slightly different words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With seven parameters and no output schema, the description does the work of explaining the return surface (watchUrl, the final flag on the run) and the async lifecycle. It never states that the response carries the run id needed to call get_run, which is the one small gap for a tool whose caller must immediately poll.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description adds real meaning beyond the schema by stating that app tests default to the newest build of their app unless buildId names another. The remaining parameters (url, mode, device, osVersion, orientation) are left entirely to their already-thorough schema descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

"Start a run of one test" gives a specific verb plus resource and scopes it to a single test, which separates it from run_project (whole project) and from get_run (read status). The naming of list_builds and get_run inside the description reinforces the boundaries.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It states the follow-up loop explicitly ("poll get_run until final is true") and points to list_builds when a non-default build is wanted. What is missing is an explicit contrast with run_project for the whole-project case, so the agent must infer that from the sibling name alone.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

All Testsy toolsInspect
Try in Browser
Claude Code, inside the Godot 4 editorFlockbay puts an AI agent in the real Godot 4 editor: it writes GDScript, runs your game and fixes what breaks. Free, or bring your own Claude Code or Codex.flockbay.comAd

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources