Testsy

Run a project

run_project

Start a run of every test in a project (a suite). Each started run has a watchUrl to watch it live without signing in. buildIds: builds to use; each app test runs the one of its own app. Tests that could not start are listed in skipped with the reason. REST: send an Idempotency-Key header to make a retry safe.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "projectId": "<projectId>"
    },
    "name": "run_project"
  }
}

Input Schema

Name	Required	Description
`url`	No	Websites: every website test starts at this address (a preview or staging deploy of the same site) and explores; app tests run their build as usual.
`mode`	No	auto (default) replays the learned path when it is fresh, else explores; cold always explores; warm only replays.
`device`	No	App tests: this run on a phone or tablet, whatever the test is set to (the test does not change; see list_devices)
`buildId`	No	One build id (ab_…), optional
`buildIds`	No	Build ids (ab_…), optional
`osVersion`	No	App tests: this run on this OS version, for example latest, iOS 26 or Android 14 (the test does not change; see list_devices)
`projectId`	Yes	Project id (p_…)
`orientation`	No	App tests: this run started in this orientation, default, portrait or landscape (the test does not change)

Schema Changelog

Changes observed during successful MCP inspections.

First observedNov 10, 2026

TDQS

A3.8/5.0

Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes beyond the annotations by disclosing outcomes the agent must handle: each run yields a watchUrl for live viewing without sign-in, and tests that fail to start appear in 'skipped' with a reason. It also explains the retry contract ('send an Idempotency-Key header to make a retry safe'), which is consistent with idempotentHint=false rather than contradicting it. Missing: whether the call returns immediately (async) or what permissions are needed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with purpose, then outputs, then parameter behavior, then the API idempotency note; nearly every sentence carries information. The buildIds sentence is slightly awkwardly worded ('runs the one of its own app') but still parses.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For an 8-parameter mutation tool with no output schema, the description covers the essential return shape (watchUrl, skipped) that would otherwise be missing. Gaps are minor: async-vs-sync behavior and buildId/buildIds precedence are unstated, and annotations already cover the safety profile.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description adds genuine meaning for buildIds ('builds to use; each app test runs the one of its own app') that the schema does not convey. It does not resolve precedence when both buildId and buildIds are supplied, which is the remaining ambiguity.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource with scope: 'Start a run of every test in a project (a suite).' The 'every test in a project' phrasing implicitly contrasts with run_test, but the description never names an alternative, so the differentiation is left to inference rather than made explicit.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is only implied: an agent can infer this is the tool for running a whole suite rather than one test. There is no statement of when to prefer run_test, no prerequisite for project setup/build availability, and no guidance on waiting (wait_run) versus watching the returned watchUrl.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

All Testsy tools Inspect

Try in Browser

Claude Code, inside the Godot 4 editorFlockbay puts an AI agent in the real Godot 4 editor: it writes GDScript, runs your game and fixes what breaks. Free, or bring your own Claude Code or Codex.flockbay.comAd

Glama MCP Gateway

Add one secure layer between your agents and this server.