Skip to main content
Glama

Test this app (autopilot)

qa_test_this

Test Android or iOS apps by building or locating the app, launching it on an emulator or simulator, running smoke and exploration flows, and generating reports with screenshots, logs, and UI dumps.

Instructions

Start here to test a mobile app (Android/iOS): finds or builds the app, picks an emulator/simulator, runs prepare > smoke > explore > report > suite as a job (mode:"execute", default; pass waitForCompletion:true or poll qa_job_status). mode:"plan" only plans, no side effects. One consent covers boot/install/build; every terminal state has a report. iOS without WDA: visual-only smoke.

Tool call template

This tool declares no required parameters. Add optional parameters from the input schema as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {},
    "name": "qa_test_this"
  }
}

Input Schema

NameRequiredDescriptionDefault
goalNoDefault: smoke, then a suite ("smoke" is fastest). Presets explore/generateSuite/stopOnNeedsInput; explicit flags win.
modeNoexecute (default, job) | plan (no side effects) | interactive (until the first question).
deviceNo
approveNo
exploreNoexecute: run guided exploration after the smoke.
goalTextNoreproduce_bug: the bug/flow to focus on.
platformNoForce a platform (default inferred).
consentIdNo
fastSmokeNoLaunch + smoke only, no suite (ignored with goal/generateSuite).
sessionIdNoReuse a session.
timeoutMsNowaitForCompletion window (default 45000, max 50000; SWIPIUM_MAX_WAIT_MS raises both; larger values are clamped); still running after it > state:"running" + jobId, poll qa_job_status.
projectRootNo
responseModeNocompact = summary + URIs only; kept as the session default for later calls.
buildIfNeededNoBuild from source when no artifact exists (default true).
generateSuiteNoexecute: also generate a POM suite from the run.
allowOutsideRootNo
preferRealDeviceNoOut of scope: returns PHYSICAL_DEVICE_UNSUPPORTED.
stopOnNeedsInputNoexecute: ask for login/test data instead of testing pre-login only.
waitForCompletionNoexecute: block until done (or timeoutMs).

Output Schema

NameRequiredDescriptionDefault
okNo
modeNo
jobIdNo
stateNorunning | completed | blocked | unsafe | needs_input | ready
consentIdNo
reportUriNo
sessionIdNo
requiresConsentNo

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observedv0.1.0

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly=false, destructive=false, openWorld=false, and the description layers meaningful extra context: consent scope for boot/install/build, the guarantee that every terminal state has a report, and the iOS visual-only limitation. It still leaves out what an installed/built app does to the user's environment and how a running job is cancelled or resumed, so it is good but not exhaustive for a tool that boots emulators and installs binaries.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the 'Start here' directive and the pipeline before any caveats, and every clause carries information. It is dense with parentheticals and mid-sentence semicolons, which slightly taxes parsing for a 19-parameter tool.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With 19 parameters, four enums, an output schema, and annotations, the description covers entry point, modes, blocking-vs-polling, consent, and platform caveats well. It omits how goal presets interact with explicit flags beyond the schema's note and what happens on a needs-input terminal state, which matter for calling it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is a solid 74%, and the schema itself documents enums, defaults, timeout clamping, and the PHYSICAL_DEVICE_UNSUPPORTED behavior, so the schema carries most of the load. The description restates a few key semantics (mode plan vs execute, waitForCompletion) without adding format or interaction detail beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('test a mobile app (Android/iOS)') plus the concrete pipeline it orchestrates (prepare > smoke > explore > report > suite) and its entry-point role ('Start here'). It does not differentiate itself from plausible siblings such as qa_test_feature or qa_explore, which sound like narrower variants of the same act, so an agent cannot fully route on this sentence alone.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives clear when-to guidance: use mode:"execute" by default, mode:"plan" for no side effects, and pass waitForCompletion:true or poll qa_job_status afterwards — an explicit alternative with the condition that selects it. It also carves out scope ('preferRealDevice ... Out of scope'), though it never says when to prefer qa_test_feature or qa_explore instead of this orchestrator.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.