Skip to main content
Glama

Run a smoke test

qa_smoke

Run a server-side smoke test that launches the app, executes baseline health checks and saved flows, then records QA notes for reporting.

Instructions

Run a smoke test server-side: optionally launch, run the baseline (snapshot quality + health + evidence screenshot), then every saved flow in .swipium/flows. Records a qa_note per workflow; call qa_report after. Needs a prepared device.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "sessionId": "<sessionId>"
    },
    "name": "qa_smoke"
  }
}

Input Schema

NameRequiredDescriptionDefault
launchNoLaunch first (default true with an appId).
runFlowsNoRun saved .swipium/flows (default true).
sessionIdYes
variablesNo${VAR} values for flows (over stored inputs, SWIPIUM_* env).

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observedv0.1.0

TDQS

A3.9/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only state that this is non-read-only, non-destructive and non-idempotent. The description adds real behavioral context beyond them: execution happens server-side, the run is multi-step and ordered, a qa_note is recorded per workflow, and a prepared device is required. It does not discuss failure modes or what happens if flows fail partway, so it is not a 5.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two dense sentences, front-loaded with the action and followed by the operational requirements, with no filler. The parenthetical baseline enumeration is long but each item is informative.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description compensates by describing the artifacts produced (a qa_note per workflow, evidence screenshot) and the natural next call (qa_report). For a multi-step orchestration tool this is nearly complete; only the return shape and partial-failure behavior are left implicit.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 75%, so the schema already documents most parameters. The description reinforces 'optionally launch' and 'every saved flow in .swipium/flows', mapping loosely to launch and runFlows, but it says nothing about the variables parameter or its overriding precedence, leaving that entirely to the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource ('Run a smoke test server-side') and enumerates what is included: baseline snapshot quality, health, evidence screenshot, and every saved flow in .swipium/flows. It is unambiguous about scope, though it never contrasts itself with closely related siblings such as qa_flow_run or qa_first_run.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives clear operating context ('Needs a prepared device') and explicit sequencing ('call qa_report after'), so an agent knows the prerequisite and the follow-up step. It lacks any when-not guidance or explicit named alternative, which keeps it short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.