Skip to main content
Glama

Test a feature (focused)

qa_test_feature

Tests a named feature in an Android or iOS app by planning test cases or executing an exploration, then reports pass, fail, or blocked results.

Instructions

Test one named feature (e.g. login, checkout). mode:"execute" (default): a job that explores toward it, records pass/fail/blocked per case, and writes a report (poll qa_job_status). mode:"plan" (read-only): scope, cases, fixtures, plan. Without sessionId it bootstraps from projectRoot (consent-gated). Auth, paywall, or missing-fixture gates report blocked, not failed.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "feature": "<feature>"
    },
    "name": "qa_test_feature"
  }
}

Input Schema

NameRequiredDescriptionDefault
modeNoexecute (default, job) | plan (read-only) | interactive (until the first question).
limitNo
deviceNoDevice/simulator to bootstrap on.
approveNo
featureYesThe feature to test, in natural language.
platformNo
consentIdNoBootstrap consent (boot/install/launch).
sessionIdNoPrepared session (else bootstrap from projectRoot).
timeoutMsNoIgnored (kept for compatibility): the run is bounded by maxScreens / maxActions; poll qa_job_status.
creativityNo
maxActionsNoDefault 20.
maxScreensNoDefault 8.
includeCodeNo
projectRootNo
generateCasesNoGenerate test cases (default true).
allowAdversarialNo

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observedv0.1.0

TDQS

A3.9/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Adds real context beyond the annotations: execute runs asynchronously as a job that must be polled with qa_job_status, records pass/fail/blocked per case, and writes a report; plan is read-only despite readOnlyHint=false on the tool overall; bootstrap requires consent. The blocked-not-failed rule for auth, paywall, and missing-fixture gates is exactly the kind of behavioral nuance annotations cannot express. It does not describe artifact retrieval or failure/retry behavior in execute mode.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Dense and front-loaded: the core action comes first, then mode semantics, then the bootstrap path, then the blocked-vs-failed caveat. The telegraphic style ("mode:"execute" (default): a job that explores toward it") is efficient, though a couple of clauses are compressed enough to require a second read.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 16-parameter orchestration tool with no output schema, the description covers the essentials: how the run executes, how results are observed (poll qa_job_status, report written), and how gated conditions surface. Gaps remain around the "interactive" mode and how to fetch the produced report or artifacts, but the calling contract is largely complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 56% across 16 parameters, and the description compensates for the most consequential ones: mode semantics, sessionId vs projectRoot bootstrap, and consentId gating. However, many parameters (limit, approve, platform, device, creativity, allowAdversarial, includeCode, generateCases) get no added meaning, and the "interactive" enum value is not explained even though it differs from the two modes described.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource — "Test one named feature (e.g. login, checkout)" — with a clear single-feature scope. It does not name or distinguish itself from nearby siblings such as qa_test_this or qa_explore, so an agent must infer the boundary, but the purpose itself is unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives concrete selection logic: mode "execute" is the default and produces a job to poll via qa_job_status, while mode "plan" is read-only and returns scope/cases/fixtures/plan. It also explains the sessionId-vs-projectRoot path and the consent gate. It stops short of explicit when-not-to-use guidance versus alternative tools like qa_explore or qa_flow_run.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.