Skip to main content
Glama

create_test

Creates an app test from plain-language steps (one per line; checks start with “Verify”), returns its ID for run_test, and after the AI-driven first pass replays the saved path.

Instructions

Create a test for an app from plain-language steps, one per line; a check is a step starting with "Verify". Returns the test id for run_test. The first run is driven by AI; once it passes, later runs replay the saved path.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "app": "<app>",
      "name": "<name>",
      "steps": "<steps>"
    },
    "name": "create_test"
  }
}

Input Schema

NameRequiredDescriptionDefault
appYesApp id (from list_apps).
nameYesTest name.
groupNoGroup id to add the test to (optional).
stepsYesThe steps, one per line.
if_existsNoWhen a test with this name already exists for the app: update replaces it, new creates another one beside it, fail refuses. Omitted: an identical test is reused, a different one is refused.

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observedv0.1.0

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotation only declares readOnlyHint=false, so the description does real work: it discloses the return value (test id) despite no output schema, and explains the two-phase execution model (AI-driven first run, then saved-path replay) which is non-obvious and affects how an agent should interpret later runs. It omits permission requirements and what happens on failure of the first AI run, keeping it short of a 5.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two compact sentences, front-loaded with the creation action, then the return value and execution model. Every clause carries information an agent needs; nothing is filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a five-parameter mutation tool with no output schema, the definition covers input format, return value, and downstream usage well. The remaining gap is modest: side effects on existing tests are delegated to the if_exists schema description, and error/permission behavior is unstated.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is already 100%, so the baseline is 3; the description adds genuine meaning beyond the schema by defining the line format for `steps` and the "Verify" convention for checks, which is not encoded in the schema. It adds less for `app`, `name`, `group`, and `if_exists`, which the schema fully documents.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ("Create a test for an app") plus the input format ("plain-language steps, one per line") and a domain rule ("a check is a step starting with 'Verify'"). It also names the adjacent lifecycle tool (run_test), so an agent can place it relative to siblings without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase "Returns the test id for run_test" tells the agent this is the setup step preceding run_test, giving clear sequencing context. It does not spell out when to prefer updating an existing test, though the schema's if_exists enum partly covers that; no explicit exclusions are stated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.