Skip to main content
Glama

Generate tests from story

generate_tests_from_story

Generates API test cases from a user story or natural-language idea for a Postman collection. Use it to turn requirements into test scenarios for a given collection source.

Instructions

Generates API test cases from a user story or natural-language idea for a Postman collection (path, URL, or UID).

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "collectionSource": "<collectionSource>",
      "userStory": "<userStory>"
    },
    "name": "generate_tests_from_story"
  }
}

Input Schema

NameRequiredDescriptionDefault
overwriteNoAllow replacing outputPath
userStoryYesUser story, acceptance criteria, or natural-language idea
outputPathNoOutput path (required for a remote collection source)
maxTestCasesNoMaximum number of scenarios
collectionSourceYesCollection (path, https URL, or Postman UID)
additionalContextNoUser answers to questions from a previous call
acceptanceCriteriaNoAcceptance criteria already separated
includePreRequestScriptsNoAllow prerequest scripts in generated cases

Output Schema

NameRequiredDescriptionDefault
llmYes
statusYes
retriesYes
skippedYes
warningsYes
testCasesYes
folderNameYes
outputPathYes
validationYes
assumptionsYes
missingInfoYes
requiredVariablesYes
acceptanceCriteriaYes

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observedv0.1.0

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=false, openWorldHint=true, destructiveHint=false, so the agent knows this is a non-destructive write with external access. The description adds nothing beyond that: it doesn't mention that output is written to a file/collection, that outputPath is required for remote sources, or that overwrite can replace existing files. Nor does it flag the clarifying-question workflow implied by additionalContext.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence with no filler; the verb and scope come first. It is efficient, though it borders on under-specification for a tool with eight parameters and a multi-step workflow.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return values needn't be explained, and the annotations cover the safety profile. However, for an open-world generative tool with an overwrite/outputPath write path and an iterative additionalContext cycle, the one-line description leaves workflow-relevant behavior undisclosed (minimum story length, generation limits, follow-up calls).

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so every parameter (maxTestCases, includePreRequestScripts, overwrite, etc.) is already documented. The description restates the 'path, URL, or UID' formats for collectionSource rather than adding new meaning, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Generates API test cases') plus the input source ('user story or natural-language idea') and the target ('a Postman collection (path, URL, or UID)'). This distinguishes it implicitly from siblings like generate_test_dataset or generate_cicd_pipeline, though it never names an alternative explicitly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No when-to-use or when-not-to-use guidance, and no mention of the related alternatives (e.g., generate_test_dataset for data, run_newman_collection for execution). The agent can infer the trigger is 'user gives a story/idea', but nothing prevents misuse or routes between similar generators.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.