Skip to main content
Glama

Generate test dataset

generate_test_dataset

Creates a JSON or CSV iteration-data file with positive, negative, and boundary-value test cases for a Postman or Newman collection, so you can run data-driven tests.

Instructions

Generates a --iteration-data file (.json or .csv) with positive, negative, and boundary-value cases.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "collectionSource": "<collectionSource>",
      "outputPath": "<outputPath>"
    },
    "name": "generate_test_dataset"
  }
}

Input Schema

NameRequiredDescriptionDefault
overwriteNoAllow replacing an existing file
outputPathYesOutput path (must end in .json or .csv)
scenarioCountNoNumber of rows
targetItemNameNoItem to parameterize (if omitted, use all variables, max. 15)
collectionSourceYesCollection (path, https URL, or Postman UID)

Output Schema

NameRequiredDescriptionDefault
rowsYes
formatYes
columnsYes
outputPathYes

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observedv0.1.0

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=false, destructiveHint=false, and openWorldHint=true, so the write-and-network profile is partly covered. The description adds genuine value by specifying the generated case categories, but it omits that the file is written to disk at outputPath, that existing files can be replaced, and that the collection may be fetched from a URL or UID.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single tightly written sentence with the key artifact front-loaded and no filler. The '(.json or .csv)' clause lightly duplicates the outputPath schema constraint but is not wasteful enough to penalize further.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists and all five parameters are documented, so the remaining burden is behavioral. For a tool that writes files and pulls a collection from an external source, the description does not convey where output lands, overwrite behavior, or how case types relate to the generated rows, leaving notable gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so scenarioCount, targetItemName, collectionSource, outputPath, and overwrite are all documented in the schema. The description adds no further meaning about these parameters (e.g. how cases map to scenarioCount or how targetItemName affects variable coverage), so the baseline of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description gives a specific verb (Generates) and a specific artifact (--iteration-data file in .json or .csv) containing positive, negative, and boundary-value cases, which pins down the output content well. It does not name or contrast with any sibling such as generate_tests_from_story or generate_newman_report, so it falls short of the top mark.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is only implied: the '--iteration-data' framing signals this feeds data-driven collection runs, but there is no explicit when-to-use statement, no prerequisites such as needing a collection source, and no pointer to an alternative tool for related generation tasks.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.