Skip to main content
Glama
verifyax

VerifyAX MCP

Official

Generate scenario

generate_scenario

Create agent evaluation test scenarios (info_exchange or interview) with skill tags and context. Validates tags upfront and supports batch generation, returning scenario UUIDs or structured errors.

Instructions

Generates a new test scenario of a given type (info_exchange or interview) with optional skill tags and context. Invalid tag names or incompatible tags fail immediately with 400/422 before a job is queued; use recommend_scenario_tags, search_scenario_tags, or list_compatible_tags to pick valid tag names. Typically takes 30s–2min; task-capable MCP clients receive a pollable task handle immediately, while others block until generation finishes. Set num_scenarios greater than 1 for batch mode (requires tag_pool). Returns the new scenario’s uuid, or batch uuids when batching, or a structured error if generation fails for other reasons after the job starts.

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
nameYesWorkspace-unique scenario name.
tagsNoSkill tag names (use recommend_scenario_tags, search_scenario_tags, or list_compatible_tags).
tag_poolNoRequired when num_scenarios > 1; universe of tag names to sample from.
total_tagsNoBatch only; tags drawn per scenario from tag_pool.
include_tagsNoBatch only; tags required in every scenario (subset of tag_pool).
num_scenariosNoBatch size. 1 = single scenario (default). Greater than 1 requires tag_pool.
scenario_typeYes
context_promptNo
max_tags_per_npcNoBatch only; caps tags per NPC.

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed3 schema fields changedv0.3.7
    • removedInput schema / additionalProperties
      Removed value: -false
    • addedInput schema / properties / max_tags_per_npc / maximum
      Added value: +9007199254740991
    • changedInput schema / properties / tags / description
      Previous value: -"Skill tag names (use list_compatible_tags to pick valid ones)."New value: +"Skill tag names (use recommend_scenario_tags, search_scenario_tags, or list_compatible_tags)."
  2. Addedv1.0.1

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations declare only that this is not read-only and not destructive. The description adds substantial context beyond that: validation failures return 400/422 synchronously before any job is queued, typical latency is 30s–2min, task-capable clients get a pollable handle while others block, and post-start failures return a structured error.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loads the core purpose and validation behavior, then layers timing, async semantics, batch mode, and return values in a logical order. Each sentence carries information, though the density of caveats makes it heavier than ideal.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema, and the description compensates by describing the return value (scenario uuid, batch uuids, or structured error). Combined with validation behavior, latency, and async handling, an agent has everything needed to call this correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 78%, so most parameters are documented structurally, but the description adds cross-parameter semantics the schema does not: num_scenarios > 1 requires tag_pool, and that mode is called "batch mode". It does not explain total_tags, include_tags, or max_tags_per_npc beyond what the schema already states, so it stops short of full compensation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Opens with a specific verb+resource ("Generates a new test scenario") and immediately enumerates the scenario_type values (info_exchange or interview). It clearly differentiates itself from the sibling tag-discovery tools by naming them as inputs to this operation rather than alternatives to it.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly routes the agent: use recommend_scenario_tags, search_scenario_tags, or list_compatible_tags to pick valid tag names. It states the condition for batch mode (num_scenarios > 1 requires tag_pool) and explains the client-dependent blocking vs task-handle behavior, so an agent knows exactly what to expect.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.