Skip to main content
Glama

run_flow_autotest

Read-only

Runs deterministic behavioural tests against flows that are ALREADY applied (compiles them to an AST and simulates a user). Call after apply_actions to verify a build; read summary and the failed checks, patch with apply_actions, re-run. Mutates nothing. The smoke layer runs on its own with no input: it walks every entry, taps every button, answers every input step, and reports crashes, dead buttons, unresolved placeholders, and values the bot failed to store. Pass scenarios to also replay specific user journeys (at most 6) — that is the only way to assert exact texts or exact stored values. Returns { passed, smoke, scenarios, summary }. passed is false when any scenario fails or is unverified. Harness gaps are reported separately and never mean the bot is broken. A summary saying coverage is "none" means nothing was testable. Nothing is sent to real users and no state is written. An outside GET call no scenario fixture answers is answered from a recorded real response: one the server itself fetched for this application in the last 24 h (the endpoint check apply_actions runs), used only for exactly the same address and query values. outsideCalls says how many calls were answered that way, how many stayed coverage gaps, and when each answer was recorded.

Input Schema

NameRequiredDescriptionDefault
flowIdsYesRequired. Ids of the ALREADY-APPLIED flows to test — normally the flows apply_actions just created or changed, taken from its idRemap. Flows they link into are compiled too but are not crawled as entries. Flows that link INTO them compile too and are crawled only for a requested flow with no trigger of its own (a flow opened from a menu button is tested through the flow that opens it); a requested flow nobody can start fails with a no_entry check.
scenariosNoOptional user journeys to replay on top of the smoke crawl. Omit to run the smoke layer alone.
applicationIdNoApplication (workspace) id. Optional: an application-scoped key (app_...) defaults to its own application, but a personal key (usr_...) has no default and omitting it fails with MCP_APPLICATION_REQUIRED. Call list_applications to get the id.

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed1 schema field changed
    • changedInput schema / properties / scenarios / items / properties / steps / description
      Previous value: -"Ordered steps, alternating an ACTION with an ASSERTION about the reply to the action right before it. When accepted checks are supplied, use {\"do\":\"expect_check\",\"checkId\":\"<accepted id>\"} for their outcomes; the server selects the observation channel. Optional contains:[\"literal journey value\"] strengthens message/notification checks only. An expect_check about someone other than the acting person (an outcome for person \"other\") needs \"other\": \"<as_contact id>\", the id that person has in this scenario (\"$SELF\" is the first person, once another one acts); e.g. {\"do\":\"start\",\"command\":\"/start\"}, {\"do\":\"tap\",\"button\":\"Apply\"}, {\"do\":\"as_contact\",\"id\":\"admin\"}, {\"do\":\"tap\",\"button\":\"Approve\"}, {\"do\":\"expect_check\",\"checkId\":\"c2\",\"other\":\"$SELF\"}. Each step is one object keyed by `do`: {\"do\":\"start\",\"command\":\"/start\"}, {\"do\":\"send\",\"text\":\"...\"}, {\"do\":\"send\",\"text\":\"...\",\"aiChooses\":\"<exit name>\"} (the AI assistant block answering this message takes that exit: an intent or tool name as the block lists it, or \"no_match\" / \"frustration\" — no AI runs in tests, so without aiChooses the block only chats and no branch behind an exit runs), {\"do\":\"send_media\",\"kind\":\"photo\"|\"video\"|\"audio\"|\"document\",\"caption\":\"...\"}, {\"do\":\"tap\",\"button\":\"<substring of the label>\"}, {\"do\":\"pay\"}, {\"do\":\"abandon_payment\",\"text\":\"...\"}, {\"do\":\"resume_delays\"}, {\"do\":\"simulate_event\",\"event\":\"member_join\"|\"join_request\"}, {\"do\":\"expect\",\"anyOf\":[\"verbatim fragment the bot really sends\"],\"not\":[\"text that must NOT appear\"]} (expect needs anyOf: a list of at least one verbatim fragment of what the bot sends after the action right before it; not (texts that must NOT appear) is optional and is checked only beside anyOf, because an expect with only not passes when the bot says nothing; a text in anyOf or not that the bot cannot send is removed: a {{…}} template, an internal id (no spaces, joined by _ . : or -), and, when the bot's texts are known, a button label or words it never writes; a removed text makes the scenario an invalid test), {\"do\":\"expect_var\",\"name\":\"<variable handle>\",\"contains\":\"...\",\"changed\":true}, {\"do\":\"seed_var\",\"name\":\"...\",\"value\":\"...\"} (seed_var value \"$SELF\" becomes the simulated user id, for a gate that compares the user id with a stored owner-id variable; a workspace-owner or tag gate is opened with assume.actors, never with seed_var), {\"do\":\"as_contact\",\"id\":\"user_b\"} (every later step acts as a SECOND person — for anything two people do together: one builds or sends, the other sees it; after the switch that person sees what the bot sent them earlier and can tap its buttons, and cannot tap a button on another person's screen), {\"do\":\"expect_peer_message\",\"to\":\"user_b\",\"contains\":\"...\",\"mediaType\":\"IMAGE\"} (mediaType is optional; assert proof photos/files on the same peer message), {\"do\":\"expect_http_request\",\"urlContains\":\"/orders\",\"method\":\"POST\",\"bodyContains\":\"orderId\",\"count\":1} (assert the outgoing request; method, bodyContains and count are optional). A scenario with no expect / expect_var / expect_peer_message step asserts nothing. At most 15 steps; malformed or truncated steps make the scenario an invalid test, never a pass."New value: +"Ordered steps, alternating an ACTION with an ASSERTION about the reply to the action right before it. When accepted checks are supplied, use {\"do\":\"expect_check\",\"checkId\":\"<accepted id>\"} for their outcomes; the server selects the observation channel. Optional contains:[\"literal journey value\"] strengthens message/notification checks only. An expect_check about someone other than the acting person (an outcome for person \"other\") needs \"other\": \"<as_contact id>\", the id that person has in this scenario (\"$SELF\" is the first person, once another one acts), or \"<numeric chat id>\" of the channel or group the bot posts to — the same value the scenario seeded into the bot's chat-id variable, with no as_contact for a chat; e.g. {\"do\":\"start\",\"command\":\"/start\"}, {\"do\":\"tap\",\"button\":\"Apply\"}, {\"do\":\"as_contact\",\"id\":\"admin\"}, {\"do\":\"tap\",\"button\":\"Approve\"}, {\"do\":\"expect_check\",\"checkId\":\"c2\",\"other\":\"$SELF\"}. Each step is one object keyed by `do`: {\"do\":\"start\",\"command\":\"/start\"}, {\"do\":\"send\",\"text\":\"...\"}, {\"do\":\"send\",\"text\":\"...\",\"aiChooses\":\"<exit name>\"} (the AI assistant block answering this message takes that exit: an intent or tool name as the block lists it, or \"no_match\" / \"frustration\" — no AI runs in tests, so without aiChooses the block only chats and no branch behind an exit runs), {\"do\":\"send_media\",\"kind\":\"photo\"|\"video\"|\"audio\"|\"document\",\"caption\":\"...\"}, {\"do\":\"tap\",\"button\":\"<substring of the label>\"}, {\"do\":\"pay\"}, {\"do\":\"abandon_payment\",\"text\":\"...\"}, {\"do\":\"resume_delays\"}, {\"do\":\"simulate_event\",\"event\":\"member_join\"|\"join_request\"}, {\"do\":\"expect\",\"anyOf\":[\"verbatim fragment the bot really sends\"],\"not\":[\"text that must NOT appear\"]} (expect needs anyOf: a list of at least one verbatim fragment of what the bot sends after the action right before it; not (texts that must NOT appear) is optional and is checked only beside anyOf, because an expect with only not passes when the bot says nothing; a text in anyOf or not that the bot cannot send is removed: a {{…}} template, an internal id (no spaces, joined by _ . : or -), and, when the bot's texts are known, a button label or words it never writes; a removed text makes the scenario an invalid test), {\"do\":\"expect_var\",\"name\":\"<variable handle>\",\"contains\":\"...\",\"changed\":true}, {\"do\":\"seed_var\",\"name\":\"...\",\"value\":\"...\"} (seed_var value \"$SELF\" becomes the simulated user id, for a gate that compares the user id with a stored owner-id variable; a workspace-owner or tag gate is opened with assume.actors, never with seed_var), {\"do\":\"as_contact\",\"id\":\"user_b\"} (every later step acts as a SECOND person — for anything two people do together: one builds or sends, the other sees it; after the switch that person sees what the bot sent them earlier and can tap its buttons, and cannot tap a button on another person's screen), {\"do\":\"expect_peer_message\",\"to\":\"user_b\",\"contains\":\"...\",\"mediaType\":\"IMAGE\"} (mediaType is optional; assert proof photos/files on the same peer message), {\"do\":\"expect_http_request\",\"urlContains\":\"/orders\",\"method\":\"POST\",\"bodyContains\":\"orderId\",\"count\":1} (assert the outgoing request; method, bodyContains and count are optional). A scenario with no expect / expect_var / expect_peer_message step asserts nothing. At most 15 steps; malformed or truncated steps make the scenario an invalid test, never a pass."
  2. Changed1 schema field changed
    • changedInput schema / properties / scenarios / items / properties / steps / description
      Previous value: -"Ordered steps, alternating an ACTION with an ASSERTION about the reply to the action right before it. When accepted checks are supplied, use {\"do\":\"expect_check\",\"checkId\":\"<accepted id>\"} for their outcomes; the server selects the observation channel. Optional contains:[\"literal journey value\"] strengthens message/notification checks only. Each step is one object keyed by `do`: {\"do\":\"start\",\"command\":\"/start\"}, {\"do\":\"send\",\"text\":\"...\"}, {\"do\":\"send\",\"text\":\"...\",\"aiChooses\":\"<exit name>\"} (the AI assistant block answering this message takes that exit: an intent or tool name as the block lists it, or \"no_match\" / \"frustration\" — no AI runs in tests, so without aiChooses the block only chats and no branch behind an exit runs), {\"do\":\"send_media\",\"kind\":\"photo\"|\"video\"|\"audio\"|\"document\",\"caption\":\"...\"}, {\"do\":\"tap\",\"button\":\"<substring of the label>\"}, {\"do\":\"pay\"}, {\"do\":\"abandon_payment\",\"text\":\"...\"}, {\"do\":\"resume_delays\"}, {\"do\":\"simulate_event\",\"event\":\"member_join\"|\"join_request\"}, {\"do\":\"expect\",\"anyOf\":[\"verbatim fragment the bot really sends\"],\"not\":[\"text that must NOT appear\"]}, {\"do\":\"expect_var\",\"name\":\"<variable handle>\",\"contains\":\"...\",\"changed\":true}, {\"do\":\"seed_var\",\"name\":\"...\",\"value\":\"...\"} (seed_var value \"$SELF\" becomes the simulated user id, for a gate that compares the user id with a stored owner-id variable; a workspace-owner or tag gate is opened with assume.actors, never with seed_var), {\"do\":\"as_contact\",\"id\":\"user_b\"} (every later step acts as a SECOND person — for anything two people do together: one builds or sends, the other sees it; after the switch that person sees what the bot sent them earlier and can tap its buttons, and cannot tap a button on another person's screen), {\"do\":\"expect_peer_message\",\"to\":\"user_b\",\"contains\":\"...\",\"mediaType\":\"IMAGE\"} (mediaType is optional; assert proof photos/files on the same peer message), {\"do\":\"expect_http_request\",\"urlContains\":\"/orders\",\"method\":\"POST\",\"bodyContains\":\"orderId\",\"count\":1} (assert the outgoing request; method, bodyContains and count are optional). A scenario with no expect / expect_var / expect_peer_message step asserts nothing. At most 15 steps; malformed or truncated steps make the scenario an invalid test, never a pass."New value: +"Ordered steps, alternating an ACTION with an ASSERTION about the reply to the action right before it. When accepted checks are supplied, use {\"do\":\"expect_check\",\"checkId\":\"<accepted id>\"} for their outcomes; the server selects the observation channel. Optional contains:[\"literal journey value\"] strengthens message/notification checks only. An expect_check about someone other than the acting person (an outcome for person \"other\") needs \"other\": \"<as_contact id>\", the id that person has in this scenario (\"$SELF\" is the first person, once another one acts); e.g. {\"do\":\"start\",\"command\":\"/start\"}, {\"do\":\"tap\",\"button\":\"Apply\"}, {\"do\":\"as_contact\",\"id\":\"admin\"}, {\"do\":\"tap\",\"button\":\"Approve\"}, {\"do\":\"expect_check\",\"checkId\":\"c2\",\"other\":\"$SELF\"}. Each step is one object keyed by `do`: {\"do\":\"start\",\"command\":\"/start\"}, {\"do\":\"send\",\"text\":\"...\"}, {\"do\":\"send\",\"text\":\"...\",\"aiChooses\":\"<exit name>\"} (the AI assistant block answering this message takes that exit: an intent or tool name as the block lists it, or \"no_match\" / \"frustration\" — no AI runs in tests, so without aiChooses the block only chats and no branch behind an exit runs), {\"do\":\"send_media\",\"kind\":\"photo\"|\"video\"|\"audio\"|\"document\",\"caption\":\"...\"}, {\"do\":\"tap\",\"button\":\"<substring of the label>\"}, {\"do\":\"pay\"}, {\"do\":\"abandon_payment\",\"text\":\"...\"}, {\"do\":\"resume_delays\"}, {\"do\":\"simulate_event\",\"event\":\"member_join\"|\"join_request\"}, {\"do\":\"expect\",\"anyOf\":[\"verbatim fragment the bot really sends\"],\"not\":[\"text that must NOT appear\"]} (expect needs anyOf: a list of at least one verbatim fragment of what the bot sends after the action right before it; not (texts that must NOT appear) is optional and is checked only beside anyOf, because an expect with only not passes when the bot says nothing; a text in anyOf or not that the bot cannot send is removed: a {{…}} template, an internal id (no spaces, joined by _ . : or -), and, when the bot's texts are known, a button label or words it never writes; a removed text makes the scenario an invalid test), {\"do\":\"expect_var\",\"name\":\"<variable handle>\",\"contains\":\"...\",\"changed\":true}, {\"do\":\"seed_var\",\"name\":\"...\",\"value\":\"...\"} (seed_var value \"$SELF\" becomes the simulated user id, for a gate that compares the user id with a stored owner-id variable; a workspace-owner or tag gate is opened with assume.actors, never with seed_var), {\"do\":\"as_contact\",\"id\":\"user_b\"} (every later step acts as a SECOND person — for anything two people do together: one builds or sends, the other sees it; after the switch that person sees what the bot sent them earlier and can tap its buttons, and cannot tap a button on another person's screen), {\"do\":\"expect_peer_message\",\"to\":\"user_b\",\"contains\":\"...\",\"mediaType\":\"IMAGE\"} (mediaType is optional; assert proof photos/files on the same peer message), {\"do\":\"expect_http_request\",\"urlContains\":\"/orders\",\"method\":\"POST\",\"bodyContains\":\"orderId\",\"count\":1} (assert the outgoing request; method, bodyContains and count are optional). A scenario with no expect / expect_var / expect_peer_message step asserts nothing. At most 15 steps; malformed or truncated steps make the scenario an invalid test, never a pass."
  3. Changed1 schema field changed
    • changedInput schema / properties / scenarios / items / properties / steps / description
      Previous value: -"Ordered steps, alternating an ACTION with an ASSERTION about the reply to the action right before it. When accepted checks are supplied, use {\"do\":\"expect_check\",\"checkId\":\"<accepted id>\"} for their outcomes; the server selects the observation channel. Optional contains:[\"literal journey value\"] strengthens message/notification checks only. Each step is one object keyed by `do`: {\"do\":\"start\",\"command\":\"/start\"}, {\"do\":\"send\",\"text\":\"...\"}, {\"do\":\"send\",\"text\":\"...\",\"aiChooses\":\"<exit name>\"} (the AI assistant block answering this message takes that exit: an intent or tool name as the block lists it, or \"no_match\" / \"frustration\" — no AI runs in tests, so without aiChooses the block only chats and no branch behind an exit runs), {\"do\":\"send_media\",\"kind\":\"photo\"|\"video\"|\"audio\"|\"document\",\"caption\":\"...\"}, {\"do\":\"tap\",\"button\":\"<substring of the label>\"}, {\"do\":\"pay\"}, {\"do\":\"abandon_payment\",\"text\":\"...\"}, {\"do\":\"resume_delays\"}, {\"do\":\"simulate_event\",\"event\":\"member_join\"}, {\"do\":\"expect\",\"anyOf\":[\"verbatim fragment the bot really sends\"],\"not\":[\"text that must NOT appear\"]}, {\"do\":\"expect_var\",\"name\":\"<variable handle>\",\"contains\":\"...\",\"changed\":true}, {\"do\":\"seed_var\",\"name\":\"...\",\"value\":\"...\"} (seed_var value \"$SELF\" becomes the simulated user id, for a gate that compares the user id with a stored owner-id variable; a workspace-owner or tag gate is opened with assume.actors, never with seed_var), {\"do\":\"as_contact\",\"id\":\"user_b\"} (every later step acts as a SECOND person — for anything two people do together: one builds or sends, the other sees it; after the switch that person sees what the bot sent them earlier and can tap its buttons, and cannot tap a button on another person's screen), {\"do\":\"expect_peer_message\",\"to\":\"user_b\",\"contains\":\"...\",\"mediaType\":\"IMAGE\"} (mediaType is optional; assert proof photos/files on the same peer message), {\"do\":\"expect_http_request\",\"urlContains\":\"/orders\",\"method\":\"POST\",\"bodyContains\":\"orderId\",\"count\":1} (assert the outgoing request; method, bodyContains and count are optional). A scenario with no expect / expect_var / expect_peer_message step asserts nothing. At most 15 steps; malformed or truncated steps make the scenario an invalid test, never a pass."New value: +"Ordered steps, alternating an ACTION with an ASSERTION about the reply to the action right before it. When accepted checks are supplied, use {\"do\":\"expect_check\",\"checkId\":\"<accepted id>\"} for their outcomes; the server selects the observation channel. Optional contains:[\"literal journey value\"] strengthens message/notification checks only. Each step is one object keyed by `do`: {\"do\":\"start\",\"command\":\"/start\"}, {\"do\":\"send\",\"text\":\"...\"}, {\"do\":\"send\",\"text\":\"...\",\"aiChooses\":\"<exit name>\"} (the AI assistant block answering this message takes that exit: an intent or tool name as the block lists it, or \"no_match\" / \"frustration\" — no AI runs in tests, so without aiChooses the block only chats and no branch behind an exit runs), {\"do\":\"send_media\",\"kind\":\"photo\"|\"video\"|\"audio\"|\"document\",\"caption\":\"...\"}, {\"do\":\"tap\",\"button\":\"<substring of the label>\"}, {\"do\":\"pay\"}, {\"do\":\"abandon_payment\",\"text\":\"...\"}, {\"do\":\"resume_delays\"}, {\"do\":\"simulate_event\",\"event\":\"member_join\"|\"join_request\"}, {\"do\":\"expect\",\"anyOf\":[\"verbatim fragment the bot really sends\"],\"not\":[\"text that must NOT appear\"]}, {\"do\":\"expect_var\",\"name\":\"<variable handle>\",\"contains\":\"...\",\"changed\":true}, {\"do\":\"seed_var\",\"name\":\"...\",\"value\":\"...\"} (seed_var value \"$SELF\" becomes the simulated user id, for a gate that compares the user id with a stored owner-id variable; a workspace-owner or tag gate is opened with assume.actors, never with seed_var), {\"do\":\"as_contact\",\"id\":\"user_b\"} (every later step acts as a SECOND person — for anything two people do together: one builds or sends, the other sees it; after the switch that person sees what the bot sent them earlier and can tap its buttons, and cannot tap a button on another person's screen), {\"do\":\"expect_peer_message\",\"to\":\"user_b\",\"contains\":\"...\",\"mediaType\":\"IMAGE\"} (mediaType is optional; assert proof photos/files on the same peer message), {\"do\":\"expect_http_request\",\"urlContains\":\"/orders\",\"method\":\"POST\",\"bodyContains\":\"orderId\",\"count\":1} (assert the outgoing request; method, bodyContains and count are optional). A scenario with no expect / expect_var / expect_peer_message step asserts nothing. At most 15 steps; malformed or truncated steps make the scenario an invalid test, never a pass."
  4. Changed1 schema field changed
    • changedInput schema / properties / scenarios / items / properties / assume / properties / actors / description
      Previous value: -"actors: who the people of this scenario are when it starts, as a list of { id, platformId?, workspaceOwner?, tags? }. id is the as_contact id of the person, or \"$SELF\" for the default one. platformId is an exact numeric Telegram user id for bots that compare a literal id; omit it to derive a stable test id. workspaceOwner: true makes that person an owner of the workspace, so {{sysvar|isWorkspaceOwner}} is true for them and false for everyone else; tags are the tag NAMES they carry, as {{sysvar|tags}} lists them. A tag or an owner is never set with seed_var."New value: +"actors: who the people of this scenario are when it starts, as a list of { id, platformId?, workspaceOwner?, tags? }. id is the as_contact id of the person, or \"$SELF\" for the default one. platformId is an exact numeric Telegram user id for bots that compare a literal id; omit it to derive a stable test id. workspaceOwner: true makes that person the workspace's REAL owner: the simulator gives them the owner's own Telegram id (the one in the workspace facts), so {{sysvar|isWorkspaceOwner}} is true for them and false for everyone else, and a message for the owner reaches them; leave platformId out for that person. A send to the owner's id or @username is a send to the owner, never to \"another user\". When no owner has linked Telegram, nobody is the owner and the owner checks fail naming the Connect Telegram step: that is expected, not a bug to fix. tags are the tag NAMES they carry, as {{sysvar|tags}} lists them. A tag or an owner is never set with seed_var."
  5. Changed2 schema fields changed
    • changedInput schema / properties / scenarios / items / properties / assume / properties / aiOutput / description
      Previous value: -"The exact text EVERY AI step answers in this scenario. No AI runs in the simulator: unset, AI steps answer \"[AI output]\", so a condition comparing the AI-written variable takes its NO branch and a failure behind it is reported as inconclusive. Pin the value the branch under test compares against (e.g. \"hot\"). It does NOT steer an AI assistant block (AI_TOOL_ROUTER): its intent and tool branches cannot be reached in a test, pinned or not."New value: +"The exact text EVERY AI step answers in this scenario. No AI runs in the simulator: unset, AI steps answer \"[AI output]\", so a condition comparing the AI-written variable takes its NO branch and a failure behind it is reported as inconclusive. Pin the value the branch under test compares against (e.g. \"hot\"). It does NOT steer an AI assistant block (AI_TOOL_ROUTER): to take one of its exits, set aiChooses on the send step the block answers."
    • changedInput schema / properties / scenarios / items / properties / steps / description
      Previous value: -"Ordered steps, alternating an ACTION with an ASSERTION about the reply to the action right before it. When accepted checks are supplied, use {\"do\":\"expect_check\",\"checkId\":\"<accepted id>\"} for their outcomes; the server selects the observation channel. Optional contains:[\"literal journey value\"] strengthens message/notification checks only. Each step is one object keyed by `do`: {\"do\":\"start\",\"command\":\"/start\"}, {\"do\":\"send\",\"text\":\"...\"}, {\"do\":\"send_media\",\"kind\":\"photo\"|\"video\"|\"audio\"|\"document\",\"caption\":\"...\"}, {\"do\":\"tap\",\"button\":\"<substring of the label>\"}, {\"do\":\"pay\"}, {\"do\":\"abandon_payment\",\"text\":\"...\"}, {\"do\":\"resume_delays\"}, {\"do\":\"simulate_event\",\"event\":\"member_join\"}, {\"do\":\"expect\",\"anyOf\":[\"verbatim fragment the bot really sends\"],\"not\":[\"text that must NOT appear\"]}, {\"do\":\"expect_var\",\"name\":\"<variable handle>\",\"contains\":\"...\",\"changed\":true}, {\"do\":\"seed_var\",\"name\":\"...\",\"value\":\"...\"} (seed_var value \"$SELF\" becomes the simulated user id, for a gate that compares the user id with a stored owner-id variable; a workspace-owner or tag gate is opened with assume.actors, never with seed_var), {\"do\":\"as_contact\",\"id\":\"user_b\"} (every later step acts as a SECOND person — for anything two people do together: one builds or sends, the other sees it; after the switch that person sees what the bot sent them earlier and can tap its buttons, and cannot tap a button on another person's screen), {\"do\":\"expect_peer_message\",\"to\":\"user_b\",\"contains\":\"...\",\"mediaType\":\"IMAGE\"} (mediaType is optional; assert proof photos/files on the same peer message), {\"do\":\"expect_http_request\",\"urlContains\":\"/orders\",\"method\":\"POST\",\"bodyContains\":\"orderId\",\"count\":1} (assert the outgoing request; method, bodyContains and count are optional). A scenario with no expect / expect_var / expect_peer_message step asserts nothing. At most 15 steps; malformed or truncated steps make the scenario an invalid test, never a pass."New value: +"Ordered steps, alternating an ACTION with an ASSERTION about the reply to the action right before it. When accepted checks are supplied, use {\"do\":\"expect_check\",\"checkId\":\"<accepted id>\"} for their outcomes; the server selects the observation channel. Optional contains:[\"literal journey value\"] strengthens message/notification checks only. Each step is one object keyed by `do`: {\"do\":\"start\",\"command\":\"/start\"}, {\"do\":\"send\",\"text\":\"...\"}, {\"do\":\"send\",\"text\":\"...\",\"aiChooses\":\"<exit name>\"} (the AI assistant block answering this message takes that exit: an intent or tool name as the block lists it, or \"no_match\" / \"frustration\" — no AI runs in tests, so without aiChooses the block only chats and no branch behind an exit runs), {\"do\":\"send_media\",\"kind\":\"photo\"|\"video\"|\"audio\"|\"document\",\"caption\":\"...\"}, {\"do\":\"tap\",\"button\":\"<substring of the label>\"}, {\"do\":\"pay\"}, {\"do\":\"abandon_payment\",\"text\":\"...\"}, {\"do\":\"resume_delays\"}, {\"do\":\"simulate_event\",\"event\":\"member_join\"}, {\"do\":\"expect\",\"anyOf\":[\"verbatim fragment the bot really sends\"],\"not\":[\"text that must NOT appear\"]}, {\"do\":\"expect_var\",\"name\":\"<variable handle>\",\"contains\":\"...\",\"changed\":true}, {\"do\":\"seed_var\",\"name\":\"...\",\"value\":\"...\"} (seed_var value \"$SELF\" becomes the simulated user id, for a gate that compares the user id with a stored owner-id variable; a workspace-owner or tag gate is opened with assume.actors, never with seed_var), {\"do\":\"as_contact\",\"id\":\"user_b\"} (every later step acts as a SECOND person — for anything two people do together: one builds or sends, the other sees it; after the switch that person sees what the bot sent them earlier and can tap its buttons, and cannot tap a button on another person's screen), {\"do\":\"expect_peer_message\",\"to\":\"user_b\",\"contains\":\"...\",\"mediaType\":\"IMAGE\"} (mediaType is optional; assert proof photos/files on the same peer message), {\"do\":\"expect_http_request\",\"urlContains\":\"/orders\",\"method\":\"POST\",\"bodyContains\":\"orderId\",\"count\":1} (assert the outgoing request; method, bodyContains and count are optional). A scenario with no expect / expect_var / expect_peer_message step asserts nothing. At most 15 steps; malformed or truncated steps make the scenario an invalid test, never a pass."
  6. Changed1 schema field changed
    • changedInput schema / properties / flowIds / description
      Previous value: -"Required. Ids of the ALREADY-APPLIED flows to test — normally the flows apply_actions just created or changed, taken from its idRemap. Flows they link into are compiled too but are not crawled as entries."New value: +"Required. Ids of the ALREADY-APPLIED flows to test — normally the flows apply_actions just created or changed, taken from its idRemap. Flows they link into are compiled too but are not crawled as entries. Flows that link INTO them compile too and are crawled only for a requested flow with no trigger of its own (a flow opened from a menu button is tested through the flow that opens it); a requested flow nobody can start fails with a no_entry check."
  7. Changed5 schema fields changed
    • changedInput schema / properties / scenarios / items / properties / assume / properties / actors / description
      Previous value: -"actors: who the people of this scenario are when it starts, as a list of { id, workspaceOwner?, tags? }. id is the as_contact id of the person, or \"$SELF\" for the default one. workspaceOwner: true makes that person an owner of the workspace, so {{sysvar|isWorkspaceOwner}} is true for them and false for everyone else; tags are the tag NAMES they carry, as {{sysvar|tags}} lists them. Without it nobody is the owner and nobody has a tag, so an owner-only or tag-only step can only refuse. A tag or an owner is never set with seed_var."New value: +"actors: who the people of this scenario are when it starts, as a list of { id, platformId?, workspaceOwner?, tags? }. id is the as_contact id of the person, or \"$SELF\" for the default one. platformId is an exact numeric Telegram user id for bots that compare a literal id; omit it to derive a stable test id. workspaceOwner: true makes that person an owner of the workspace, so {{sysvar|isWorkspaceOwner}} is true for them and false for everyone else; tags are the tag NAMES they carry, as {{sysvar|tags}} lists them. A tag or an owner is never set with seed_var."
    • addedInput schema / properties / scenarios / items / properties / assume / properties / actors / items / properties / platformId
      Added value: +{
      +  "description": "Exact numeric Telegram user id, for example the id a legacy admin gate compares against.",
      +  "pattern": "^[1-9]\\d{0,15}$",
      +  "type": "string"
      +}
    • addedInput schema / properties / scenarios / items / properties / assume / properties / actors / items / properties / telegramMembership
      Added value: +{
      +  "description": "What getChatMember reports for this actor; overrides the scenario-wide membership.",
      +  "enum": [
      +    "member",
      +    "not_member"
      +  ],
      +  "type": "string"
      +}
    • addedInput schema / properties / scenarios / items / properties / assume / properties / httpResponses
      Added value: +{
      +  "description": "Mock outside HTTP calls for this scenario. These are local responses; no request is sent to the service.",
      +  "items": {
      +    "additionalProperties": false,
      +    "properties": {
      +      "body": {
      +        "description": "JSON body returned to the bot code."
      +      },
      +      "method": {
      +        "description": "Optional uppercase HTTP method to match.",
      +        "type": "string"
      +      },
      +      "networkError": {
      +        "description": "Reject the request with this transport error instead of returning an HTTP response.",
      +        "type": "string"
      +      },
      +      "requestBodyContains": {
      +        "description": "Optional literal substring required in the outgoing request body.",
      +        "minLength": 1,
      +        "type": "string"
      +      },
      +      "status": {
      +        "maximum": 599,
      +        "minimum": 100,
      +        "type": "integer"
      +      },
      +      "times": {
      +        "description": "Answer at most this many matching calls, then use the next fixture.",
      +        "exclusiveMinimum": 0,
      +        "type": "integer"
      +      },
      +      "urlPattern": {
      +        "description": "Substring of the outgoing URL to match.",
      +        "minLength": 1,
      +        "type": "string"
      +      }
      +    },
      +    "required": [
      +      "urlPattern"
      +    ],
      +    "type": "object"
      +  },
      +  "maxItems": 12,
      +  "type": "array"
      +}
    • changedInput schema / properties / scenarios / items / properties / steps / description
      Previous value: -"Ordered steps, alternating an ACTION with an ASSERTION about the reply to the action right before it. When accepted checks are supplied, use {\"do\":\"expect_check\",\"checkId\":\"<accepted id>\"} for their outcomes; the server selects the observation channel. Optional contains:[\"literal journey value\"] strengthens message/notification checks only. Each step is one object keyed by `do`: {\"do\":\"start\",\"command\":\"/start\"}, {\"do\":\"send\",\"text\":\"...\"}, {\"do\":\"send_media\",\"kind\":\"photo\"|\"video\"|\"audio\"|\"document\",\"caption\":\"...\"}, {\"do\":\"tap\",\"button\":\"<substring of the label>\"}, {\"do\":\"pay\"}, {\"do\":\"abandon_payment\",\"text\":\"...\"}, {\"do\":\"resume_delays\"}, {\"do\":\"simulate_event\",\"event\":\"member_join\"}, {\"do\":\"expect\",\"anyOf\":[\"verbatim fragment the bot really sends\"],\"not\":[\"text that must NOT appear\"]}, {\"do\":\"expect_var\",\"name\":\"<variable handle>\",\"contains\":\"...\",\"changed\":true}, {\"do\":\"seed_var\",\"name\":\"...\",\"value\":\"...\"} (seed_var value \"$SELF\" becomes the simulated user id, for a gate that compares the user id with a stored owner-id variable; a workspace-owner or tag gate is opened with assume.actors, never with seed_var), {\"do\":\"as_contact\",\"id\":\"user_b\"} (every later step acts as a SECOND person — for anything two people do together: one builds or sends, the other sees it; after the switch that person sees what the bot sent them earlier and can tap its buttons, and cannot tap a button on another person's screen), {\"do\":\"expect_peer_message\",\"to\":\"user_b\",\"contains\":\"...\"}. A scenario with no expect / expect_var / expect_peer_message step asserts nothing. At most 15 steps; malformed steps are dropped rather than failing the run."New value: +"Ordered steps, alternating an ACTION with an ASSERTION about the reply to the action right before it. When accepted checks are supplied, use {\"do\":\"expect_check\",\"checkId\":\"<accepted id>\"} for their outcomes; the server selects the observation channel. Optional contains:[\"literal journey value\"] strengthens message/notification checks only. Each step is one object keyed by `do`: {\"do\":\"start\",\"command\":\"/start\"}, {\"do\":\"send\",\"text\":\"...\"}, {\"do\":\"send_media\",\"kind\":\"photo\"|\"video\"|\"audio\"|\"document\",\"caption\":\"...\"}, {\"do\":\"tap\",\"button\":\"<substring of the label>\"}, {\"do\":\"pay\"}, {\"do\":\"abandon_payment\",\"text\":\"...\"}, {\"do\":\"resume_delays\"}, {\"do\":\"simulate_event\",\"event\":\"member_join\"}, {\"do\":\"expect\",\"anyOf\":[\"verbatim fragment the bot really sends\"],\"not\":[\"text that must NOT appear\"]}, {\"do\":\"expect_var\",\"name\":\"<variable handle>\",\"contains\":\"...\",\"changed\":true}, {\"do\":\"seed_var\",\"name\":\"...\",\"value\":\"...\"} (seed_var value \"$SELF\" becomes the simulated user id, for a gate that compares the user id with a stored owner-id variable; a workspace-owner or tag gate is opened with assume.actors, never with seed_var), {\"do\":\"as_contact\",\"id\":\"user_b\"} (every later step acts as a SECOND person — for anything two people do together: one builds or sends, the other sees it; after the switch that person sees what the bot sent them earlier and can tap its buttons, and cannot tap a button on another person's screen), {\"do\":\"expect_peer_message\",\"to\":\"user_b\",\"contains\":\"...\",\"mediaType\":\"IMAGE\"} (mediaType is optional; assert proof photos/files on the same peer message), {\"do\":\"expect_http_request\",\"urlContains\":\"/orders\",\"method\":\"POST\",\"bodyContains\":\"orderId\",\"count\":1} (assert the outgoing request; method, bodyContains and count are optional). A scenario with no expect / expect_var / expect_peer_message step asserts nothing. At most 15 steps; malformed or truncated steps make the scenario an invalid test, never a pass."
  8. Changed2 schema fields changed
    • addedInput schema / properties / scenarios / items / properties / assume / properties / actors
      Added value: +{
      +  "description": "actors: who the people of this scenario are when it starts, as a list of { id, workspaceOwner?, tags? }. id is the as_contact id of the person, or \"$SELF\" for the default one. workspaceOwner: true makes that person an owner of the workspace, so {{sysvar|isWorkspaceOwner}} is true for them and false for everyone else; tags are the tag NAMES they carry, as {{sysvar|tags}} lists them. Without it nobody is the owner and nobody has a tag, so an owner-only or tag-only step can only refuse. A tag or an owner is never set with seed_var.",
      +  "items": {
      +    "additionalProperties": false,
      +    "properties": {
      +      "id": {
      +        "description": "The as_contact id of the person, or \"$SELF\" for the default one.",
      +        "type": "string"
      +      },
      +      "tags": {
      +        "description": "Tag NAMES this person carries when the scenario starts.",
      +        "items": {
      +          "type": "string"
      +        },
      +        "type": "array"
      +      },
      +      "workspaceOwner": {
      +        "description": "true: this person is an owner of the workspace ({{sysvar|isWorkspaceOwner}} is true for them only).",
      +        "type": "boolean"
      +      }
      +    },
      +    "required": [
      +      "id"
      +    ],
      +    "type": "object"
      +  },
      +  "type": "array"
      +}
    • changedInput schema / properties / scenarios / items / properties / steps / description
      Previous value: -"Ordered steps, alternating an ACTION with an ASSERTION about the reply to the action right before it. When accepted checks are supplied, use {\"do\":\"expect_check\",\"checkId\":\"<accepted id>\"} for their outcomes; the server selects the observation channel. Optional contains:[\"literal journey value\"] strengthens message/notification checks only. Each step is one object keyed by `do`: {\"do\":\"start\",\"command\":\"/start\"}, {\"do\":\"send\",\"text\":\"...\"}, {\"do\":\"send_media\",\"kind\":\"photo\"|\"video\"|\"audio\"|\"document\",\"caption\":\"...\"}, {\"do\":\"tap\",\"button\":\"<substring of the label>\"}, {\"do\":\"pay\"}, {\"do\":\"abandon_payment\",\"text\":\"...\"}, {\"do\":\"resume_delays\"}, {\"do\":\"simulate_event\",\"event\":\"member_join\"}, {\"do\":\"expect\",\"anyOf\":[\"verbatim fragment the bot really sends\"],\"not\":[\"text that must NOT appear\"]}, {\"do\":\"expect_var\",\"name\":\"<variable handle>\",\"contains\":\"...\",\"changed\":true}, {\"do\":\"seed_var\",\"name\":\"...\",\"value\":\"...\"} (seed_var value \"$SELF\" becomes the simulated user id, for owner/admin gates), {\"do\":\"as_contact\",\"id\":\"user_b\"} (every later step acts as a SECOND person — for anything two people do together: one builds or sends, the other sees it; after the switch that person sees what the bot sent them earlier and can tap its buttons, and cannot tap a button on another person's screen), {\"do\":\"expect_peer_message\",\"to\":\"user_b\",\"contains\":\"...\"}. A scenario with no expect / expect_var / expect_peer_message step asserts nothing. At most 15 steps; malformed steps are dropped rather than failing the run."New value: +"Ordered steps, alternating an ACTION with an ASSERTION about the reply to the action right before it. When accepted checks are supplied, use {\"do\":\"expect_check\",\"checkId\":\"<accepted id>\"} for their outcomes; the server selects the observation channel. Optional contains:[\"literal journey value\"] strengthens message/notification checks only. Each step is one object keyed by `do`: {\"do\":\"start\",\"command\":\"/start\"}, {\"do\":\"send\",\"text\":\"...\"}, {\"do\":\"send_media\",\"kind\":\"photo\"|\"video\"|\"audio\"|\"document\",\"caption\":\"...\"}, {\"do\":\"tap\",\"button\":\"<substring of the label>\"}, {\"do\":\"pay\"}, {\"do\":\"abandon_payment\",\"text\":\"...\"}, {\"do\":\"resume_delays\"}, {\"do\":\"simulate_event\",\"event\":\"member_join\"}, {\"do\":\"expect\",\"anyOf\":[\"verbatim fragment the bot really sends\"],\"not\":[\"text that must NOT appear\"]}, {\"do\":\"expect_var\",\"name\":\"<variable handle>\",\"contains\":\"...\",\"changed\":true}, {\"do\":\"seed_var\",\"name\":\"...\",\"value\":\"...\"} (seed_var value \"$SELF\" becomes the simulated user id, for a gate that compares the user id with a stored owner-id variable; a workspace-owner or tag gate is opened with assume.actors, never with seed_var), {\"do\":\"as_contact\",\"id\":\"user_b\"} (every later step acts as a SECOND person — for anything two people do together: one builds or sends, the other sees it; after the switch that person sees what the bot sent them earlier and can tap its buttons, and cannot tap a button on another person's screen), {\"do\":\"expect_peer_message\",\"to\":\"user_b\",\"contains\":\"...\"}. A scenario with no expect / expect_var / expect_peer_message step asserts nothing. At most 15 steps; malformed steps are dropped rather than failing the run."
  9. Changed1 schema field changed
    • changedInput schema / properties / scenarios / items / properties / steps / description
      Previous value: -"Ordered steps, alternating an ACTION with an ASSERTION about the reply to the action right before it. Each step is one object keyed by `do`: {\"do\":\"start\",\"command\":\"/start\"}, {\"do\":\"send\",\"text\":\"...\"}, {\"do\":\"send_media\",\"kind\":\"photo\"|\"video\"|\"audio\"|\"document\",\"caption\":\"...\"}, {\"do\":\"tap\",\"button\":\"<substring of the label>\"}, {\"do\":\"pay\"}, {\"do\":\"abandon_payment\",\"text\":\"...\"}, {\"do\":\"resume_delays\"}, {\"do\":\"simulate_event\",\"event\":\"member_join\"}, {\"do\":\"expect\",\"anyOf\":[\"verbatim fragment the bot really sends\"],\"not\":[\"text that must NOT appear\"]}, {\"do\":\"expect_var\",\"name\":\"<variable handle>\",\"contains\":\"...\",\"changed\":true}, {\"do\":\"seed_var\",\"name\":\"...\",\"value\":\"...\"} (seed_var value \"$SELF\" becomes the simulated user id, for owner/admin gates), {\"do\":\"as_contact\",\"id\":\"user_b\"} (every later step acts as a SECOND person — for anything two people do together: one builds or sends, the other sees it; after the switch that person sees what the bot sent them earlier and can tap its buttons, and cannot tap a button on another person's screen), {\"do\":\"expect_peer_message\",\"to\":\"user_b\",\"contains\":\"...\"}. A scenario with no expect / expect_var / expect_peer_message step asserts nothing. At most 15 steps; malformed steps are dropped rather than failing the run."New value: +"Ordered steps, alternating an ACTION with an ASSERTION about the reply to the action right before it. When accepted checks are supplied, use {\"do\":\"expect_check\",\"checkId\":\"<accepted id>\"} for their outcomes; the server selects the observation channel. Optional contains:[\"literal journey value\"] strengthens message/notification checks only. Each step is one object keyed by `do`: {\"do\":\"start\",\"command\":\"/start\"}, {\"do\":\"send\",\"text\":\"...\"}, {\"do\":\"send_media\",\"kind\":\"photo\"|\"video\"|\"audio\"|\"document\",\"caption\":\"...\"}, {\"do\":\"tap\",\"button\":\"<substring of the label>\"}, {\"do\":\"pay\"}, {\"do\":\"abandon_payment\",\"text\":\"...\"}, {\"do\":\"resume_delays\"}, {\"do\":\"simulate_event\",\"event\":\"member_join\"}, {\"do\":\"expect\",\"anyOf\":[\"verbatim fragment the bot really sends\"],\"not\":[\"text that must NOT appear\"]}, {\"do\":\"expect_var\",\"name\":\"<variable handle>\",\"contains\":\"...\",\"changed\":true}, {\"do\":\"seed_var\",\"name\":\"...\",\"value\":\"...\"} (seed_var value \"$SELF\" becomes the simulated user id, for owner/admin gates), {\"do\":\"as_contact\",\"id\":\"user_b\"} (every later step acts as a SECOND person — for anything two people do together: one builds or sends, the other sees it; after the switch that person sees what the bot sent them earlier and can tap its buttons, and cannot tap a button on another person's screen), {\"do\":\"expect_peer_message\",\"to\":\"user_b\",\"contains\":\"...\"}. A scenario with no expect / expect_var / expect_peer_message step asserts nothing. At most 15 steps; malformed steps are dropped rather than failing the run."
  10. Changed1 schema field changed
    • changedInput schema / properties / scenarios / items / properties / steps / description
      Previous value: -"Ordered steps, alternating an ACTION with an ASSERTION about the reply to the action right before it. Each step is one object keyed by `do`: {\"do\":\"start\",\"command\":\"/start\"}, {\"do\":\"send\",\"text\":\"...\"}, {\"do\":\"send_media\",\"kind\":\"photo\"|\"video\"|\"audio\"|\"document\",\"caption\":\"...\"}, {\"do\":\"tap\",\"button\":\"<substring of the label>\"}, {\"do\":\"pay\"}, {\"do\":\"abandon_payment\",\"text\":\"...\"}, {\"do\":\"resume_delays\"}, {\"do\":\"simulate_event\",\"event\":\"member_join\"}, {\"do\":\"expect\",\"anyOf\":[\"verbatim fragment the bot really sends\"],\"not\":[\"text that must NOT appear\"]}, {\"do\":\"expect_var\",\"name\":\"<variable handle>\",\"contains\":\"...\",\"changed\":true}, {\"do\":\"seed_var\",\"name\":\"...\",\"value\":\"...\"} (seed_var value \"$SELF\" becomes the simulated user id, for owner/admin gates), {\"do\":\"as_contact\",\"id\":\"user_b\"} (every later step acts as a SECOND person — for anything two people do together: one builds or sends, the other sees it), {\"do\":\"expect_peer_message\",\"to\":\"user_b\",\"contains\":\"...\"}. A scenario with no expect / expect_var / expect_peer_message step asserts nothing. At most 15 steps; malformed steps are dropped rather than failing the run."New value: +"Ordered steps, alternating an ACTION with an ASSERTION about the reply to the action right before it. Each step is one object keyed by `do`: {\"do\":\"start\",\"command\":\"/start\"}, {\"do\":\"send\",\"text\":\"...\"}, {\"do\":\"send_media\",\"kind\":\"photo\"|\"video\"|\"audio\"|\"document\",\"caption\":\"...\"}, {\"do\":\"tap\",\"button\":\"<substring of the label>\"}, {\"do\":\"pay\"}, {\"do\":\"abandon_payment\",\"text\":\"...\"}, {\"do\":\"resume_delays\"}, {\"do\":\"simulate_event\",\"event\":\"member_join\"}, {\"do\":\"expect\",\"anyOf\":[\"verbatim fragment the bot really sends\"],\"not\":[\"text that must NOT appear\"]}, {\"do\":\"expect_var\",\"name\":\"<variable handle>\",\"contains\":\"...\",\"changed\":true}, {\"do\":\"seed_var\",\"name\":\"...\",\"value\":\"...\"} (seed_var value \"$SELF\" becomes the simulated user id, for owner/admin gates), {\"do\":\"as_contact\",\"id\":\"user_b\"} (every later step acts as a SECOND person — for anything two people do together: one builds or sends, the other sees it; after the switch that person sees what the bot sent them earlier and can tap its buttons, and cannot tap a button on another person's screen), {\"do\":\"expect_peer_message\",\"to\":\"user_b\",\"contains\":\"...\"}. A scenario with no expect / expect_var / expect_peer_message step asserts nothing. At most 15 steps; malformed steps are dropped rather than failing the run."
  11. Changed1 schema field changed
    • changedInput schema / properties / scenarios / items / properties / assume / properties / aiOutput / description
      Previous value: -"The exact text EVERY AI step answers in this scenario. No AI runs in the simulator: unset, AI steps answer \"[AI output]\", so a condition comparing the AI-written variable takes its NO branch and a failure behind it is reported as inconclusive. Pin the value the branch under test compares against (e.g. \"hot\")."New value: +"The exact text EVERY AI step answers in this scenario. No AI runs in the simulator: unset, AI steps answer \"[AI output]\", so a condition comparing the AI-written variable takes its NO branch and a failure behind it is reported as inconclusive. Pin the value the branch under test compares against (e.g. \"hot\"). It does NOT steer an AI assistant block (AI_TOOL_ROUTER): its intent and tool branches cannot be reached in a test, pinned or not."
  12. Changed1 schema field changed
    • changedInput schema / properties / scenarios / items / properties / steps / description
      Previous value: -"Ordered steps, alternating an ACTION with an ASSERTION about the reply to the action right before it. Each step is one object keyed by `do`: {\"do\":\"start\",\"command\":\"/start\"}, {\"do\":\"send\",\"text\":\"...\"}, {\"do\":\"send_media\",\"kind\":\"photo\"|\"video\"|\"audio\"|\"document\",\"caption\":\"...\"}, {\"do\":\"tap\",\"button\":\"<substring of the label>\"}, {\"do\":\"pay\"}, {\"do\":\"abandon_payment\",\"text\":\"...\"}, {\"do\":\"resume_delays\"}, {\"do\":\"simulate_event\",\"event\":\"member_join\"}, {\"do\":\"expect\",\"anyOf\":[\"verbatim fragment the bot really sends\"],\"not\":[\"text that must NOT appear\"]}, {\"do\":\"expect_var\",\"name\":\"<variable handle>\",\"contains\":\"...\",\"changed\":true}, {\"do\":\"seed_var\",\"name\":\"...\",\"value\":\"...\"} (seed_var value \"$SELF\" becomes the simulated user id, for owner/admin gates). At most 15 steps; malformed steps are dropped rather than failing the run."New value: +"Ordered steps, alternating an ACTION with an ASSERTION about the reply to the action right before it. Each step is one object keyed by `do`: {\"do\":\"start\",\"command\":\"/start\"}, {\"do\":\"send\",\"text\":\"...\"}, {\"do\":\"send_media\",\"kind\":\"photo\"|\"video\"|\"audio\"|\"document\",\"caption\":\"...\"}, {\"do\":\"tap\",\"button\":\"<substring of the label>\"}, {\"do\":\"pay\"}, {\"do\":\"abandon_payment\",\"text\":\"...\"}, {\"do\":\"resume_delays\"}, {\"do\":\"simulate_event\",\"event\":\"member_join\"}, {\"do\":\"expect\",\"anyOf\":[\"verbatim fragment the bot really sends\"],\"not\":[\"text that must NOT appear\"]}, {\"do\":\"expect_var\",\"name\":\"<variable handle>\",\"contains\":\"...\",\"changed\":true}, {\"do\":\"seed_var\",\"name\":\"...\",\"value\":\"...\"} (seed_var value \"$SELF\" becomes the simulated user id, for owner/admin gates), {\"do\":\"as_contact\",\"id\":\"user_b\"} (every later step acts as a SECOND person — for anything two people do together: one builds or sends, the other sees it), {\"do\":\"expect_peer_message\",\"to\":\"user_b\",\"contains\":\"...\"}. A scenario with no expect / expect_var / expect_peer_message step asserts nothing. At most 15 steps; malformed steps are dropped rather than failing the run."
  13. Changed1 schema field changed
    • addedInput schema / properties / scenarios / items / properties / assume
      Added value: +{
      +  "additionalProperties": false,
      +  "description": "Simulator assumptions this journey runs under.",
      +  "properties": {
      +    "aiOutput": {
      +      "description": "The exact text EVERY AI step answers in this scenario. No AI runs in the simulator: unset, AI steps answer \"[AI output]\", so a condition comparing the AI-written variable takes its NO branch and a failure behind it is reported as inconclusive. Pin the value the branch under test compares against (e.g. \"hot\").",
      +      "type": "string"
      +    },
      +    "telegramMembership": {
      +      "description": "What a Telegram membership gate reports. Default \"member\".",
      +      "enum": [
      +        "member",
      +        "not_member"
      +      ],
      +      "type": "string"
      +    }
      +  },
      +  "type": "object"
      +}
  14. Added

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description adds substantial behavioural detail beyond the readOnly/destructive annotations: it says nothing is mutated or sent to real users, explains what the smoke layer does, lists the report fields, clarifies how harness gaps and 'none' coverage should be interpreted, and describes how unanswered outside GET calls are handled from recorded responses.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is dense and front-loaded: purpose, usage, smoke behavior, scenarios, and return shape arrive in a logical order. It is longer than ideal, but nearly every sentence carries operational meaning for a complex simulator tool.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity, lack of an output schema, and rich annotations, the description is complete enough: it explains the return keys, the distinction between scenario failures and harness gaps, coverage 'none', and the mocked/recorded outside-call behavior. An agent has enough context to call it correctly and interpret results.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and the schema already documents parameters in depth, so the baseline is 3. The description still adds high-level selection semantics: flowIds are the already-applied flows normally from idRemap, scenarios are optional journeys capped at 6, and scenarios are the only way to assert exact texts or stored values.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: it runs deterministic behavioural tests against already-applied flows by compiling them to an AST and simulating a user. It clearly distinguishes the tool from apply_actions and validate_actions, and an agent can tell it is a verification step rather than a mutation step.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It explicitly says to call after apply_actions to verify a build, then patch and re-run. It also explains when to use the smoke layer alone versus when to pass scenarios, and notes that scenarios are the only way to assert exact texts or stored values.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.