Skip to main content
Glama

run_flow_autotest

Read-only

Runs deterministic behavioural tests against flows that are ALREADY applied (compiles them to an AST and simulates a user). Call after apply_actions to verify a build; read summary and the failed checks, patch with apply_actions, re-run. Mutates nothing. The smoke layer runs on its own with no input: it walks every entry, taps every button, answers every input step, and reports crashes, dead buttons, unresolved placeholders, and values the bot failed to store. Pass scenarios to also replay specific user journeys (at most 6) — that is the only way to assert exact texts or exact stored values. Returns { passed, smoke, scenarios, summary }. passed is false when any check or scenario failed; a summary saying coverage is "none" means nothing was testable, so a green verdict there proves nothing. Nothing is sent to real users and no state is written.

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
flowIdsYesRequired. Ids of the ALREADY-APPLIED flows to test — normally the flows apply_actions just created or changed, taken from its idRemap. Flows they link into are compiled too but are not crawled as entries.
scenariosNoOptional user journeys to replay on top of the smoke crawl. Omit to run the smoke layer alone.
applicationIdNoApplication (workspace) id. Optional: an application-scoped key (app_...) defaults to its own application, but a personal key (usr_...) has no default and omitting it fails with MCP_APPLICATION_REQUIRED. Call list_applications to get the id.

TDQS

A4.5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and openWorldHint=false, and the description reinforces this with 'Mutates nothing. Nothing is sent to real users and no state is written.' It goes further by explaining determinism, AST compilation, and the important caveat that a summary of 'none' means a green verdict proves nothing. This adds significant behavioral context beyond the annotations with no contradiction.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is dense but well-structured. It leads with the core purpose, then usage, then smoke-layer details, then scenarios, then return format and a critical caveat. Every sentence earns its place; no filler. It is appropriately sized for the tool's complexity without being bloated.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given there is no output schema, the description fully compensates by stating the return object { passed, smoke, scenarios, summary } and explaining the meaning of 'passed' and the 'none' caveat. It also covers error handling (malformed steps are dropped), linked-flow behavior, and scenario constraints (at most 6, 15 steps). Nothing an agent needs to call it correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description does not add new meaning to parameters beyond what the schema already provides—e.g., flowIds' idRemap source and applicationId's default behavior are already in the schema. The description's extra context (like 'normally the flows apply_actions just created or changed') is redundant with schema text. It adds no distinct value here.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource: 'Runs deterministic behavioural tests against flows'. It clearly distinguishes itself from siblings by stating it runs after apply_actions, and explicitly contrasts the smoke layer with scenario-based assertions, so an agent can tell it apart from apply_actions and validate_actions without opening their schemas.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly says 'Call after apply_actions to verify a build' and later 'patch with apply_actions, re-run', which gives a clear workflow. It also explains when to pass scenarios ('the only way to assert exact texts or exact stored values'), but does not explicitly name alternatives or state when NOT to use it. The guidance is strong but not exhaustive.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.

TDQS

A4.1/5.0
Disambiguation4/5

Each get_/list_ tool targets a distinct resource and the descriptions carefully separate similar pairs like get_application_context vs get_workspace_summary and get_broadcast_details vs get_broadcast_analytics. The main ambiguity is apply_actions vs validate_actions, since one runs the other's validation, but the descriptions make that relationship explicit.

Naming Consistency4/5

Names consistently follow verb_object snake_case and use familiar verbs like create, get, list, and update. Minor deviations such as read_messages instead of list_messages, apply_actions/validate_actions with plural nouns, and run_flow_autotest are still predictable and readable.

Tool Count2/5

33 tools exceeds the 25-tool threshold and makes the surface feel heavy, even though the domain is broad. Many of the read-only getters are individually useful but could be consolidated, such as merging module catalog/details or workspace/application context.

Completeness4/5

Core workflows are covered: application lifecycle, flow editing through apply_actions, deployment, contacts, broadcasts, modules, and message/event reads. Minor gaps exist, such as no dedicated delete for contacts, applications, or broadcasts, and no rollback for deployments, but agents can work around or avoid these.