Skip to main content
Glama

flowforge_run_test

Starts an asynchronous agent test run against an assistant, returning a queued run ID to poll for status and per-iteration results.

Instructions

Start an agent test run against an assistant.

Runs are asynchronous: this returns the run row (status queued). Poll flowforge_get_test_run with the returned run id for status and per-iteration results.

Args: params: test_id, assistant_id, optional branch_id, n_iterations, draft, response_format.

Returns: JSON/Markdown with the queued run (including its id).

Error Handling: Returns "Error: Resource not found..." if the test_id is wrong.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "params": "<params>"
    },
    "name": "flowforge_run_test"
  }
}

Input Schema

NameRequiredDescriptionDefault
paramsYes

Output Schema

NameRequiredDescriptionDefault
resultYes

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observedv0.2.0

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare non-readOnly, non-idempotent, openWorld, non-destructive, so the description earns credit for adding the genuinely new fact that execution is asynchronous and returns only a queued row. The polling instruction and the not-found error path are exactly the behavioral context an agent needs beyond the hints. It stops short of saying whether repeated calls create duplicate runs beyond what idempotentHint=false implies.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with purpose, then async contract, then args/returns/errors in a scannable structure. The Args list is somewhat redundant with the schema and could be trimmed, but nothing is padded or misleading.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-wrapper-parameter mutation with an output schema present, the description covers the decisive unknowns: async behavior, the queued status, the follow-up poll tool, and the error case. Remaining gaps (permissions, rate limits, what n_iterations aggregates into) are minor and partly covered by schema defaults.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The Args block enumerates test_id, assistant_id, branch_id, n_iterations, draft and response_format but adds no semantics beyond what the nested schema already documents for each field (the enumerated names match the schema properties one-for-one). It does usefully separate required from optional, which is modest added value over a 0% top-level coverage signal.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The first line states a specific verb and resource: "Start an agent test run against an assistant." It also implicitly distinguishes itself from the read-side sibling by naming flowforge_get_test_run as the polling endpoint, so an agent can tell run-start from run-status without opening schemas.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives a clear usage protocol: call this to start, then poll flowforge_get_test_run with the returned run id, and it warns that a bad test_id yields "Error: Resource not found...". It does not state when to prefer flowforge_manage_tests or flowforge_list_tests instead, so a small exclusion gap remains.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.