Skip to main content
Glama

Run integration tests

run_integration_tests

Runs integration flows and writes JSON, Markdown, and JUnit reports for release-gate and PR evidence, returning failing-step summaries with diagnostic output.

Instructions

Run integration flows (default: all of tests/integration/) and write .qa-mcp/integration-report/ (JSON evidence for the Release Gate, Markdown for the PR, JUnit). Returns the summary and the failing steps with the closest message seen, query results or job output.

Tool call template

This tool declares no required parameters. Add optional parameters from the input schema as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {},
    "name": "run_integration_tests"
  }
}

Input Schema

NameRequiredDescriptionDefault
tagsNo
pathsNo
environmentNo

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observedv0.1.0

TDQS

B3.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations supplied, the description does real work: it discloses side effects (writes .qa-mcp/integration-report/ in JSON, Markdown, and JUnit), the purpose of each artifact (Release Gate evidence, PR comment), and the shape of the returned summary including failing steps and job output. It omits runtime expectations, environment side effects of running tests, and whether execution is blocking.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single dense sentence with the verb and default scope front-loaded, followed by artifact and return-value detail. Every clause carries information; the nested parenthetical makes it slightly harder to scan but nothing is wasted.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a test-runner with no output schema and no annotations, the description covers scope, file side effects, and return contents well. It falls short on the three undocumented parameters and on prerequisites/environment assumptions, which an agent needs before invoking it confidently.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0% across three parameters (tags, paths, environment), so the description must compensate. It only partially does: the default path hints at what 'paths' means, but 'tags' and 'environment' are never explained and their formats/accepted values are left entirely to inference.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ("Run integration flows") plus a concrete default scope ("all of tests/integration/"), which lets an agent distinguish this from config or report-reading siblings. It stops short of naming which sibling to use instead, so it is clear but not fully differentiated.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The parenthetical default implies 'call with no args to run everything,' which is an implicit usage hint. There is no explicit when-to-use/when-not guidance, no mention of prerequisites such as init_integration_config, and no routing toward validate_flow for single-flow checks.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.