Skip to main content
Glama

run_test_group

Execute a predefined group of tests for WordPress/WooCommerce plugins using configuration from qit.json to validate functionality and security.

Instructions

Run a group of tests defined in qit.json configuration file.

⚠️ QIT CLI not detected. QIT CLI not found. Please install it using one of these methods:

  1. Via Composer (recommended): composer require woocommerce/qit-cli --dev

  2. Set QIT_CLI_PATH environment variable: export QIT_CLI_PATH=/path/to/qit

  3. Ensure 'qit' is available in your system PATH

For more information, visit: https://github.com/woocommerce/qit-cli

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
groupYesName of the test group defined in qit.json
configNoPath to qit.json configuration file
waitNoWait for all tests to complete before returning
jsonNoReturn output in JSON format

Implementation Reference

  • The async handler function for the 'run_test_group' tool. It constructs CLI arguments for the 'run:group' command using the provided group name and optional flags (config, wait, json), sets an appropriate timeout, and executes the command via executeAndFormat.
    handler: async (args: {
      group: string;
      config?: string;
      wait?: boolean;
      json?: boolean;
    }) => {
      const cmdArgs = buildArgs("run:group", [args.group], {
        config: args.config,
        wait: args.wait,
        json: args.json,
      });
    
      const timeout = args.wait ? 1800000 : 300000;
    
      return executeAndFormat(cmdArgs, { timeout });
    },
  • Zod input schema defining parameters for the run_test_group tool: required 'group' string, optional 'config', 'wait', and 'json'.
    inputSchema: z.object({
      group: z.string().describe("Name of the test group defined in qit.json"),
      config: z
        .string()
        .optional()
        .describe("Path to qit.json configuration file"),
      wait: z
        .boolean()
        .optional()
        .describe("Wait for all tests to complete before returning"),
      json: z
        .boolean()
        .optional()
        .describe("Return output in JSON format"),
    }),
  • The complete definition object for the 'run_test_group' tool, including name, description, inputSchema, and handler, exported as part of testExecutionTools.
    run_test_group: {
      name: "run_test_group",
      description:
        "Run a group of tests defined in qit.json configuration file.",
      inputSchema: z.object({
        group: z.string().describe("Name of the test group defined in qit.json"),
        config: z
          .string()
          .optional()
          .describe("Path to qit.json configuration file"),
        wait: z
          .boolean()
          .optional()
          .describe("Wait for all tests to complete before returning"),
        json: z
          .boolean()
          .optional()
          .describe("Return output in JSON format"),
      }),
      handler: async (args: {
        group: string;
        config?: string;
        wait?: boolean;
        json?: boolean;
      }) => {
        const cmdArgs = buildArgs("run:group", [args.group], {
          config: args.config,
          wait: args.wait,
          json: args.json,
        });
    
        const timeout = args.wait ? 1800000 : 300000;
    
        return executeAndFormat(cmdArgs, { timeout });
      },
    },
  • Registration of testExecutionTools (containing run_test_group) by spreading into the allTools export used by the MCP server.
    export const allTools = {
      ...authTools,
      ...testExecutionTools,
  • src/server.ts:29-35 (registration)
    MCP server lists all tools from allTools, converting Zod schemas to JSON schema for the ListToolsRequestHandler.
    const tools = Object.entries(allTools).map(([_, tool]) => ({
      name: tool.name,
      description: cliInfo
        ? tool.description
        : `${tool.description}\n\n⚠️ QIT CLI not detected. ${getQitCliNotFoundError()}`,
      inputSchema: zodToJsonSchema(tool.inputSchema),
    }));
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries full burden for behavioral disclosure. It mentions the tool runs test groups from a configuration file, but fails to describe what 'running' entails (e.g., execution environment, side effects, timeouts, output format). The CLI installation troubleshooting is behavioral context but not about the tool's operation itself. Significant gaps remain for a tool that presumably executes tests.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness2/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is poorly structured and not front-loaded. The first sentence is useful, but the remaining 80% is installation troubleshooting that doesn't belong in a tool description. This wastes space and buries the actual purpose. While concise in word count, the structure fails to prioritize tool functionality.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a test execution tool with no annotations and no output schema, the description is incomplete. It mentions the configuration file but doesn't explain what happens during execution, what outputs to expect, error conditions, or how it relates to sibling tools. The CLI troubleshooting is contextually misplaced rather than helpful for understanding the tool's operation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already fully documents all 4 parameters. The description adds no parameter-specific information beyond implying the 'config' parameter references 'qit.json'. This meets the baseline of 3 when schema coverage is high, but adds minimal extra value.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose3/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states 'Run a group of tests defined in qit.json configuration file' which provides a clear verb ('run') and resource ('group of tests'), but it doesn't distinguish this tool from sibling tools like 'run_test' or explain how test groups differ from individual tests. The purpose is understandable but lacks sibling differentiation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines1/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives like 'run_test' or 'get_group_status'. Instead, it's dominated by installation instructions for missing CLI dependencies, which are irrelevant to usage decisions. There's no mention of prerequisites, when-not-to-use scenarios, or comparisons with sibling tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Install Server

Other Tools

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/woocommerce/qit-mcp'

If you have feedback or need assistance with the MCP directory API, please join our Discord server