QIT MCP Server

Overview Schema Related Servers Score Discussions

run_test

Execute WordPress and WooCommerce plugin tests locally in Docker or via GitHub Actions. Run activation, security, e2e, performance, and compatibility tests with configurable PHP, WordPress, and WooCommerce versions.

Instructions

Run a QIT test on a plugin or theme. Local tests (activation, e2e, performance, woo-api, woo-e2e) run in Docker locally. Managed tests (api, compatibility, malware, phpcompatibility, phpstan, plugin-check, security, validation) are enqueued on GitHub Actions. Version flags - Full (php/wp/woo): activation, e2e, performance, compatibility, woo-api, woo-e2e; WP/Woo only: phpstan, phpcompatibility; None: security, malware, validation, api, plugin-check. Local-only flags (env_vars, volumes, additional_plugins/themes, object_cache, tunnel, ui, test_packages): activation, e2e, performance.

⚠️ QIT CLI not detected. QIT CLI not found. Please install it using one of these methods:

Via Composer (recommended): composer require woocommerce/qit-cli --dev
Set QIT_CLI_PATH environment variable: export QIT_CLI_PATH=/path/to/qit
Ensure 'qit' is available in your system PATH

For more information, visit: https://github.com/woocommerce/qit-cli

Input Schema

TableJSON Schema

Name	Required	Description
`type`	Yes	The type of test to run
`plugin`	Yes	Plugin slug, path to ZIP file, or path to plugin directory
`zip`	No	Custom source for the plugin/theme (local ZIP, local directory, or URL to a .zip file)
`php_version`	No	PHP version to use (e.g., '8.1', '8.2', '8.3')
`wp_version`	No	WordPress version to use (e.g., '6.4', '6.5', 'stable', 'rc')
`wc_version`	No	WooCommerce version to use (e.g., '8.5', '9.0', 'latest')
`additional_plugins`	No	Additional plugins to install (slug or slug:version format)
`additional_themes`	No	Additional themes to install (slug or slug:version format)
`test_packages`	No	Test packages to include
`volumes`	No	Docker volumes to mount (host:container format)
`env_vars`	No	Environment variables to set (key-value pairs)
`object_cache`	No	Enable Redis object cache
`tunnel`	No	Enable tunnelling method (cloudflare, ngrok)
`ui`	No	Run tests in Playwright UI mode (for e2e tests)
`config`	No	Path to qit.json configuration file
`wait`	No	Wait for managed test to complete before returning (default: true)
`async`	No	Enqueue test and return immediately without waiting
`json`	No	Return output in JSON format

Implementation Reference

src/tools/test-execution.ts:134-231 (handler)

The main execution logic for the 'run_test' tool. It determines if the test is managed or local, builds CLI flags based on test type support for versions and local-only flags, handles array parameters by pushing multiple flags, and calls executeAndFormat with appropriate timeout.

handler: async (args: {
  type: (typeof testTypes)[number];
  plugin: string;
  zip?: string;
  php_version?: string;
  wp_version?: string;
  wc_version?: string;
  additional_plugins?: string[];
  additional_themes?: string[];
  test_packages?: string[];
  volumes?: string[];
  env_vars?: Record<string, string>;
  object_cache?: boolean;
  tunnel?: (typeof tunnelMethods)[number];
  ui?: boolean;
  config?: string;
  wait?: boolean;
  async?: boolean;
  json?: boolean;
}) => {
  const command = `run:${args.type}`;
  const positional = [args.plugin];

  // Managed tests run on GitHub Actions - default to async mode
  // Local tests run in Docker - wait for completion
  const isManagedTest = managedTestTypes.has(args.type);
  const useAsync = args.async ?? (isManagedTest ? true : false);

  const isLocalOnlyTest = localOnlyFlagTests.has(args.type);

  const flags: Record<string, string | boolean | undefined> = {
    zip: args.zip,
    config: args.config,
    wait: args.wait,
    async: useAsync,
    json: args.json,
    // For managed tests, include the report URL in the output
    "print-report-url": isManagedTest ? true : undefined,
  };

  // Only add version flags if the test type supports them
  // CLI errors on unknown flags, so we must be selective
  // Use canonical flag names from the trunk QIT CLI
  if (versionFlagSupport.full.has(args.type)) {
    flags.php_version = args.php_version;
    flags.wordpress_version = args.wp_version;
    flags.woocommerce_version = args.wc_version;
  } else if (versionFlagSupport.wpWooOnly.has(args.type)) {
    // phpstan, phpcompatibility - no PHP version flag
    flags.wordpress_version = args.wp_version;
    flags.woocommerce_version = args.wc_version;
  }
  // For versionFlagSupport.none (security, malware, etc.) - don't add any version flags

  // Local-only flags: only for activation, e2e, performance
  if (isLocalOnlyTest) {
    flags.object_cache = args.object_cache;
    flags.tunnel = args.tunnel;
    flags.ui = args.ui;
  }

  const cmdArgs = buildArgs(command, positional, flags);

  // Add array-based flags (these need special handling)
  // These are also local-only flags
  if (isLocalOnlyTest) {
    if (args.additional_plugins?.length) {
      for (const plugin of args.additional_plugins) {
        cmdArgs.push("--plugin", plugin);
      }
    }
    if (args.additional_themes?.length) {
      for (const theme of args.additional_themes) {
        cmdArgs.push("--theme", theme);
      }
    }
    if (args.test_packages?.length) {
      for (const pkg of args.test_packages) {
        cmdArgs.push("--test-package", pkg);
      }
    }
    if (args.volumes?.length) {
      for (const volume of args.volumes) {
        cmdArgs.push("--volume", volume);
      }
    }
    if (args.env_vars) {
      for (const [key, value] of Object.entries(args.env_vars)) {
        cmdArgs.push("--env", `${key}=${value}`);
      }
    }
  }

  // Managed tests return quickly (async), local tests may take longer
  const timeout = useAsync ? 120000 : 1800000;

  return executeAndFormat(cmdArgs, { timeout });
},

src/tools/test-execution.ts:60-133 (schema)

Zod input schema for the 'run_test' tool, defining all parameters with descriptions, optionality, and types for test execution configuration.

inputSchema: z.object({
  type: z
    .enum(testTypes)
    .describe("The type of test to run"),
  plugin: z
    .string()
    .describe(
      "Plugin slug, path to ZIP file, or path to plugin directory"
    ),
  zip: z
    .string()
    .optional()
    .describe("Custom source for the plugin/theme (local ZIP, local directory, or URL to a .zip file)"),
  php_version: z
    .string()
    .optional()
    .describe("PHP version to use (e.g., '8.1', '8.2', '8.3')"),
  wp_version: z
    .string()
    .optional()
    .describe("WordPress version to use (e.g., '6.4', '6.5', 'stable', 'rc')"),
  wc_version: z
    .string()
    .optional()
    .describe("WooCommerce version to use (e.g., '8.5', '9.0', 'latest')"),
  additional_plugins: z
    .array(z.string())
    .optional()
    .describe("Additional plugins to install (slug or slug:version format)"),
  additional_themes: z
    .array(z.string())
    .optional()
    .describe("Additional themes to install (slug or slug:version format)"),
  test_packages: z
    .array(z.string())
    .optional()
    .describe("Test packages to include"),
  volumes: z
    .array(z.string())
    .optional()
    .describe("Docker volumes to mount (host:container format)"),
  env_vars: z
    .record(z.string())
    .optional()
    .describe("Environment variables to set (key-value pairs)"),
  object_cache: z
    .boolean()
    .optional()
    .describe("Enable Redis object cache"),
  tunnel: z
    .enum(tunnelMethods)
    .optional()
    .describe("Enable tunnelling method (cloudflare, ngrok)"),
  ui: z
    .boolean()
    .optional()
    .describe("Run tests in Playwright UI mode (for e2e tests)"),
  config: z
    .string()
    .optional()
    .describe("Path to qit.json configuration file"),
  wait: z
    .boolean()
    .optional()
    .describe("Wait for managed test to complete before returning (default: true)"),
  async: z
    .boolean()
    .optional()
    .describe("Enqueue test and return immediately without waiting"),
  json: z
    .boolean()
    .optional()
    .describe("Return output in JSON format"),
}),

src/tools/test-execution.ts:57-232 (registration)

The 'run_test' tool is defined and exported as part of testExecutionTools in this file, which is later spread into allTools.

run_test: {
  name: "run_test",
  description: `Run a QIT test on a plugin or theme. Local tests (${[...localTestTypes].join(", ")}) run in Docker locally. Managed tests (${[...managedTestTypes].join(", ")}) are enqueued on GitHub Actions. Version flags - Full (php/wp/woo): ${[...versionFlagSupport.full].join(", ")}; WP/Woo only: ${[...versionFlagSupport.wpWooOnly].join(", ")}; None: ${[...versionFlagSupport.none].join(", ")}. Local-only flags (env_vars, volumes, additional_plugins/themes, object_cache, tunnel, ui, test_packages): ${[...localOnlyFlagTests].join(", ")}.`,
  inputSchema: z.object({
    type: z
      .enum(testTypes)
      .describe("The type of test to run"),
    plugin: z
      .string()
      .describe(
        "Plugin slug, path to ZIP file, or path to plugin directory"
      ),
    zip: z
      .string()
      .optional()
      .describe("Custom source for the plugin/theme (local ZIP, local directory, or URL to a .zip file)"),
    php_version: z
      .string()
      .optional()
      .describe("PHP version to use (e.g., '8.1', '8.2', '8.3')"),
    wp_version: z
      .string()
      .optional()
      .describe("WordPress version to use (e.g., '6.4', '6.5', 'stable', 'rc')"),
    wc_version: z
      .string()
      .optional()
      .describe("WooCommerce version to use (e.g., '8.5', '9.0', 'latest')"),
    additional_plugins: z
      .array(z.string())
      .optional()
      .describe("Additional plugins to install (slug or slug:version format)"),
    additional_themes: z
      .array(z.string())
      .optional()
      .describe("Additional themes to install (slug or slug:version format)"),
    test_packages: z
      .array(z.string())
      .optional()
      .describe("Test packages to include"),
    volumes: z
      .array(z.string())
      .optional()
      .describe("Docker volumes to mount (host:container format)"),
    env_vars: z
      .record(z.string())
      .optional()
      .describe("Environment variables to set (key-value pairs)"),
    object_cache: z
      .boolean()
      .optional()
      .describe("Enable Redis object cache"),
    tunnel: z
      .enum(tunnelMethods)
      .optional()
      .describe("Enable tunnelling method (cloudflare, ngrok)"),
    ui: z
      .boolean()
      .optional()
      .describe("Run tests in Playwright UI mode (for e2e tests)"),
    config: z
      .string()
      .optional()
      .describe("Path to qit.json configuration file"),
    wait: z
      .boolean()
      .optional()
      .describe("Wait for managed test to complete before returning (default: true)"),
    async: z
      .boolean()
      .optional()
      .describe("Enqueue test and return immediately without waiting"),
    json: z
      .boolean()
      .optional()
      .describe("Return output in JSON format"),
  }),
  handler: async (args: {
    type: (typeof testTypes)[number];
    plugin: string;
    zip?: string;
    php_version?: string;
    wp_version?: string;
    wc_version?: string;
    additional_plugins?: string[];
    additional_themes?: string[];
    test_packages?: string[];
    volumes?: string[];
    env_vars?: Record<string, string>;
    object_cache?: boolean;
    tunnel?: (typeof tunnelMethods)[number];
    ui?: boolean;
    config?: string;
    wait?: boolean;
    async?: boolean;
    json?: boolean;
  }) => {
    const command = `run:${args.type}`;
    const positional = [args.plugin];

    // Managed tests run on GitHub Actions - default to async mode
    // Local tests run in Docker - wait for completion
    const isManagedTest = managedTestTypes.has(args.type);
    const useAsync = args.async ?? (isManagedTest ? true : false);

    const isLocalOnlyTest = localOnlyFlagTests.has(args.type);

    const flags: Record<string, string | boolean | undefined> = {
      zip: args.zip,
      config: args.config,
      wait: args.wait,
      async: useAsync,
      json: args.json,
      // For managed tests, include the report URL in the output
      "print-report-url": isManagedTest ? true : undefined,
    };

    // Only add version flags if the test type supports them
    // CLI errors on unknown flags, so we must be selective
    // Use canonical flag names from the trunk QIT CLI
    if (versionFlagSupport.full.has(args.type)) {
      flags.php_version = args.php_version;
      flags.wordpress_version = args.wp_version;
      flags.woocommerce_version = args.wc_version;
    } else if (versionFlagSupport.wpWooOnly.has(args.type)) {
      // phpstan, phpcompatibility - no PHP version flag
      flags.wordpress_version = args.wp_version;
      flags.woocommerce_version = args.wc_version;
    }
    // For versionFlagSupport.none (security, malware, etc.) - don't add any version flags

    // Local-only flags: only for activation, e2e, performance
    if (isLocalOnlyTest) {
      flags.object_cache = args.object_cache;
      flags.tunnel = args.tunnel;
      flags.ui = args.ui;
    }

    const cmdArgs = buildArgs(command, positional, flags);

    // Add array-based flags (these need special handling)
    // These are also local-only flags
    if (isLocalOnlyTest) {
      if (args.additional_plugins?.length) {
        for (const plugin of args.additional_plugins) {
          cmdArgs.push("--plugin", plugin);
        }
      }
      if (args.additional_themes?.length) {
        for (const theme of args.additional_themes) {
          cmdArgs.push("--theme", theme);
        }
      }
      if (args.test_packages?.length) {
        for (const pkg of args.test_packages) {
          cmdArgs.push("--test-package", pkg);
        }
      }
      if (args.volumes?.length) {
        for (const volume of args.volumes) {
          cmdArgs.push("--volume", volume);
        }
      }
      if (args.env_vars) {
        for (const [key, value] of Object.entries(args.env_vars)) {
          cmdArgs.push("--env", `${key}=${value}`);
        }
      }
    }

    // Managed tests return quickly (async), local tests may take longer
    const timeout = useAsync ? 120000 : 1800000;

    return executeAndFormat(cmdArgs, { timeout });
  },
},

src/server.ts:29-35 (registration)

In the MCP server, allTools (which includes run_test) is used to list tools for the ListToolsRequestHandler, converting Zod schemas to JSON schema.

const tools = Object.entries(allTools).map(([_, tool]) => ({
  name: tool.name,
  description: cliInfo
    ? tool.description
    : `${tool.description}\n\n⚠️ QIT CLI not detected. ${getQitCliNotFoundError()}`,
  inputSchema: zodToJsonSchema(tool.inputSchema),
}));

src/tools/index.ts:12-12 (registration)
testExecutionTools (containing run_test) is spread into the central allTools export, aggregating all tools for the MCP server.
```
...testExecutionTools,
```

Tool Definition Quality

C2.9/5.0

Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It describes execution methods (local Docker vs. GitHub Actions) and test type categorizations, but fails to cover critical behavioral aspects such as error handling, output format, runtime implications, or side effects. For a complex tool with 18 parameters, this is a significant gap in transparency.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness2/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is poorly structured and verbose. It mixes tool functionality with installation instructions and error messages ('⚠️ QIT CLI not detected...'), which are irrelevant to the tool's purpose. The first paragraph is dense with technical details but lacks clear organization, making it hard to parse efficiently. Sentences do not earn their place as they include extraneous content.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's high complexity (18 parameters, no annotations, no output schema), the description is incomplete. It omits explanations of return values, error conditions, and practical usage examples. While it covers test type distinctions, it fails to provide a holistic understanding needed for effective tool invocation, especially for a mutation tool with significant behavioral implications.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema description coverage is 100%, so the schema already documents all 18 parameters thoroughly. The description adds some context by linking certain parameters to specific test types (e.g., 'Local-only flags...: activation, e2e, performance'), but this is minimal value beyond the schema. The baseline score of 3 is appropriate as the schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: 'Run a QIT test on a plugin or theme.' It specifies the verb ('Run') and resource ('QIT test on a plugin or theme'), making the action clear. However, it doesn't explicitly differentiate this tool from sibling tools like 'run_test_group' or 'list_tests', which prevents a perfect score.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides implied usage guidance by detailing which test types are local vs. managed and which flags apply to which tests (e.g., 'Local tests... run in Docker locally. Managed tests... are enqueued on GitHub Actions.'). However, it lacks explicit when-to-use instructions compared to alternatives like 'run_test_group' or prerequisites for invoking the tool, leaving some ambiguity.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Install Server

Latest Blog Posts

Tool Definition Quality Score (TDQS)
By punkpeye on April 3, 2026.
mcp
The Hackers Who Tracked My Sleep Cycle
By punkpeye on March 26, 2026.
security
Open Source Has a Bot Problem
By punkpeye on March 19, 2026.
open source

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/woocommerce/qit-mcp'

If you have feedback or need assistance with the MCP directory API, please join our Discord server