Skip to main content
Glama
Test Automation MCP Server

Test Automation MCP Server

by hmodali2611

Test Automation MCP Server

CI M8ven Score

What it is

A Model Context Protocol server that gives AI assistants (Claude, GitHub Copilot, Cursor) safe, structured access to a test automation stack.

Related MCP server: TestPilot MCP

Why it exists

An assistant in chat usually cannot run a suite, read why tests failed, pull test data, or file a bug with evidence. This server gives it those actions through a small set of defined tools, with safety built in, so an investigation is repeatable and shareable instead of being a one-off chat answer.

Scope and limits

In scope:

  • Tools: list_test_suites, run_tests, get_test_results, get_failure_details, query_test_data, write_investigation_report, create_jira_bug (draft file when no Jira credentials are set).

  • Prompt triage_failures: run, inspect, re-run to rule out flakiness, classify, report, and file bugs only for real defects.

  • Works with any framework that writes JUnit XML.

Out of scope, per the original README:

  • A built-in Playwright or Appium driver.

  • Interactive debugging, breakpoints, or live device control.

Other limits from the original README:

  • This server runs suites you configure in the config file and reads the JUnit XML those suites produce. The examples show a Playwright web suite and an Appium plus TestNG iOS suite because both produce JUnit XML.

  • Debugging here means inspecting failure details: error message, stack trace, and captured output for a failing test.

  • create_jira_bug files a live Jira issue only when Jira credentials are set. Without them, it writes a draft file to reports/jira-drafts/ and reports that no issue was created.

  • An investigation is repeatable through the triage_failures prompt, and it is logged and shareable through write_investigation_report, which writes a markdown report to reports/investigations/, plus the JUnit report and the Jira issue or draft.

Features and how it works

From a chat, an assistant can run a suite, read why tests failed, pull test data, write a shareable investigation report, and file a Jira bug with the evidence.

"Run the checkout suite and file bugs for anything that's a real defect."

 assistant -> run_tests(sample-e2e)          6 tests: 3 passed, 2 failed, 1 skipped
           -> get_failure_details(promo)     Expected "$39.20" Received "$49.00" · API: PROMO_EXPIRED
           -> run_tests(filter: "order")     still times out -> flaky / environment, no bug
           -> query_test_data(promoCodes)    SPRING10 is marked expired in test data -> test data issue
           -> create_jira_bug(...)           only for confirmed product defects

Tool

What it does

list_test_suites

Lists configured suites, their commands and report paths

run_tests

Runs a suite (optionally filtered to matching tests) and returns pass/fail counts plus each failure

get_test_results

Summarizes the latest JUnit report without re-running

get_failure_details

Error message, stack trace and captured output for a failing test

query_test_data

Looks up accounts, products, promo codes, etc. from a JSON fixture file

write_investigation_report

Writes a shareable markdown report for one triaged test, with its classification, evidence, and action

create_jira_bug

Files a Jira Cloud bug; writes a draft file instead when no credentials are set

Prompt: triage_failures walks the assistant through run, inspect, re-run to rule out flakiness, classify (product defect, test issue, environment), write an investigation report for each failure, and file bugs only for real defects.

Works with any framework that writes JUnit XML: Playwright, TestNG, Maven Surefire, JUnit 5, pytest and most CI tools all produce JUnit XML, so one parser covers web and mobile suites. See examples/mcp-tests.playwright-and-testng.json for a Playwright web suite next to an Appium + TestNG iOS suite.

Design choices, from the original README:

  • The model never gets a raw shell. It can only run suites defined in the config file. Test filters are shell-quoted, so a test name cannot inject commands.

  • Small, model-friendly outputs. Tools return counts and one line per failure; details come only when asked. That keeps context windows small on large suites.

  • Errors are tool results, not crashes. A bad suite name or a missing report comes back as isError: true with a clear message the assistant can act on.

  • Safe by default. Jira writes are drafts until credentials are provided, and read-only tools are annotated readOnlyHint.

Prerequisites

An environment that can run npm install, npm test and npm run build, as in Quick start below. No other prerequisite is stated in the original README.

Quick start

Install from npm

Package test-automation-mcp-server, version 1.0.0. Use it without cloning, via npx, in Claude Desktop or Claude Code (claude_desktop_config.json / .mcp.json):

{
  "mcpServers": {
    "test-automation": {
      "command": "npx",
      "args": ["-y", "test-automation-mcp-server", "/absolute/path/to/mcp-tests.config.json"],
      "env": {
        "JIRA_BASE_URL": "https://your-company.atlassian.net",
        "JIRA_EMAIL": "you@company.com",
        "JIRA_API_TOKEN": "..."
      }
    }
  }
}

Or install it globally with npm install -g test-automation-mcp-server, then run test-automation-mcp /absolute/path/to/mcp-tests.config.json.

Leave the Jira variables out to keep create_jira_bug in draft mode (drafts go to reports/jira-drafts/).

Install from source

npm install
npm test          # 14 tests: JUnit parsing + every tool end to end over an in-memory MCP transport, plus Jira https guard
npm run build

Connect it to Claude Desktop or Claude Code (claude_desktop_config.json / .mcp.json):

{
  "mcpServers": {
    "test-automation": {
      "command": "node",
      "args": ["/absolute/path/to/test-automation-mcp-server/dist/index.js", "/absolute/path/to/mcp-tests.config.json"],
      "env": {
        "JIRA_BASE_URL": "https://your-company.atlassian.net",
        "JIRA_EMAIL": "you@company.com",
        "JIRA_API_TOKEN": "..."
      }
    }
  }
}

Leave the Jira variables out to keep create_jira_bug in draft mode (drafts go to reports/jira-drafts/).

Usage

Configure suites in a config file, for example:

{
  "suites": [
    {
      "name": "web-e2e",
      "command": "npx playwright test --reporter=junit",
      "cwd": "../my-web-app",
      "junitReport": "test-results/junit.xml",
      "grepFlag": "--grep"
    }
  ],
  "testDataFile": "data/test-data.json",
  "jira": { "projectKey": "QA", "issueType": "Bug" }
}

Then use the tools listed above, or the triage_failures prompt, to run, inspect, classify, report and, for confirmed product defects only, file a Jira bug.

Testing

npm test          # 14 tests: JUnit parsing + every tool end to end over an in-memory MCP transport, plus Jira https guard

Verified in CI with a real Android emulator (2026-11-01, GitHub Actions run 41373946277, 1m 57s): a real emulator booted, ApiDemos installed, and run_tests ran 2 tests in 7.3s with 1 passed and 1 failed. The failure was a real UiAutomator2 NoSuchElement error for testMissingElement, and an investigation report was saved. The passing test is identified only by elimination (2 tests total, only testMissingElement failed), not by a named pass line. Jira filing remains draft only without credentials, and Claude or Copilot use has not been tested in a live client session.

Project layout

The original README does not include a project layout table, so none is claimed here. The files named in the original README include the config examples, the JUnit report outputs, reports/investigations/ for investigation reports, and reports/jira-drafts/ for Jira drafts.

Tech

TypeScript · MCP TypeScript SDK · Zod · Vitest · JUnit XML · Jira Cloud REST API

Run with a real model in CI

The original README does not describe a real-model CI workflow for this repo, so none is claimed here.

License

MIT, Copyright (c) 2026 Harika Modali.

Available Tools

6 tools
create_jira_bugFile a Jira bugA

File a bug for a confirmed product defect. Include reproduction steps, expected vs actual result and the failing test name. Without Jira credentials this writes a draft file instead of creating a real issue.

Inputs: summary (required), description (required), labels, priority.

get_failure_detailsGet failure detailsA
Read-only

Return the error message, stack trace and captured output for failing tests whose name contains the given text.

Inputs: suite (required), testName (required).

get_test_resultsGet latest resultsB
Read-only

Summarize the most recent JUnit report for a suite without re-running it.

Inputs: suite (required).

list_test_suitesList test suitesA
Read-only

List the test suites this server can run, with their commands and report locations.

Inputs: no named parameters.

query_test_dataQuery test dataA
Read-only

Look up test accounts, products or other fixtures, filtered by exact field values.

Inputs: entity (required), limit, filter.

run_testsRun a test suiteA

Run a configured test suite (optionally filtered) and return pass/fail counts and the failing tests. Use get_failure_details next to dig into a specific failure.

Inputs: suite (required), filter.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    TDQS

    A3.9/5.0

    Scored across 6 tools

    Disambiguation5/5

    Each tool targets a distinct step in the test workflow: discovering suites, running tests, reading recent results, drilling into failures, querying fixtures, and filing a bug. The run_tests vs. get_test_results distinction is explicit: one executes, the other summarizes an existing report. get_failure_details is also clearly scoped to digging into named failing tests.

    Naming Consistency5/5

    All tools use consistent snake_case with a clear verb_noun pattern: list_test_suites, run_tests, get_test_results, get_failure_details, query_test_data, create_jira_bug. The convention is predictable throughout with no mixed styles.

    Tool Count5/5

    Six tools is well-scoped for a test automation server, covering the core workflow without redundant or filler operations. Each tool has a clear role in the run-inspect-debug-file lifecycle.

    Completeness4/5

    The surface covers the main workflow from listing suites through running tests, inspecting failures, querying fixtures, and filing a Jira bug. Minor gaps remain, such as historical results beyond the most recent report and suite configuration/management, but agents can work around them.

    Maintenance

    ActivityMaintained
    ResponsivenessNo recent community issues

    Related MCP Connectors

    Related MCP Servers

    • A
      license
      Not graded
      quality
      C
      maintenance
      Connect AI agents to your test results, insights, and targets. Query test runs, failures, flaky tests, and regressions across frameworks including Playwright, Jest, Pytest, Cypress and more.
      33 npm
      MIT
    • A
      license
      Not graded
      quality
      D
      maintenance
      Enables AI models (Claude, ChatGPT, GitHub Copilot) to run and analyze local tests, rerun failures, and orchestrate QA workflows using existing UI and API test frameworks.
      13 npm
      MIT
    • A
      license
      Not graded
      quality
      B
      maintenance
      Enables AI assistants to answer QA questions in a single call, covering run reports, failure grouping by root cause, flaky versus regressed test detection, and coverage gaps. Also exposes raw TestRail API access when a question needs it.
      MIT