Skip to main content
Glama

testmcp

An MCP server that wraps existing test frameworks (Jest, Vitest, Pytest) and exposes structured, LLM-optimized results via MCP tools.

Why?

Test frameworks were designed for humans. LLMs get ANSI escape codes, watch mode prompts, and 5000-line outputs that blow up context windows. testmcp fixes this with:

  • Progressive disclosure — Get a summary first, drill into failures on demand

  • Structured output — JSON, not terminal formatting

  • Diff-aware testing — Run only tests affected by your git changes

  • Time-budgeted execution — Set a timeout, get partial results

  • Source enrichment — Failure reports include the relevant source code around each assertion

  • Watch mode protection — Impossible to accidentally enter interactive mode

Related MCP server: Autonomous QA Engineer MCP

Installation

git clone https://github.com/r-marques/testmcp.git
cd testmcp
yarn install
yarn build

Configure as MCP server

Claude Code (recommended):

claude mcp add testmcp -- node /absolute/path/to/testmcp/dist/index.js

This registers testmcp as a local stdio MCP server. Restart Claude Code after adding.

Other MCP clients (Cursor, Windsurf, etc.):

Add to your MCP configuration file:

{
  "mcpServers": {
    "testmcp": {
      "command": "node",
      "args": ["/absolute/path/to/testmcp/dist/index.js"]
    }
  }
}

Verify it works

After restarting your MCP client, the testmcp tools should be available. In Claude Code you can verify with:

claude mcp list
# Should show: testmcp: node /path/to/testmcp/dist/index.js - ✓ Connected

Tools

Test Execution

Tool

Purpose

discover

Auto-detect test frameworks in a project

run_tests

Execute tests, return a compact summary

run_affected

Git-aware: only run tests affected by changes

get_failures

Drill into failures with source context

get_test_detail

Single test deep-dive with full stack trace

rerun_failed

Re-execute only previously failed tests

get_coverage

Coverage data from a previous run

list_runs

List recent test run summaries

CI Log Parsing

Tool

Purpose

parse_log

Parse raw CI/test log text and extract structured test results. Auto-detects framework.

list_artifacts

List artifacts from a GitHub Actions run

parse_artifact

Download a GitHub Actions artifact and parse it as test results

The CI tools enable a workflow where the LLM reads a CI workflow file to identify the right artifact, then uses parse_artifact to get structured results — no manual log parsing needed.

How It Works

The Progressive Disclosure Flow

Instead of dumping thousands of lines of test output, testmcp gives you information in layers:

Step 1: Run tests — Get a compact summary

run_tests({ projectDir: "/app" })
→ { total: 200, passed: 197, failed: 3, failedTests: ["auth > login > rejects expired token", ...] }

Step 2: Drill into failures — Get failure details with source context

get_failures({ runId: "abc-123" })
→ For each failure:
   - Concise error message
   - 7 lines of source code around the assertion
   - Test file and line number

Step 3: Deep dive — Full stack trace for a single test (only when needed)

get_test_detail({ runId: "abc-123", testName: "auth > login > rejects expired token" })
→ Complete error output with full stack trace

This approach uses 4-5x fewer tokens compared to raw test output in typical failure scenarios, because you never pay for the 197 passing tests you don't care about.

Diff-Aware Test Selection

run_affected analyzes your git changes through three layers:

  1. Direct changes — Test files you modified are included

  2. Naming conventions — Changed src/api/client.ts? Looks for tests/api/client.test.ts

  3. Import scanning — Greps test files for imports of changed source files

Each selected test includes a reason explaining why it was chosen.

Time-Budgeted Execution

run_tests({ projectDir: "/app", timeout: 30000 })

If the suite takes longer than 30 seconds, the process is killed and partial results are recovered. The response includes timedOut: true so you know results are incomplete.

Supported Frameworks

Framework

Detection

Output Parsing

Jest

jest.config.*, package.json

--json (native JSON reporter)

Vitest

vitest.config.*, package.json

--reporter=json (supports both v1.x tasks format and v2+ Jest-style format)

Pytest

pyproject.toml, pytest.ini, conftest.py

Layered fallback: pytest-reportlog JSONL → --junitxml (built-in) → verbose stdout parsing

testmcp auto-detects the framework and package manager (npm, yarn, pnpm, poetry). No configuration needed.

Pytest Fallback Chain

Pytest has no built-in JSON reporter, so testmcp uses a three-layer fallback strategy:

  1. pytest-reportlog (primary) — JSONL format via --report-log. Streaming-friendly, survives timeouts. Requires pip install pytest-reportlog.

  2. --junitxml (fallback) — Built into pytest, zero dependencies. Standard JUnit XML format.

  3. Verbose stdout (last resort) — Parses -v --tb=short output with regex. Works everywhere.

The server automatically falls through the chain if a plugin isn't installed — no configuration needed.

Architecture

src/
├── index.ts              # Entry: McpServer + StdioServerTransport
├── server.ts             # 11 MCP tool registrations & handlers
├── types.ts              # Core types
├── store.ts              # In-memory test run storage (LRU, 50 runs)
├── adapters/
│   ├── base.ts           # Abstract adapter interface
│   ├── jest.ts           # Jest adapter
│   ├── vitest.ts         # Vitest adapter (v1.x + v2+ formats)
│   └── pytest.ts         # Pytest adapter (reportlog → junitxml → verbose)
├── ci/
│   ├── log-parser.ts     # Framework detection + raw CI log parsing
│   └── artifacts.ts      # GitHub Actions artifact listing + download + parsing
├── git/
│   └── diff-analyzer.ts  # Git diff → affected test files
├── enrichment/
│   └── source-context.ts # Stack trace parser + source snippets
└── utils/
    ├── process.ts        # Child process runner with timeout
    └── detect.ts         # Framework auto-detection

Key Design Decisions

  • Stdio transport only — No HTTP server, no ports. Works as a local subprocess for Claude Code and IDE integrations.

  • Subprocess isolation — Frameworks are spawned as child processes with CI=true, TERM=dumb. Zero coupling to framework versions.

  • In-memory store — Test runs stored by UUID with LRU eviction at 50 runs. Enables progressive drill-down without re-running tests.

  • Minimal dependencies — Only @modelcontextprotocol/sdk and zod. Everything else uses Node.js built-ins.

Development

yarn build        # Compile TypeScript
yarn dev          # Run in dev mode (tsx)
yarn test         # Run test suite (Vitest)
yarn test:watch   # Run tests in watch mode

License

MIT

Available Tools

11 tools
discoverC

Auto-detect test frameworks configured in a project directory

Inputs: projectDir (required).

get_coverageB

Get coverage data from a previous test run.

Inputs: runId (required), files.

get_failuresB

Get detailed failure information for a test run, including source context around each failure.

Inputs: runId (required).

get_test_detailB

Get complete details for a single test, including full stack trace.

Inputs: runId (required), testName (required).

list_artifactsB

List artifacts from a GitHub Actions run. Use with parse_artifact to download and parse test result artifacts.

Inputs: repo (required), runId (required).

list_runsC

List recent test run summaries.

Inputs: limit.

parse_artifactB

Download a GitHub Actions artifact and parse it as test results. Supports JUnit XML, Jest JSON, Vitest JSON, and pytest-reportlog JSONL.

Inputs: repo (required), runId (required), artifactName (required).

parse_logA

Parse raw CI/test log text and extract structured test results. Auto-detects framework (Jest, Vitest, Pytest) from the log content.

Inputs: text (required), framework.

rerun_failedB

Re-execute only the tests that failed in a previous run.

Inputs: runId (required), timeout.

run_affectedA

Run only tests affected by git changes. Analyzes diff to find related test files.

Inputs: projectDir (required), base, timeout, coverage.

run_testsA

Run tests and return a compact summary. Use get_failures to drill into failures.

Inputs: projectDir (required), timeout, coverage, fileGlob, +2 more.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    TDQS

    A3.5/5.0

    Scored across 11 tools

    Disambiguation4/5

    Most tools have distinct resource+action targets: run_tests vs run_affected, list_runs vs list_artifacts, and get_failures vs get_test_detail are separable. However, get_failures and get_test_detail both surface failure detail, and parse_log and parse_artifact both 'parse' test data, creating mild potential for misselection.

    Naming Consistency4/5

    Nearly all names follow a consistent snake_case verb_noun pattern (list_runs, run_tests, get_failures, parse_artifact, rerun_failed). The lone outlier is 'discover', a bare verb with no noun object, which is a minor deviation from the otherwise uniform convention.

    Tool Count5/5

    11 tools is well-scoped for a test-runner server, with each tool covering a distinct facet (discovery, execution, failure drill-down, coverage, artifact parsing). No redundant or filler tools appear.

    Completeness4/5

    The surface covers the full test lifecycle: discovery, running (full and affected), failure inspection, reruns, coverage, and CI log/artifact parsing. Minor gaps exist (e.g., no config/watch or artifact listing beyond GitHub Actions), but core workflows are complete.

    Maintenance

    ActivityInactive
    ResponsivenessOutdated data

    Related MCP Connectors

    Related MCP Servers