Skip to main content
Glama
mcp-test-runner

mcp-test-runner

by LovRanRan

mcp-test-runner

MCP server for deterministic local test execution and normalized test result reporting.

mcp-test-runner is the verification layer for codebase onboarding agents. It exposes focused MCP tools for running pytest and Jest, parsing test output, and summarizing coverage so downstream agents can mark claims as verified, unverified, or contradicted by real execution.

Codebase Onboarding Stack

mcp-test-runner is the verification layer in a three-server MCP tool stack for Project 6 wayfinder, a codebase onboarding agent.

  • mcp-repo-mapper maps repository structure, languages, entry points, framework evidence, and Python dependency edges.

  • mcp-ast-explorer provides symbol-grounded Python definition, signature, reference, call-chain, and class-hierarchy lookups.

  • mcp-test-runner runs local pytest/Jest checks and coverage summaries so agent claims can be verified against execution.

In wayfinder, this server turns high-risk code understanding claims into verified, unverified, or contradicted evidence from real test execution.

Related MCP server: MCP Testing Assistant

Status

This repository is a Python-first v1 MCP test runner. It supports bounded pytest execution, Jest command execution, single-test targeting, pytest coverage summaries, and normalized pytest/Jest JSON parsing.

The server does not use an LLM. Test results come from subprocess execution and structured parser output.

Tools

Tool

Purpose

health()

Returns ok for smoke checks.

run_pytest(path, test_filter?, timeout_seconds?, cpu_seconds?, memory_mb?)

Run pytest in a bounded working directory and return raw command output.

run_jest(path, test_filter?, timeout_seconds?, cpu_seconds?, memory_mb?)

Run Jest through npx jest and return raw command output.

run_single_test(path, test_id, framework?, timeout_seconds?, cpu_seconds?, memory_mb?)

Run one pytest node id or one Jest test-name pattern.

parse_test_output(stdout, framework)

Normalize test runner JSON output.

get_coverage_summary(path, framework?, timeout_seconds?, cpu_seconds?, memory_mb?)

Return pytest-cov JSON coverage totals.

Supported Scope

  • Python 3.11+ package.

  • FastMCP 2.x server.

  • pytest execution with JSON report output via pytest-json-report.

  • Jest command execution via npx jest --json --outputFile.

  • pytest and Jest JSON normalization into one TestRunResult schema.

  • Single pytest node id execution and Jest test-name targeting.

  • pytest-cov coverage summary from .coverage.json.

  • Subprocess timeout plus POSIX resource.setrlimit CPU / memory caps.

Current Limitations

  • v1 does not use Docker. It uses cwd validation, subprocess timeouts, and POSIX resource limits.

  • Resource limits require a POSIX platform that supports resource.setrlimit.

  • Jest execution expects Node.js plus project-local or npx-resolvable Jest.

  • Jest single-test targeting uses --testNamePattern; it does not parse Jest file-specific node ids.

  • Coverage summary is pytest-only in v1.

  • Tools return raw command output for execution; call parse_test_output to normalize framework JSON.

Local Development

Install dependencies:

uv sync --extra dev

The PyPI distribution name is lovranran-mcp-test-runner because mcp-test-runner is already taken on PyPI. The installed console script remains mcp-test-runner.

Run the MCP server:

uv run mcp-test-runner

Run verification:

uv run ruff check .
uv run mypy
uv run pytest

License

MIT

Available Tools

6 tools
get_coverage_summaryD

Inputs: path (required), framework, memory_mb, cpu_seconds, +1 more. Returns: framework, report_path, total_lines, covered_lines, +1 more

healthD

Inputs: no named parameters. Returns: result

parse_test_outputD

Inputs: stdout (required), framework. Returns: failed, passed, skipped, failures

run_jestD

Inputs: path (required), memory_mb, cpu_seconds, test_filter, +1 more. Returns: stderr, stdout, exit_code, timed_out

run_pytestD

Inputs: path (required), memory_mb, cpu_seconds, test_filter, +1 more. Returns: stderr, stdout, exit_code, timed_out

run_single_testD

Inputs: path (required), test_id (required), framework, memory_mb, +2 more. Returns: stderr, stdout, exit_code, timed_out

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    TDQS

    D1.9/5.0

    Scored across 6 tools

    Disambiguation4/5

    The tool names clearly differentiate between different testing frameworks (pytest vs jest), single test execution, output parsing, coverage, and health. Without descriptions, there is slight potential confusion between run_pytest and run_single_test, but names are sufficiently distinct.

    Naming Consistency3/5

    Three tools use the 'run_' prefix (run_pytest, run_jest, run_single_test), but parse_test_output, get_coverage_summary, and health deviate. The pattern is mixed but still readable and not chaotic.

    Tool Count5/5

    With 6 tools, the server covers essential test runner operations: running tests by framework, running a single test, parsing output, coverage, and health check. Each tool serves a distinct purpose without unnecessary bloat.

    Completeness4/5

    The tool set covers core testing workflows: running tests (multiple frameworks), analyzing results (parse output, coverage), and health monitoring. Minor gaps like a combined test runner or report generation exist but do not hinder basic usage.

    Related MCP Connectors

    Related MCP Servers

    • F
      license
      Not graded
      quality
      D
      maintenance
      A Node.js server that integrates with pytest to facilitate the ModelContextProtocol (MCP) service tools, enabling test execution recording and environment tracking.
      6
      -
    • -
      license
      Not graded
      quality
      Not graded
      maintenance
      Enables automated testing and coverage reporting for MCP services with test execution, file generation, and mock service creation. Provides comprehensive testing infrastructure including Jest integration, coverage reports, and health checks for the MCP ecosystem.
      -
    • A
      license
      Not graded
      quality
      B
      maintenance
      MCP server for test impact analysis and code intelligence. Maps tests to code and git history to determine impacted tests, risk scores, and ownership for AI coding agents.
      8
      MIT
    • A
      license
      Not graded
      quality
      D
      maintenance
      A universal AI-powered testing server built on the Model Context Protocol (MCP). Allows AI agents to inspect, execute, test, monitor, debug, and report on software projects.
      3
      GNU Lesser General Public v2.1 only