-
securityF
license-
qualityA Model Context Protocol server that integrates with the Qase test management platform, allowing users to create and retrieve test cases, manage test runs, and interact with Qase projects.
Last updated -
6
1
The Test Runner MCP server provides a unified interface for executing and parsing test results across multiple testing frameworks:
Run tests for multiple frameworks: Bats, Pytest, Flutter, Jest, Go, Rust, and generic commands
Capture and parse test results into structured output while preserving raw output for debugging
Configure execution parameters including working directories, environment variables, and timeouts (default: 300000ms)
Security features to validate and sanitize commands, with configurable overrides for specific elevated permissions
Enhanced framework support such as automatic environment setup for Flutter and detailed error handling for Rust (RUST_BACKTRACE=1)
CI/CD integration with flexible automation and pipeline support for arbitrary command execution
Integrates with Dependabot for automated dependency updates
Allows running and parsing Flutter tests, with enhanced support including environment setup, error handling, and detailed output processing
Integrates with GitHub Actions for continuous integration, including automated testing on Node.js 18.x and 20.x, test results uploaded as artifacts
Allows running and parsing Jest (JavaScript Testing Framework) tests and processing their outputs
Allows running and parsing Pytest (Python Testing Framework) tests and processing their outputs
Test Runner MCP
A Model Context Protocol (MCP) server for running and parsing test results from multiple testing frameworks. This server provides a unified interface for executing tests and processing their outputs, supporting:
Bats (Bash Automated Testing System)
Pytest (Python Testing Framework)
Flutter Tests
Jest (JavaScript Testing Framework)
Go Tests
Rust Tests (Cargo test)
Generic (for arbitrary command execution)
Installation
npm install test-runner-mcp
Prerequisites
The following test frameworks need to be installed for their respective test types:
Bats:
apt-get install batsorbrew install batsPytest:
pip install pytestFlutter: Follow Flutter installation guide
Jest:
npm install --save-dev jestGo: Follow Go installation guide
Rust: Follow Rust installation guide
Usage
Configuration
Add the test-runner to your MCP settings (e.g., in claude_desktop_config.json or cline_mcp_settings.json):
{
"mcpServers": {
"test-runner": {
"command": "node",
"args": ["/path/to/test-runner-mcp/build/index.js"],
"env": {
"NODE_PATH": "/path/to/test-runner-mcp/node_modules",
// Flutter-specific environment (required for Flutter tests)
"FLUTTER_ROOT": "/opt/homebrew/Caskroom/flutter/3.27.2/flutter",
"PUB_CACHE": "/Users/username/.pub-cache",
"PATH": "/opt/homebrew/Caskroom/flutter/3.27.2/flutter/bin:/usr/local/bin:/usr/bin:/bin"
}
}
}
}
Note: For Flutter tests, ensure you replace:
/opt/homebrew/Caskroom/flutter/3.27.2/flutterwith your actual Flutter installation path/Users/username/.pub-cachewith your actual pub cache pathUpdate PATH to include your system's actual paths
You can find these values by running:
# Get Flutter root
flutter --version
# Get pub cache path
echo $PUB_CACHE # or default to $HOME/.pub-cache
# Get Flutter binary path
which flutter
Running Tests
Use the run_tests tool with the following parameters:
{
"command": "test command to execute",
"workingDir": "working directory for test execution",
"framework": "bats|pytest|flutter|jest|go|rust|generic",
"outputDir": "directory for test results",
"timeout": "test execution timeout in milliseconds (default: 300000)",
"env": "optional environment variables",
"securityOptions": "optional security options for command execution"
}
Example for each framework:
// Bats
{
"command": "bats test/*.bats",
"workingDir": "/path/to/project",
"framework": "bats",
"outputDir": "test_reports"
}
// Pytest
{
"command": "pytest test_file.py -v",
"workingDir": "/path/to/project",
"framework": "pytest",
"outputDir": "test_reports"
}
// Flutter
{
"command": "flutter test test/widget_test.dart",
"workingDir": "/path/to/project",
"framework": "flutter",
"outputDir": "test_reports",
"FLUTTER_ROOT": "/opt/homebrew/Caskroom/flutter/3.27.2/flutter",
"PUB_CACHE": "/Users/username/.pub-cache",
"PATH": "/opt/homebrew/Caskroom/flutter/3.27.2/flutter/bin:/usr/local/bin:/usr/bin:/bin"
}
// Jest
{
"command": "jest test/*.test.js",
"workingDir": "/path/to/project",
"framework": "jest",
"outputDir": "test_reports"
}
// Go
{
"command": "go test ./...",
"workingDir": "/path/to/project",
"framework": "go",
"outputDir": "test_reports"
}
// Rust
{
"command": "cargo test",
"workingDir": "/path/to/project",
"framework": "rust",
"outputDir": "test_reports"
}
// Generic (for arbitrary commands, CI/CD tools, etc.)
{
"command": "act -j build",
"workingDir": "/path/to/project",
"framework": "generic",
"outputDir": "test_reports"
}
// Generic with security overrides
{
"command": "sudo docker-compose -f docker-compose.test.yml up",
"workingDir": "/path/to/project",
"framework": "generic",
"outputDir": "test_reports",
"securityOptions": {
"allowSudo": true
}
}
Security Features
The test-runner includes built-in security features to prevent execution of potentially harmful commands, particularly for the generic framework:
Command Validation
Blocks
sudoandsuby defaultPrevents dangerous commands like
rm -rf /Blocks file system write operations outside of safe locations
Environment Variable Sanitization
Filters out potentially dangerous environment variables
Prevents overriding critical system variables
Ensures safe path handling
Configurable Security
Override security restrictions when necessary via
securityOptionsFine-grained control over security features
Default safe settings for standard test usage
Security options you can configure:
{
"securityOptions": {
"allowSudo": false, // Allow sudo commands
"allowSu": false, // Allow su commands
"allowShellExpansion": true, // Allow shell expansion like $() or backticks
"allowPipeToFile": false // Allow pipe to file operations (> or >>)
}
}
Flutter Test Support
The test runner includes enhanced support for Flutter tests:
Environment Setup
Automatic Flutter environment configuration
PATH and PUB_CACHE setup
Flutter installation verification
Error Handling
Stack trace collection
Assertion error handling
Exception capture
Test failure detection
Output Processing
Complete test output capture
Stack trace preservation
Detailed error reporting
Raw output preservation
Rust Test Support
The test runner provides specific support for Rust's cargo test:
Environment Setup
Automatically sets RUST_BACKTRACE=1 for better error messages
Output Parsing
Parses individual test results
Captures detailed error messages for failed tests
Identifies ignored tests
Extracts summary information
Generic Test Support
For CI/CD pipelines, GitHub Actions via act, or any other command execution, the generic framework provides:
Automatic Output Analysis
Attempts to segment output into logical blocks
Identifies section headers
Detects pass/fail indicators
Provides reasonable output structure even for unknown formats
Flexible Integration
Works with arbitrary shell commands
No specific format requirements
Perfect for integration with tools like
act, Docker, and custom scripts
Security Features
Command validation to prevent harmful operations
Can be configured to allow specific elevated permissions when necessary
Output Format
The test runner produces structured output while preserving complete test output:
interface TestResult {
name: string;
passed: boolean;
output: string[];
rawOutput?: string; // Complete unprocessed output
}
interface TestSummary {
total: number;
passed: number;
failed: number;
duration?: number;
}
interface ParsedResults {
framework: string;
tests: TestResult[];
summary: TestSummary;
rawOutput: string; // Complete command output
}
Results are saved in the specified output directory:
test_output.log: Raw test outputtest_errors.log: Error messages if anytest_results.json: Structured test resultssummary.txt: Human-readable summary
Development
Setup
Clone the repository
Install dependencies:
npm installBuild the project:
npm run build
Running Tests
npm test
The test suite includes tests for all supported frameworks and verifies both successful and failed test scenarios.
CI/CD
The project uses GitHub Actions for continuous integration:
Automated testing on Node.js 18.x and 20.x
Test results uploaded as artifacts
Dependabot configured for automated dependency updates
Contributing
Fork the repository
Create your feature branch
Commit your changes
Push to the branch
Create a Pull Request
License
This project is licensed under the MIT License - see the LICENSE file for details.
hybrid server
The server is able to function both locally and remotely, depending on the configuration or use case.
Tools
Facilitates unified execution and result parsing for various testing frameworks, including Bats, Pytest, Flutter, Jest, and Go, through a Model Context Protocol interface.
Related Resources
Related MCP Servers
- AsecurityFlicenseAqualityA Model Context Protocol server that allows AI assistants to execute and manage JMeter performance tests through natural language commands.Last updated -645
- -securityAlicense-qualityA Model Context Protocol server that provides programmatic APIs for running mutation tests with mutmut, analyzing results, and improving test coverage in Python projects.Last updated -MIT License
- AsecurityFlicenseAqualityA testing server that demonstrates Model Context Protocol features, providing access to user data, todos, and system information through resources, tools for user management and calculations, and prompt templates for various analyses.Last updated -4