Integrations
Integrates with Dependabot for automated dependency updates
Allows running and parsing Flutter tests, with enhanced support including environment setup, error handling, and detailed output processing
Integrates with GitHub Actions for continuous integration, including automated testing on Node.js 18.x and 20.x, test results uploaded as artifacts
Test Runner MCP
A Model Context Protocol (MCP) server for running and parsing test results from multiple testing frameworks. This server provides a unified interface for executing tests and processing their outputs, supporting:
- Bats (Bash Automated Testing System)
- Pytest (Python Testing Framework)
- Flutter Tests
- Jest (JavaScript Testing Framework)
- Go Tests
- Rust Tests (Cargo test)
- Generic (for arbitrary command execution)
Installation
Prerequisites
The following test frameworks need to be installed for their respective test types:
- Bats:
apt-get install bats
orbrew install bats
- Pytest:
pip install pytest
- Flutter: Follow Flutter installation guide
- Jest:
npm install --save-dev jest
- Go: Follow Go installation guide
- Rust: Follow Rust installation guide
Usage
Configuration
Add the test-runner to your MCP settings (e.g., in claude_desktop_config.json
or cline_mcp_settings.json
):
Note: For Flutter tests, ensure you replace:
/opt/homebrew/Caskroom/flutter/3.27.2/flutter
with your actual Flutter installation path/Users/username/.pub-cache
with your actual pub cache path- Update PATH to include your system's actual paths
You can find these values by running:
Running Tests
Use the run_tests
tool with the following parameters:
Example for each framework:
Security Features
The test-runner includes built-in security features to prevent execution of potentially harmful commands, particularly for the generic
framework:
- Command Validation
- Blocks
sudo
andsu
by default - Prevents dangerous commands like
rm -rf /
- Blocks file system write operations outside of safe locations
- Blocks
- Environment Variable Sanitization
- Filters out potentially dangerous environment variables
- Prevents overriding critical system variables
- Ensures safe path handling
- Configurable Security
- Override security restrictions when necessary via
securityOptions
- Fine-grained control over security features
- Default safe settings for standard test usage
- Override security restrictions when necessary via
Security options you can configure:
Flutter Test Support
The test runner includes enhanced support for Flutter tests:
- Environment Setup
- Automatic Flutter environment configuration
- PATH and PUB_CACHE setup
- Flutter installation verification
- Error Handling
- Stack trace collection
- Assertion error handling
- Exception capture
- Test failure detection
- Output Processing
- Complete test output capture
- Stack trace preservation
- Detailed error reporting
- Raw output preservation
Rust Test Support
The test runner provides specific support for Rust's cargo test
:
- Environment Setup
- Automatically sets RUST_BACKTRACE=1 for better error messages
- Output Parsing
- Parses individual test results
- Captures detailed error messages for failed tests
- Identifies ignored tests
- Extracts summary information
Generic Test Support
For CI/CD pipelines, GitHub Actions via act
, or any other command execution, the generic framework provides:
- Automatic Output Analysis
- Attempts to segment output into logical blocks
- Identifies section headers
- Detects pass/fail indicators
- Provides reasonable output structure even for unknown formats
- Flexible Integration
- Works with arbitrary shell commands
- No specific format requirements
- Perfect for integration with tools like
act
, Docker, and custom scripts
- Security Features
- Command validation to prevent harmful operations
- Can be configured to allow specific elevated permissions when necessary
Output Format
The test runner produces structured output while preserving complete test output:
Results are saved in the specified output directory:
test_output.log
: Raw test outputtest_errors.log
: Error messages if anytest_results.json
: Structured test resultssummary.txt
: Human-readable summary
Development
Setup
- Clone the repository
- Install dependencies:Copy
- Build the project:Copy
Running Tests
The test suite includes tests for all supported frameworks and verifies both successful and failed test scenarios.
CI/CD
The project uses GitHub Actions for continuous integration:
- Automated testing on Node.js 18.x and 20.x
- Test results uploaded as artifacts
- Dependabot configured for automated dependency updates
Contributing
- Fork the repository
- Create your feature branch
- Commit your changes
- Push to the branch
- Create a Pull Request
License
This project is licensed under the MIT License - see the LICENSE file for details.
You must be authenticated.
local-only server
The server can only run on the client's local machine because it depends on local resources.
Tools
Facilitates unified execution and result parsing for various testing frameworks, including Bats, Pytest, Flutter, Jest, and Go, through a Model Context Protocol interface.