# Secrets Detection Plugin (Rust)
High-performance secret detection plugin implemented in Rust with optimized regex pattern matching.
## Features
- **Optimized Pattern Matching**: High-performance regex-based secret detection
- **Zero-Copy PyO3 Integration**: Direct Python object traversal without JSON serialization overhead
- **Pattern Detection**: AWS keys, API tokens, private keys, database credentials, and more
- **Optimized for Large Payloads**: Direct PyO3 traversal avoids Python→JSON→Python round-trip overhead
## Prerequisites
- **Python**: 3.11+ (ABI3 compatible)
- **Rust**: Latest stable toolchain (1.70+)
- **maturin**: Python package builder for Rust extensions
- **Virtual Environment**: Activated MCP Gateway venv
Install prerequisites:
```bash
# Install Rust toolchain
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
# Install maturin
pip install maturin
```
## Build
```bash
cd plugins_rust/secrets_detection
maturin develop --release
```
The plugin automatically integrates with the Python fallback implementation in `plugins/secrets_detection/`, providing 2-8x performance improvements when available.
## Configuration
The plugin supports extensive configuration through the MCP Gateway plugin system:
```yaml
# plugins/config.yaml
secrets_detection:
enabled: true
config:
enabled:
aws_access_key_id: true
aws_secret_access_key: true
google_api_key: true
slack_token: true
private_key_block: true
jwt_like: true
hex_secret_32: true
base64_24: true
redact: false # Replace secrets with redaction_text
redaction_text: "***REDACTED***"
block_on_detection: true # Block requests containing secrets
min_findings_to_block: 1 # Threshold for blocking
```
## Integration with MCP Gateway
The Rust plugin seamlessly integrates as an acceleration layer for the existing Python plugin:
1. **Automatic Fallback**: If Rust plugin fails to load, falls back to Python implementation
2. **Hook Integration**: Supports `prompt_pre_fetch`, `tool_post_invoke`, and `resource_post_fetch` hooks
3. **Zero Configuration**: Drop-in replacement requiring no code changes
4. **Performance Logging**: Reports 2-8x speedup when Rust implementation is active
## Performance Comparison
Compare Python vs Rust implementations:
```bash
# From plugin directory
python compare_performance.py
# With custom iterations
python compare_performance.py --iterations 100 --warmup 10
```
The benchmark tests Rust vs Python implementations across multiple data sizes.
## Benchmarks
Run Criterion benchmarks:
```bash
cargo bench
```
Results are saved to `target/criterion/` with HTML reports.
## Performance Results
### Apple M1 Max Benchmarks
Tested on Apple M1 Max (10,000 iterations + 100 warmup):
| Scenario | Python | Rust | Speedup |
|----------|--------|------|---------|
| **1KB (no secrets)** | 0.073 ms | 0.010 ms | **7.17x** 🚀 |
| **1KB (with secrets)** | 0.076 ms | 0.021 ms | **3.65x** 🚀 |
| **5KB (no secrets)** | 0.348 ms | 0.042 ms | **8.25x** 🚀 |
| **5KB (with secrets)** | 0.369 ms | 0.092 ms | **4.01x** 🚀 |
**Key Findings:**
- **3.6-8.2x speedup** across all scenarios
- **Best performance gains** on clean data (no secrets): up to 8.25x faster
- **Significant improvements** even with secret detection: 3.6-4x faster
- **Consistent performance** across different data sizes and patterns
### CPU Architecture Performance
- **Apple Silicon (M1/M2)**: Consistent 1.3-1.6x speedup with optimized regex
- **x86_64**: Similar performance characteristics expected
- **ARM64**: Good performance across ARM-based systems
- **Cross-Platform**: Consistent behavior across all supported architectures
## Development
### Quick Commands
```bash
make install # Build and install plugin
make test # Run Rust unit tests
make test-all # Complete test suite (install, unit tests, integration)
make compare # Python vs Rust performance comparison
make bench # Run Criterion benchmarks
```
### Running Tests
```bash
# Rust unit tests
make test
# Complete test suite (recommended)
make test-all # Installs plugin, runs cargo test and Python tests
# Python unit tests
make test-python
# Performance comparison
make compare # Full comparison
make compare-quick # Fewer iterations
make compare-detailed # More iterations
```
### Adding New Patterns
1. Add pattern to `src/patterns.rs`
2. Update `PATTERNS` constant
3. Add corresponding test
4. Update both Python and Rust implementations
### Performance Profiling
```bash
# Criterion benchmarks
make bench
# Flamegraph profiling (heavy workload)
make flamegraph
```
The `flamegraph` target generates an interactive CPU profiling visualization:
1. Processes 1 million messages with realistic secret patterns
2. Creates `flamegraph.svg` showing CPU time distribution
3. Open the SVG in a browser to explore the interactive visualization
**Flamegraph shows:**
- Time spent in regex pattern matching
- String allocation and manipulation overhead
- Function call hierarchy and hot paths
- Performance bottlenecks in the detection pipeline
This is more useful than benchmark flamegraphs as it avoids Criterion's parallel execution overhead and focuses on the actual secret detection workload.
