SpeechPulse
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@SpeechPulse Analyze emotion and urgency in audio file /path/to/call.wav"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
SpeechPulse
Voice Emotion Understanding MCP Server
SpeechPulse analyzes speech audio to detect emotions, assess urgency, and detect sarcasm using prosodic features (pitch, energy, rhythm). Built with pure Python standard library for zero ML dependencies in the Lite tier.
Features
Emotion Detection: Recognizes 7 emotions (happy, excited, angry, sad, tired, anxious, neutral) using coefficient of variation (CV) thresholds
Urgency Assessment: 4-level urgency detection (low, medium, high, critical) based on speaking patterns
Sarcasm Detection: Identifies sarcasm by comparing text sentiment with audio emotion
Zero ML Dependencies: Lite tier uses pure Python standard library (no numpy/scipy/librosa)
MCP Compatible: Exposes tools via Model Context Protocol for integration with Claude Desktop and other MCP clients
Related MCP server: genpark-multimodal-audio-emotion-valence-detector-skill
Installation
From PyPI (when published)
pip install speechpulseFrom Source
git clone https://github.com/sophieMiao/speechpulse.git
cd speechpulse
pip install -e ".[dev]"Quick Start
As MCP Server
Add to your MCP client configuration (e.g., Claude Desktop):
{
"mcpServers": {
"speechpulse": {
"command": "python",
"args": ["-m", "speechpulse"],
"env": {
"SPEECHPULSE_TIER": "lite"
}
}
}
}As Python Library
from speechpulse.analyzer import SpeechAnalyzer
# Initialize analyzer
analyzer = SpeechAnalyzer()
# Analyze emotion
result = analyzer.analyze("path/to/audio.wav")
print(f"Primary emotion: {result['emotion']['primary']}")
# Assess urgency
urgency = analyzer.assess_urgency("path/to/audio.wav")
print(f"Urgency level: {urgency.level}")
# Detect sarcasm (requires text in Lite tier)
sarcasm = analyzer.detect_sarcasm(
"path/to/audio.wav",
text="这真是太棒了"
)
print(f"Is sarcastic: {sarcasm.is_sarcastic}")
# Full analysis
full = analyzer.full_analysis("path/to/audio.wav", text="我受够了!")
print(full['summary'])
print(full['interpretation'])CLI Usage
# Start MCP server with stdio transport (default)
python -m speechpulse
# Start with SSE transport
python -m speechpulse --transport sse --port 8080
# Enable verbose logging
python -m speechpulse -vMCP Tools
analyze_audio
Analyze audio for emotion and basic features.
Parameters:
audio_path(string, required): Path to WAV audio filetext(string, optional): Transcription text for context
Returns: Emotion detection results, speaker state, and raw audio features
assess_urgency
Assess urgency level from audio prosody.
Parameters:
audio_path(string, required): Path to audio filetext(string, optional): Text for keyword-based urgency detection
Returns: Urgency score, level, and reasoning
detect_sarcasm
Detect sarcasm by comparing text sentiment with audio emotion.
Parameters:
audio_path(string, required): Path to audio filetext(string, optional): Transcription text (recommended)
Returns: Sarcasm detection result with confidence and indicators
full_analysis
Perform complete analysis (emotion + urgency + sarcasm).
Parameters:
audio_path(string, required): Path to audio filetext(string, optional): Transcription text
Returns: Complete analysis with summary and interpretation
health_check
Check server health and capabilities.
Returns: Status, version, tier, and available capabilities
Architecture
speechpulse/
├── types.py # Core data types (AudioFeatures, EmotionResult, etc.)
├── config.py # Configuration management
├── utils.py # Audio loading and processing utilities
├── audio_features.py # Feature extraction (pitch, energy, etc.)
├── emotion.py # CV-based emotion rule engine
├── urgency.py # Urgency assessment logic
├── sarcasm.py # Sarcasm detection
├── analyzer.py # Main analysis pipeline
├── server.py # MCP server implementation
├── asr.py # ASR stub (Standard/Pro tier)
└── ml_emotion.py # ML emotion stub (Pro tier)Technical Details
Audio Processing
Pure Python: Uses only
wave,struct,math, andarraymodulesFormat Support: WAV files with 8/16/24/32-bit PCM
Resampling: Linear interpolation to 16kHz
Framing: 32ms frames with 50% overlap, Hamming window
Feature Extraction
Pitch: Autocorrelation-based F0 detection (50-500 Hz range)
Energy: RMS energy per frame
Zero Crossing Rate: Voice/unvoiced discrimination
Silence Ratio: Pause pattern analysis
Emotion Recognition
Uses coefficient of variation (CV = std/mean) to avoid gender bias while maintaining discriminative power:
# Example: Happy emotion rule (using coefficient of variation)
"happy": {
"conditions": [
("pitch_cv", ">", 0.15), # High pitch variation (lively)
("energy_mean", ">", 0.3), # Moderate-high energy
("energy_cv", ">", 0.2), # Energy fluctuation
],
"weight": 0.8,
}Urgency Assessment
Based on 5 factors:
Speaking rate (fast/medium/slow)
Volume level (high/medium/low)
Pitch variation (high/medium/low)
Pause pattern (few/normal/many pauses)
Keyword detection (when text provided)
Tiers
Lite Tier (Current)
✅ Rule-based emotion recognition
✅ Prosodic urgency assessment
✅ Keyword-based sarcasm detection
✅ Pure Python (no ML dependencies)
❌ No ASR (provide text manually)
❌ WAV format only
Standard Tier (Planned)
ASR with faster-whisper
Additional audio formats (MP3, FLAC, etc.)
Speaker diarization
Pro Tier (Planned)
Qwen2-Audio integration
Context-aware emotion analysis
Nuanced emotion detection
Real-time streaming
Development
Setup
# Clone repository
git clone https://github.com/sophieMiao/speechpulse.git
cd speechpulse
# Create virtual environment
python -m venv venv
source venv/bin/activate # Windows: venv\Scripts\activate
# Install in development mode
pip install -e ".[dev]"Running Tests
# Run all tests
python -m pytest tests/
# Run specific test file
python tests/test_all.py
# Run integration tests
python tests/test_integration.pyDemo
# Run demo script
python examples/demo.pyConfiguration
Environment variables:
Variable | Default | Description |
|
| Service tier (lite/standard/pro) |
|
| Target sample rate |
|
| Analysis frame size |
|
| Frame hop size |
Limitations
Lite tier requires text for sarcasm detection: Provide transcription via
textparameterWAV format only: Convert other formats to WAV before analysis
Rule-based emotions: ML-based nuanced emotion detection in Pro tier
Optimized for Chinese/English: Full multilingual support in Pro tier
Contributing
Fork the repository
Create a feature branch (
git checkout -b feature/amazing-feature)Commit changes (
git commit -m 'Add amazing feature')Push to branch (
git push origin feature/amazing-feature)Open a Pull Request
License
This project is licensed under the MIT License - see the LICENSE file for details.
Acknowledgments
Built with MCP SDK
Inspired by prosodic analysis research in speech emotion recognition
CV approach based on gender-fair emotion recognition research
Support
GitHub Issues: https://github.com/sophieMiao/speechpulse/issues
Made with ❤️ for voice emotion understanding
Available Tools
5 toolsanalyze_audioA
Analyze audio for emotion and basic features.
This tool analyzes speech audio to detect the speaker's emotional state and extract basic audio features. For Lite tier, ASR is not included, so provide the 'text' parameter if you have a transcription.
Args: audio_path: Path to the audio file (WAV format supported) text: Optional transcription text for context
Returns: Dictionary containing: - transcription: None for Lite tier (ASR not included) - note: Information about Lite tier limitations - emotion: Object with primary emotion, confidence, secondary emotion, scores - speaker_state: Object with energy_level and stress_indicator - features: Raw audio features (duration, pitch, energy, etc.)
Example: { "transcription": null, "note": "Lite tier does not include ASR...", "emotion": { "primary": "happy", "confidence": 0.85, "secondary": "excited", "scores": {"happy": 0.8, "excited": 0.6, ...} }, "speaker_state": { "energy_level": "high", "stress_indicator": "low" }, "features": {...} }
Inputs: audio_path (required), text. Returns: result
assess_urgencyA
Assess urgency level from audio.
This tool evaluates the urgency level of speech based on prosodic features like speaking rate, volume, pitch variation, and pause patterns.
Args: audio_path: Path to the audio file (WAV format supported) text: Optional transcription text for keyword-based urgency detection
Returns: Dictionary containing: - score: Urgency score (0.0 to 1.0) - level: Urgency level ("low", "medium", "high", "critical") - reasoning: List of factors contributing to the urgency assessment - factors: Detailed breakdown of contributing factors
Example: { "score": 0.75, "level": "high", "reasoning": ["Fast speaking rate detected", "High volume variation"], "factors": { "speaking_rate": "fast", "volume_level": "high", "pitch_variation": "high", "pause_pattern": "few_pauses" } }
Inputs: audio_path (required), text. Returns: result
detect_sarcasmA
Detect sarcasm by comparing text sentiment with audio emotion.
This tool detects sarcasm by analyzing the mismatch between the sentiment of the text and the emotional tone of the audio. For Lite tier, the 'text' parameter is required for accurate detection.
Args: audio_path: Path to the audio file (WAV format supported) text: Transcription text (recommended for Lite tier)
Returns: Dictionary containing: - is_sarcastic: Boolean indicating sarcasm detection - confidence: Confidence score (0.0 to 1.0) - indicators: List of indicators that suggest sarcasm - text_emotion: Detected emotion from text (if available) - audio_emotion: Detected emotion from audio
Example: { "is_sarcastic": true, "confidence": 0.82, "indicators": ["Positive text with negative audio tone"], "text_emotion": "positive", "audio_emotion": "sad" }
Inputs: audio_path (required), text. Returns: result
full_analysisA
Perform complete analysis including emotion, urgency, and sarcasm.
This tool performs a comprehensive analysis of speech audio, combining emotion recognition, urgency assessment, and sarcasm detection into a single coherent result.
Args: audio_path: Path to the audio file (WAV format supported) text: Optional transcription text (recommended for complete analysis)
Returns: Dictionary containing: - summary: Human-readable summary of the analysis - transcription: None for Lite tier (ASR not included) - note: Information about Lite tier limitations - emotion_analysis: Complete emotion analysis results - urgency_assessment: Complete urgency assessment results - sarcasm_detection: Complete sarcasm detection results - raw_features: Raw audio features extracted - interpretation: Contextual interpretation (if text provided)
Example: { "summary": "说话者表现出开心的情绪。带有明显的紧迫感(high级别)。", "transcription": null, "note": "Lite tier does not include ASR...", "emotion_analysis": {...}, "urgency_assessment": {...}, "sarcasm_detection": {...}, "raw_features": {...}, "interpretation": "用户语气急促且带有焦虑情绪;建议尽快联系处理。" }
Inputs: audio_path (required), text. Returns: result
health_checkA
Check server health status.
This tool can be used to verify that the SpeechPulse MCP server is running and functioning correctly.
Returns: Dictionary containing: - status: "healthy" or "unhealthy" - version: Server version - tier: Current tier ("lite", "standard", or "pro") - capabilities: List of available capabilities
Example: { "status": "healthy", "version": "0.1.0", "tier": "lite", "capabilities": [ "emotion_analysis", "urgency_assessment", "sarcasm_detection" ] }
Inputs: no named parameters. Returns: result
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
TDQS
Scored across 5 tools
Tools have distinct focuses (emotion, urgency, sarcasm, health) but full_analysis overlaps by combining all three analyses, creating potential confusion for an agent choosing between individual and combined tools.
Most tools follow a verb_noun pattern (analyze_audio, assess_urgency, detect_sarcasm) but full_analysis and health_check deviate, mixing noun phrases and lacking consistent verb usage.
With 5 tools, the server covers core audio analysis tasks (emotion, urgency, sarcasm) plus a combined analysis and health check, which is well-scoped and reasonable for the domain.
The set covers emotion, urgency, and sarcasm detection, but lacks a dedicated transcription tool (ASR is optional via parameter) and other potential features like speaker identification, leaving moderate gaps.
Related MCP Connectors
Analyze consented voice recordings for wellbeing and communication coaching.
Pronunciation assessment, phoneme scoring, speaker voice ID, audio transcription, speech synthesis.
Hosted speech-to-text + speech emotion/tone analysis for agents. No install; trial keys built in.
Sentiment, toxicity, entity extraction, PII, translation, summary, QA, fraud scoring, safety audit.
Related MCP Servers
- AlicenseAqualityCmaintenanceProvides local audio analysis tools for LLMs, enabling transcription, conversation dynamics, prosody analysis, and visual inspection without API keys.8MIT
- FlicenseNot gradedqualityNot gradedmaintenanceEnables analysis of acoustic vocal emotion, prosodic valence, and customer frustration from audio inputs through a zero-dependency MCP server.1-
- FlicenseNot gradedqualityBmaintenanceEnables MCP-compliant agents to analyze speech prosody and vocal sentiment from audio-derived input, measuring speech rate, pitch variance, and user cognitive fatigue. Exposes these deterministic results over JSON-RPC so clients can dynamically calibrate their response pacing.7-
- FlicenseNot gradedqualityNot gradedmaintenanceEnables agents to analyze speech prosody and vocal sentiment, including speech rate, pitch variance, and cognitive fatigue, to dynamically calibrate response pacing.1-