Rheo

Fluid. Human. Sovereign.

100% Offline • Zero-Shot Voice Cloning • Multi-Engine Neural Synthesis • Zero-Latency Dictation • MCP Agent Native

"Everything flows, nothing stands still."
— Heraclitus (panta rhei)

Rheo is built from the ground up from the user's perspective—not an enterprise billing API. Most synthetic voice platforms are predatory subscription black boxes that harvest your vocal biometric signature, inject watermarks, throttle bandwidth, and leak personal speech transcripts to external cloud servers. Rheo feels weightless, natural, and sovereign: studio-grade zero-shot voice cloning, real-time multi-engine speech synthesis, system-wide continuous dictation, and native AI agent tooling running 100% locally on your own silicon.

The Brag Sheet: Why Rheo Dominates Alternative Voice Systems

"Cloud voice platforms ask you to trust their servers, their API rate limits, and their ongoing monthly billing. Rheo asks you to trust zero servers—because it executes directly on your own GPU."

Head-to-Head Showdown: Rheo vs. Industry Platforms

Architectural Scorecard (10 / 10 Across All Dimensions)

Related MCP server: VoxMesh

The Philosophy: Built for Humans (rheo · Panta Rhei)

Voice is human identity. When speech synthesis software is built as a metering tool for cloud vendors, it treats human expression as a billable transaction. When built from a user's perspective:

• The vocal stream is continuous. Derived from the ancient Greek verb rheo (to flow, to stream), the project treats voice not as static rendered audio, but as an ongoing flow of thought. Spoken words become text instantly; written thoughts become expressive voice effortlessly.

• Biometric sovereignty is non-negotiable. Your vocal clone is a digital extension of your personhood. Rheo ensures that your reference recordings, cloned voice profiles, and generated audio never touch an external server or third-party training set.

• Frictionless native ergonomics. You should never have to open a browser tab, log in, manage credits, and download a .wav file just to hear text spoken. Rheo integrates into your native operating system with global keyboard shortcuts, local database persistence, and system-level audio routing.

• Open weights and open engineering. Proprietary voice models can be modified, censored, or deprecated at a vendor's whim. Rheo anchors itself on open-weights foundation models, providing permanent access to vocal generation tools under the AGPL-3.0 copyleft license.

Real, Verifiable Tech Stack

Every technology listed below is active and running in the production codebase:

System Architecture & Data Flow

+-------------------------------------------------------------------------------+
|                             Operating System Shell                            |
|             (Global Keyboard Chord / Native Application Window)               |
+---------------------------------------+---------------------------------------+
                                        |
                                        v
+-------------------------------------------------------------------------------+
|                             Rheo Desktop Client                               |
|                         (Tauri v2 + Rust Core Engine)                         |
|                                                                               |
|   +-----------------------------------------------------------------------+   |
|   |                         React Workspace UI                            |   |
|   |        (Bento Dashboard • Voice Foundry • Resonance Timeline)         |   |
|   +-----------------------------------+-----------------------------------+   |
+---------------------------------------|---------------------------------------+
                                        | IPC / HTTP (Port 17493)
                                        v
+-------------------------------------------------------------------------------+
|                             Rheo Neural Backend                               |
|                           (FastAPI + Async Python)                            |
|                                                                               |
|   +----------------------------+             +----------------------------+   |
|   |       Model Router         |             |      Dictation Engine      |   |
|   |  (Device & VRAM Topology)  |             |  (Whisper PyTorch / MLX)   |   |
|   +--------------+-------------+             +--------------+-------------+   |
|                  |                                          |                 |
|                  v                                          v                 |
|   +----------------------------+             +----------------------------+   |
|   |    Neural Synthesis Core   |             |         SQLite WAL         |   |
|   |   (Qwen3 / Chatterbox /    |             |    (Profiles & History)    |   |
|   |     TADA / Kokoro / Lux)   |             +----------------------------+   |
|   +--------------+-------------+                            |                 |
|                  |                                          v                 |
|                  v                           +----------------------------+   |
|   +----------------------------+             |         MCP Server         |   |
|   | Tensor Acceleration Engine |             |    (AI Agent Tools)        |   |
|   | (CUDA / ROCm / Metal / CPU)|             +----------------------------+   |
|   +----------------------------+                                              |
+-------------------------------------------------------------------------------+

Core Guarantees

1. Zero External Network Dependencies: Once model weights are cached locally, Rheo can operate in complete air-gapped isolation. No telemetry pings, no remote license validations, and no cloud analytics.

2. Deterministic Audio Pipeline: Audio buffers are processed with 32-bit floating-point precision through customizable effect chains (equalizers, compressors, reverbs, and pitch shifters) before export.

3. Hardware-Aware Memory Management: Models are dynamically loaded and offloaded to prevent VRAM exhaustion. Lightweight models like Kokoro remain memory-resident for instant response, while larger models load on demand.

4. Agent-First Tool Calling: With native Model Context Protocol (MCP) support, external agents running in Claude Desktop, Cursor, or terminal environments can discover voice profiles, generate speech files, and inspect history without human intervention.

5. Portable Voice Identity Packages: Voice profiles can be exported as self-contained .rheo.zip archives containing reference audio, extracted timbre embeddings, paralinguistic configurations, and metadata for seamless transfer between workstations.

Supported Neural Engines & Benchmark Matrix

Rheo incorporates multiple specialized speech synthesis engines, allowing users to select the ideal model for their computational budget and artistic requirements:

Key Capabilities & Studio Workflows

1. Zero-Shot Neural Voice Cloning

Clone any speaking voice with pristine acoustic accuracy from a 3 to 10 second audio recording. Rheo's voice extraction pipeline isolates vocal timbre, harmonic frequencies, and speech pacing while removing background noise, generating a reusable voice profile that can speak any supported language.

2. Multi-Engine Voice Foundry

Freely switch between neural architectures depending on your creative needs:

• Deploy Kokoro for instant real-time conversational feedback and fast proofing.

• Switch to Qwen3-TTS for rich, expressive audiobook narration with complex sentence structures.

• Utilize TADA for theatrical character acting with authentic emotional inflection, laughter, and sighs.

3. Global System-Wide Dictation

Hold your configured keyboard chord from anywhere in your operating system to record your voice. Upon release, Rheo transcribes your speech locally using Whisper and types the formatted, punctuated text directly into your active window (IDE, terminal, browser, or word processor) with zero network latency.

4. Resonance Multi-Track Timeline

Arrange, layer, and edit multi-speaker audio projects within the built-in desktop studio:

• Assemble full podcast episodes, audiobooks, or game dialogue with dedicated speaker tracks.

• Adjust speed, pitch, and timing per segment.

• Apply non-destructive audio effects including compression, parametric EQ, and spatial reverb.

• Export uncompressed 24-bit WAV, MP3, or FLAC formats.

5. Model Context Protocol (MCP) for AI Agents

Rheo natively implements the Model Context Protocol, allowing AI assistants to directly interact with your local voice studio:

• list_profiles: Discover available cloned and preset voice personas.

• generate_speech: Synthesize speech files directly from agent reasoning loops.

• create_story: Automatically assemble multi-character narrative timelines.

• transcribe_audio: Process local audio recordings for automated summarization.

6. Modern Bento Studio Dashboard

Built using modern UI engineering patterns, Rheo's interface provides:

• Stream Matrix: Live status overview of synthesis queues, active tasks, and recent generation history.

• Resonance Calendar: Visual monthly history tracker highlighting daily voice generation activity.

• Priority Streams: Persisted task manager for organizing voice projects and recording sessions.

• Voice Personas Grid: Visual catalog of custom and preset voice clones with instant playback previews.

Installation & Quickstart

System Prerequisites

• Operating System: Windows 10/11 (64-bit), macOS 12.0+ (Apple Silicon or Intel), or Linux (Ubuntu 22.04+, Fedora 38+, Arch Linux)

• Package Manager: Bun >=1.0.0 or Node.js >=20.0

• Python Environment: Python 3.11, 3.12, or 3.13

• Rust Toolchain: Rust 1.75+ with cargo (required for desktop shell compilation)

• Hardware Accelerators (Recommended):

  • NVIDIA: CUDA 12.1 or newer with cuDNN

  • AMD: ROCm 6.0 or newer

  • Apple: Apple Silicon M1/M2/M3/M4 with Metal support

Step-by-Step Setup

# 1. Clone the repository
git clone https://github.com/MasterZ1311/Rheo.git
cd Rheo

# 2. Install frontend and desktop dependencies
bun install

# 3. Create and activate a Python virtual environment
python -m venv .venv
source .venv/bin/activate  # On Windows PowerShell: .venv\Scripts\Activate.ps1

# 4. Install backend neural dependencies
pip install --upgrade pip
pip install -r backend/requirements.txt

# 5. Launch the integrated development desktop studio
bun run dev

Running Components Independently

Run Backend API Server Standalone

uvicorn backend.main:app --host 127.0.0.1 --port 17493 --reload

Run Frontend Web Studio in Browser

cd app
bun run dev

Run Model Context Protocol (MCP) Server for Agents

python -m backend.mcp_server.server

Verification & Testing Suite

Rheo enforces strict automated testing gates across both its frontend and backend codebases:

Run Backend Pytest Suite

python -m pytest backend/tests/ -v

• Status: 164 passed, 10 skipped, 0 failed (100% pass rate)

• Coverage: API routing, voice cloning pipelines, audio format validation, profile management, and database transactions.

Run Frontend Vitest Suite

cd app
bun run test

• Status: 4 test files passed, 9 tests passed (100% pass rate)

• Suites: DashboardHeader.test.tsx, Sidebar.test.tsx, BentoDashboard.test.tsx, UserNameModal.test.tsx.

Run Production Build Verification

# Build frontend web bundles
cd app
bun run build

# Verify desktop native compilation
cd ../tauri/src-tauri
cargo check

Architecture & Engineering Documentation

Deep technical specifications and architectural decision records are maintained in the repository:

• Architecture Deep-Dive: System topology, Rust-Python IPC boundaries, and threading models.

• Neural Engine Specifications: Model weights, tensor requirements, and flow matching mathematical details.

• Voice Cloning Pipeline: Latent embedding extraction, reference normalization, and zero-shot synthesis.

• Global Dictation Architecture: Low-latency keyboard hooks, buffer streaming, and desktop input emulation.

• Audio Effects & DSP Chain: 32-bit floating point processing, parametric EQ, and spatial reverb graphs.

• Model Context Protocol (MCP) Guide: Agent tool schemas, Cursor integration, and stdio communication setup.

• GPU Acceleration & Tuning: CUDA optimization, ROCm driver configurations, and MLX memory tuning.

• Docker Deployment: Headless server mode, containerized GPU pass-through, and web UI serving.

• API Reference Specification: Full OpenAPI 3.1.0 schema documenting all 30+ REST endpoints.

Open Source License & Sovereign Guarantee

Rheo is proudly licensed under the GNU Affero General Public License v3.0 (AGPL-3.0-or-later).

Rheo — Sovereign Local-First AI Voice Studio & Neural Speech Engine
Copyright (C) 2026 MasterZ1311 <thenappanmasterz1311@gmail.com>

This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU Affero General Public License for more details.

You should have received a copy of the GNU Affero General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.