# Qwen3-ASR Simple API Server (Port 1212)

A fast, lightweight, and agentic-ready Speech-to-Text API server powered by **[Qwen/Qwen3-ASR-1.7B-hf](https://huggingface.co/Qwen/Qwen3-ASR-1.7B-hf)**.

Built with **FastAPI**, **Uvicorn**, and **Transformers**, this server provides real-time and batch transcription with chunking, line-level timestamps, automated logging, native **OpenAI API compatibility**, and **Model Context Protocol (MCP)** tool calling for AI agents.

---

## Highlights & Features

- **Default Port `1212`**: Accessible locally at `http://localhost:1212` and across LAN (`0.0.0.0`).
- **Local Model Checkpoint**: Automatically downloads weights to `./checkpoint/Qwen3-ASR-1.7B-hf` on first launch.
- **Video File Support (FFmpeg Audio Extraction)**: Accepts popular video formats (`.mp4`, `.mkv`, `.mov`, `.avi`, `.webm`, `.flv`, `.wmv`, `.m4v`, `.ts`) and extracts 16kHz audio streams on-the-fly with minimal memory overhead.
- **Auto-Installer Prompt at Launch**: Detects if FFmpeg is installed at launch. If found, enables video transcription immediately without asking. If missing, prompts the user to install with a single keystroke via `winget install ffmpeg` (Windows), `brew install ffmpeg` (macOS), or Linux package managers; if declined, the server proceeds smoothly in audio-only mode.
- **Configurable Video Mode**: Video support can be toggled ON or OFF at runtime via `/api/config`, CLI flags (`--enable-video` / `--disable-video`), or directly from the Web Playground Settings tab.
- **Chunked Processing for Efficiency**: Long audio and video files are split into overlapping chunks (e.g. 30s chunks with 5s stride) to prevent GPU out-of-memory (OOM) errors, allowing podcasts, lectures, and hour-long videos to run smoothly on consumer GPUs.
- **Full Text in One Call**: Audio processed in chunks under the hood is stitched back together into a single, cohesive response.
- **Line & Segment Timestamps**: Returns `start` and `end` times for each segment/line, both in numeric seconds and formatted `HH:MM:SS.mmm`.
- **Subtitles & Export Formats**: Supports `json`, `verbose_json`, `srt` (SubRip), `vtt` (WebVTT), and `text`.
- **Configurable Logging in `.logs/`**:
  - **Default**: Saves text transcripts (`transcript.txt` and `metadata.json`) into `.logs/`, but does **not** save audio to conserve disk space.
  - **Customizable**: Audio and text logging can be toggled globally via environment variables or per-request in API calls.
- **OpenAI-Compatible Endpoint (`/v1/audio/transcriptions`)**: Drop-in replacement for Whisper. Works out-of-the-box with the official OpenAI Python/Node SDKs, LangChain, LlamaIndex, AutoGen, and CrewAI for both audio and video files.
- **Model Context Protocol (MCP) Integration**: Includes `mcp_server.py` to seamlessly connect with Claude Desktop, Cursor, Antigravity, and any MCP client.
- **Multi-Modal Audio & Video Inputs**: Accepts media via multipart form upload, local file path (zero-copy for local agents), base64 string, or remote HTTP URL.
- **Live SSE Streaming (`/api/transcribe/stream`)**: Streams chunked transcription events in real time.
- **One-Click Launchers**:
  - `run.bat` for Windows
  - `run.sh` for Linux & macOS
  - Automatically initializes `.venv` with `--system-site-packages`, installs dependencies, verifies/prompts for FFmpeg, downloads checkpoint if missing, and starts the server.

---

## Quick Start (One-Click)

### Windows
Double-click `run.bat` or run in terminal:
```cmd
run.bat
```
*(If FFmpeg is not found, you will be prompted: `Do you want to install FFmpeg using 'winget install ffmpeg' (Y/N)?`. Answering `Y` automatically installs it.)*

### Linux / macOS
Make executable and run:
```bash
chmod +x run.sh
./run.sh
```
*(If FFmpeg is missing, `run.sh` will prompt to install via Homebrew on macOS or your distro's package manager on Linux.)*

### Manual Run
```bash
# 1. Activate virtual environment
# Windows:
.venv\Scripts\activate
# Linux / macOS:
source .venv/bin/activate

# 2. (Optional) Download model checkpoint explicitly
python download_model.py

# 3. Start server on port 1212 (video enabled by default if FFmpeg is present)
python server.py --port 1212

# Optional video flags:
# Force disable video mode:
python server.py --port 1212 --disable-video
# Force enable video mode:
python server.py --port 1212 --enable-video
```

---

## FFmpeg Installation & Video Support Guide

Video transcription requires FFmpeg to extract audio tracks from container files (`.mp4`, `.mkv`, `.mov`, `.avi`, `.webm`, etc.).

### Automatic Installation (At Launch)
When starting the server via `run.bat`, `run.sh`, or `python server.py`:
- **If FFmpeg is already installed**: The server automatically detects it, sets video conversion mode to **ON**, and starts immediately without prompting.
- **If FFmpeg is not found**: The terminal prompts:
  ```text
  ======================================================================
  [NOTICE] FFmpeg was not found on your system.
  Video file processing (MP4, MKV, MOV, WebM, etc.) requires FFmpeg
  to extract audio for transcription.
  ======================================================================
  Do you want to install FFmpeg using '<platform-command>'? (Y/N):
  ```
  - Selecting **`Y` (Yes)** runs the installation command and enables video conversion upon completion.
  - Selecting **`N` (No)** sets video conversion to **OFF** and starts the server in audio-only mode.

### Manual Installation by Operating System

#### Windows
**Option 1: Windows Package Manager (Recommended)**
```cmd
winget install ffmpeg
```
*(or `winget install Gyan.FFmpeg`)*

**Option 2: Chocolatey**
```cmd
choco install ffmpeg
```

**Option 3: Scoop**
```cmd
scoop install ffmpeg
```

**Option 4: Manual ZIP**
1. Download a release build from [gyan.dev/ffmpeg/builds](https://www.gyan.dev/ffmpeg/builds/).
2. Extract to `C:\ffmpeg`.
3. Add `C:\ffmpeg\bin` to your system `PATH`.

---

#### macOS
**Option 1: Homebrew (Recommended)**
```bash
brew install ffmpeg
```

**Option 2: MacPorts**
```bash
sudo port install ffmpeg
```

---

#### Linux

**Ubuntu / Debian / Linux Mint:**
```bash
sudo apt update && sudo apt install -y ffmpeg
```

**Fedora / RHEL / CentOS:**
```bash
sudo dnf install -y ffmpeg
```

**Arch Linux / Manjaro:**
```bash
sudo pacman -S --noconfirm ffmpeg
```

**openSUSE:**
```bash
sudo zypper install ffmpeg
```

---

### Configuring Video Mode (Runtime & API)

Video conversion mode can be toggled on or off at any time:

1. **Web Playground / Dashboard**:
   Open [http://localhost:1212/docs](http://localhost:1212/docs) -> Click **Settings & Defaults** -> Toggle **Video Conversion (FFmpeg Audio Extraction)** -> Click **Save & Sync Server Defaults**.
2. **API Endpoint (`POST /api/config`)**:
   ```bash
   # Enable video mode
   curl -X POST http://localhost:1212/api/config \
     -H "Content-Type: application/json" \
     -d '{"enable_video": true}'

   # Disable video mode (audio-only)
   curl -X POST http://localhost:1212/api/config \
     -H "Content-Type: application/json" \
     -d '{"enable_video": false}'
   ```
3. **CLI Arguments**:
   ```bash
   python server.py --enable-video
   python server.py --disable-video
   ```

Once started:
- **Interactive Web Console & Testing Studio**: [http://localhost:1212/docs](http://localhost:1212/docs) (or [http://localhost:1212](http://localhost:1212))
- **Swagger UI Fallback**: [http://localhost:1212/swagger](http://localhost:1212/swagger)
- **ReDoc Documentation**: [http://localhost:1212/redoc](http://localhost:1212/redoc)
- **Health Check & Device Info**: [http://localhost:1212/health](http://localhost:1212/health)
- **Runtime Configuration**: [http://localhost:1212/api/config](http://localhost:1212/api/config)

---

## Agentic AI Integration Guide

### 1. Using with OpenAI Python SDK (Drop-in Replacement)
Point any tool-calling agent to `http://localhost:1212/v1`:

```python
from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:1212/v1",
    api_key="not-needed"
)

with open("audio.wav", "rb") as f:
    transcript = client.audio.transcriptions.create(
        model="qwen3-asr",
        file=f,
        response_format="verbose_json"
    )

print("Full Text:", transcript.text)
for segment in transcript.segments:
    print(f"[{segment['start']}s -> {segment['end']}s]: {segment['text']}")
```

### 2. Using Model Context Protocol (MCP) in Claude Desktop / Antigravity
Add the following to your `claude_desktop_config.json` or Antigravity MCP settings:

```json
{
  "mcpServers": {
    "qwen3-asr": {
      "command": "h:/PROJECTS/PYTHON/Qwen3-ASR-simple-API/.venv/Scripts/python.exe",
      "args": [
        "h:/PROJECTS/PYTHON/Qwen3-ASR-simple-API/mcp_server.py"
      ]
    }
  }
}
```

The MCP server exposes:
- `transcribe_audio(audio_path_or_url, language, chunk_length_s, return_timestamps)`
- `get_server_status()`

### 3. Native OpenAI Tool Calling Schema
To provide tool calling to an LLM agent, query `GET http://localhost:1212/api/tools/schema` or use this definition:

```json
{
  "type": "function",
  "function": {
    "name": "transcribe_audio",
    "description": "Transcribe speech from an audio file or URL into text using local Qwen3-ASR with line timestamps and chunking.",
    "parameters": {
      "type": "object",
      "properties": {
        "audio_path": {
          "type": "string",
          "description": "Absolute path to local audio file on disk (e.g. 'C:/recordings/meeting.wav')."
        },
        "audio_url": {
          "type": "string",
          "description": "Public URL to audio stream or file."
        },
        "chunk_length_s": {
          "type": "number",
          "description": "Duration of each processing chunk in seconds. Default is 30.0.",
          "default": 30.0
        },
        "return_timestamps": {
          "type": "boolean",
          "description": "Return start and end timestamps for each line/segment.",
          "default": true
        },
        "language": {
          "type": "string",
          "description": "Language code hint (e.g. 'en', 'zh', 'es', 'fr', 'de'). Auto-detects if omitted."
        },
        "output_format": {
          "type": "string",
          "enum": ["json", "verbose_json", "srt", "vtt", "text"],
          "default": "json"
        }
      },
      "required": []
    }
  }
}
```

---

## API Reference & Endpoints

### 1. `POST /api/transcribe` (JSON Body)
Best for AI Agents transmitting file paths, URLs, or base64 strings without multipart HTTP complexity.

**Request Payload:**
```json
{
  "audio_path": "H:/audio/sample.mp3",
  "chunk_length_s": 30.0,
  "stride_length_s": 5.0,
  "return_timestamps": true,
  "language": "en",
  "output_format": "json",
  "save_text": true,
  "save_audio": false
}
```

**Response (`200 OK`):**
```json
{
  "text": "Welcome to today's episode where we explore automated speech recognition.",
  "segments": [
    {
      "id": 0,
      "start": 0.0,
      "end": 3.45,
      "start_time": "00:00:00.000",
      "end_time": "00:00:03.450",
      "text": "Welcome to today's episode"
    },
    {
      "id": 1,
      "start": 3.50,
      "end": 6.82,
      "start_time": "00:00:03.500",
      "end_time": "00:00:06.820",
      "text": "where we explore automated speech recognition."
    }
  ],
  "duration": 6.82,
  "latency_seconds": 0.42,
  "language": "en"
}
```

---

### 2. `POST /api/transcribe/upload` (Multipart File Upload)
Best for web dashboards or direct file uploads.

**cURL Example:**
```bash
curl -X POST "http://localhost:1212/api/transcribe/upload" \
  -F "file=@sample.wav" \
  -F "chunk_length_s=30" \
  -F "return_timestamps=true" \
  -F "output_format=srt"
```

**SRT Output Example:**
```
1
00:00:00,000 --> 00:00:03,450
Welcome to today's episode

2
00:00:03,500 --> 00:00:06,820
where we explore automated speech recognition.
```

---

### 3. `POST /v1/audio/transcriptions` (OpenAI Drop-In)
Compatible with the OpenAI Whisper transcription schema.

**cURL Example:**
```bash
curl -X POST "http://localhost:1212/v1/audio/transcriptions" \
  -H "Content-Type: multipart/form-data" \
  -F "file=@sample.wav" \
  -F "model=qwen3-asr" \
  -F "response_format=verbose_json"
```

---

### 4. `POST /api/transcribe/stream` (Live Server-Sent Events)
Streams chunk-by-chunk transcription as each segment completes.

**Example SSE Events:**
```
data: {"event": "chunk", "chunk_id": 0, "start": 0.0, "end": 10.0, "text": "Hello world", "progress": 0.33}

data: {"event": "chunk", "chunk_id": 1, "start": 8.0, "end": 20.0, "text": "This is real-time transcription.", "progress": 0.66}

data: {"event": "done", "full_text": "Hello world This is real-time transcription.", "total_chunks": 2, "duration": 20.0}
```

---

### 5. `GET /health` & `GET /api/status`
Returns service status, loaded device (GPU/CPU), and VRAM utilization.

```json
{
  "status": "healthy",
  "model_id": "Qwen/Qwen3-ASR-1.7B-hf",
  "checkpoint_dir": "H:\\PROJECTS\\PYTHON\\Qwen3-ASR-simple-API\\checkpoint\\Qwen3-ASR-1.7B-hf",
  "is_model_loaded": true,
  "device_info": {
    "device": "cuda:0",
    "device_name": "NVIDIA GeForce RTX 4070 Ti SUPER",
    "dtype": "torch.bfloat16",
    "vram_total_mb": 12282.0,
    "vram_allocated_mb": 3410.5
  },
  "default_logging": {
    "save_text": true,
    "save_audio": false,
    "log_directory": "H:\\PROJECTS\\PYTHON\\Qwen3-ASR-simple-API\\.logs"
  }
}
```

---

## Parameter Reference Guide

| Parameter | Type | Default | Description |
| :--- | :--- | :--- | :--- |
| `audio_path` | `string` | `null` | Local filepath on server machine (fastest, zero-copy). |
| `audio_url` | `string` | `null` | Remote URL to audio file (server downloads and transcribes). |
| `audio_base64` | `string` | `null` | Base64 encoded audio string. |
| `chunk_length_s` | `float` | `30.0` | Chunk size in seconds for memory efficiency. Prevents GPU OOM. Set to `null` or `0` to disable chunking. |
| `stride_length_s` | `float` | `5.0` | Overlap duration between chunks to avoid cutting words at boundaries. |
| `return_timestamps`| `bool` | `true` | When true, returns start and end timestamps for each segment/line. |
| `language` | `string` | `null` | Optional language code hint (`en`, `zh`, etc.). Auto-detects if omitted. |
| `output_format` | `string` | `json` | Format: `json`, `verbose_json`, `srt`, `vtt`, or `text`. |
| `save_text` | `bool` | `true` | Saves transcript and metadata to `.logs/`. |
| `save_audio` | `bool` | `false`| Saves input audio to `.logs/audio.wav`. Disabled by default to save disk space. |

---

## Logging System (`.logs/`)

Transcriptions are organized into the `.logs/` folder:

```
.logs/
  ├── transcriptions.jsonl       <-- Summary log (one JSON line per request)
  └── 20260915_123000_uuid/
      ├── transcript.txt         <-- Full text transcript
      ├── metadata.json          <-- Request metadata, timings, segments, and parameters
      └── audio.wav              <-- Only saved if save_audio=True
```

### Environment Variables
You can configure defaults in `.env` or system environment:
- `PORT=1212`
- `HOST=0.0.0.0`
- `SAVE_LOG_TEXT=true`
- `SAVE_LOG_AUDIO=false`
- `LOG_DIR=./.logs`

---

## Testing & Verification

Run the automated test suite to verify the server:
```bash
# Start the server in one terminal:
run.bat

# In another terminal, run test client:
.venv\Scripts\python.exe test_client.py
```