local-whisper-mcp

English | Français

Local speech-to-text service, in a single process: the FastAPI REST API and the MCP Streamable HTTP server share the same JobManager, the same job queue and the same Whisper model cache.

No cloud API calls. Audio is downloaded, transcribed, and the job metadata is kept on local disk.

Copyright (C) 2026 Laurent Lemercier. Licensed under the GNU AGPL v3.0.

Contents

• Features

• Requirements

• Installation

• Configuration

• Getting started

• REST API

• MCP tools

• Models

• Supported audio formats

• Docker

• Tests

• Security

• Project layout

• Known limitations

• License

Related MCP server: faster-whisper-mcp

Features

• Asynchronous submission: every transcription returns a job_id immediately. The REST call responds 202 Accepted; the client then polls for state.

• Three audio sources: remote URL, multipart upload, or Base64.

• Two interfaces: REST and MCP Streamable HTTP, on the same port, with no second process.

• Webhooks: when a job ends (success or failure), the service POSTs the job state to a callback URL.

• Transcription metrics: audio duration, processing duration, and real-time factor (real_time_factor).

• SSRF protection: download and callback URLs are validated, including on every redirect.

• On-disk job persistence, readable again after a restart.

Requirements

• Python 3.11 or later.

• FFmpeg is only required for the Docker containers. Locally, audio duration is determined by PyAV (a bundled native library) and, as a fallback, by Mutagen — no external binary is installed.

• Disk space: model weights are downloaded on first use (roughly 500 MB for small, roughly 1.5 GB for medium).

Installation

git clone https://github.com/laurentlemercier/local-whisper-mcp.git
cd local-whisper-mcp

python -m venv .venv

# Windows (PowerShell)
.venv\Scripts\Activate.ps1

# Linux / macOS
source .venv/bin/activate

pip install -r requirements.txt

Configuration

Every variable is read from the process environment at startup (app/config.py). The application does not load a .env file: the .env.example in this repository serves as reference only and is not applied automatically.

Getting started

# Set a token, then start
export API_TOKEN="a-long-random-token"       # Linux / macOS
$env:API_TOKEN="a-long-random-token"        # Windows PowerShell

uvicorn app.main:app --host 0.0.0.0 --port 8000

• REST: http://localhost:8000

• MCP: http://localhost:8000/mcp

• Interactive OpenAPI documentation: http://localhost:8000/docs

Why uvicorn app.main:app instead of python -m app.run? app/run.py calls mount_mcp(app), which mounts the MCP server on /mcp a second time even though app/main.py has already mounted it. The second mount is unreachable: it creates an unused session manager. Going through uvicorn app.main:app avoids the duplication. See Known limitations.

For development with automatic reload:

uvicorn app.main:app --reload

REST API

Every route except /health requires the Authorization: Bearer <API_TOKEN> header.

Submit a URL

curl -X POST http://localhost:8000/v1/transcriptions \
  -H "Authorization: Bearer $API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
        "source": { "type": "url", "url": "https://example.com/audio.m4a" },
        "model": "small",
        "language": "en"
      }'

{ "id": "tr_3f9a1c2b8e7d4051", "status": "queued", "model": "small", ... }

Submit a file

curl -X POST http://localhost:8000/v1/transcriptions/upload \
  -H "Authorization: Bearer $API_TOKEN" \
  -F "file=@./audio.m4a" \
  -F "model=small" \
  -F "language=en"

Fetch the result

curl -H "Authorization: Bearer $API_TOKEN" \
  http://localhost:8000/v1/transcriptions/tr_3f9a1c2b8e7d4051/result

409 while the job is unfinished, 500 if the job failed, 200 once status is completed.

Possible states are queued, downloading, running, completed and failed.

Job completion webhook

Add callback_url (and callback_headers if needed) to the request. Once the transcription finishes, successfully or not, the service performs a JSON POST with the job state. callback_url and callback_headers are excluded from the body sent.

MCP tools

MCP Streamable HTTP server on /mcp, named local-whisper-speech-to-text.

Every tool takes an optional api_token parameter, to be filled with the same value as API_TOKEN.

Example MCP client configuration:

{
  "mcpServers": {
    "local-whisper": {
      "type": "http",
      "url": "http://localhost:8000/mcp",
      "headers": { "Authorization": "Bearer a-long-random-token" }
    }
  }
}

Models

Two models are accepted: small (default) and medium.

They are loaded on demand at first use and kept in memory for the lifetime of the process. Loading medium needs roughly three times the memory of small; on CPU, int8 is the right compromise. language: "auto" lets Whisper detect the language.

Supported audio formats

.m4a .mp3 .wav .ogg .opus .webm .flac

The extension is checked on receipt. For an upload, a name without a valid extension is rejected; for a URL or Base64 payload, the extension is derived from the path and, failing that, from the response MIME type.

Docker

The build context is the repository root; the image only contains requirements.txt and app/ (see .dockerignore).

cp docker/.env.example docker/.env

# Fill in DATA_PATH, MODELS_PATH, CURRENT_UID and CURRENT_GID in docker/.env
docker compose -f docker/docker-compose.service.yml up -d

docker/.env.example documents the expected variables. In particular:

• CURRENT_UID / CURRENT_GID stop the container from writing to DATA_PATH as root.

• MODELS_PATH acts as a persistent HuggingFace cache, so weights are not re-downloaded on every recreate.

The image is based on python:3.11-slim pinned by digest, with ffmpeg installed. Its entrypoint is uvicorn app.main:app, so it does not run the duplicate mount described above.

Tests

The tests are manual and require a running service as well as a local audio file.

cd tests

.\test-speech-service-v2.3.ps1 `
    -ApiToken "a-long-random-token" `
    -AudioFile "C:\paths\to\my\audio.m4a"

The script chains: /health probe, Bearer authentication, REST upload, state polling until completion, result retrieval, then discovery and invocation of the MCP tools. It deletes the jobs it created, unless -KeepJobs is passed.

It also automatically creates a virtual environment in tests/.venv and installs the official MCP SDK there — which is why that directory is ignored by git. Pass -PythonExe to choose the interpreter, -SkipMcp or -SkipRestUpload to narrow the scope. Without -AudioFile, the REST and MCP tests are skipped with a warning.

tests/mcp_client.py is a standalone MCP client, usable on its own:

python tests/mcp_client.py \
    --endpoint http://localhost:8000/mcp \
    --api-token "$API_TOKEN" \
    --audio-file ./audio.m4a \
    --model small \
    --language en

Security

Authentication

If API_TOKEN is empty, the service requires no authentication and checks none. Do not expose it to a network in that state. REST uses the Authorization: Bearer header; MCP tools expect the api_token parameter.

SSRF protection

Download and callback URLs go through the same validator (app/security.py), which rejects:

• any scheme other than http and https;

• credentials embedded in the URL (https://user:pass@host/...);

• hosts absent from the allow-list, when ALLOWED_URL_HOSTS is set;

• localhost;

• any IP address the host resolves to that is private, loopback, link-local, reserved, multicast or unspecified.

Redirects are followed manually, at most six times, and every target is revalidated before being requested. Size is checked against the Content-Length header and then against the body actually received.

To fetch audio from an internal network, set ALLOW_PRIVATE_URLS=true — and prefer a host allow-list over that relaxation.

Network exposure

The service provides neither TLS nor rate limiting. Put it behind a reverse proxy that terminates TLS, and restrict submissions to trusted clients: transcription is expensive in CPU time.

Project layout

app/
  config.py     Environment variables and data directories
  models.py     Pydantic schemas (Job, Source, Result, Statuses)
  security.py   Bearer token and SSRF validation
  source.py     URL download, Base64, extension checking
  jobs.py       JobManager: queue, JSON persistence, locks
  whisper.py    Model cache and faster-whisper invocation
  worker.py     Processing thread: download, transcription, webhook
  mcp.py        MCP server and its five tools
  main.py       FastAPI application, REST routes, MCP mount
  run.py        Alternative entrypoint (see Known limitations)
data/
  input/        Downloaded or uploaded audio, one directory per job
  jobs/         State of each job as JSON
docker/         Dockerfile and compose file for a containerised deployment
tests/          PowerShell test script and Python MCP client

A job is laid out as follows:

data/jobs/tr_3f9a1c2b8e7d4051.json          state, metrics, result
data/input/tr_3f9a1c2b8e7d4051/audio.m4a    source audio
data/input/tr_3f9a1c2b8e7d4051/source.json  original URL, if remote source

Known limitations

• Single-threaded queue. Only one job is transcribed at a time. The queue lives in memory: it does not survive a restart, and an in-flight job is lost.

• Duplicate MCP mount. app/main.py:549 mounts the MCP server on /mcp, then app/run.py:5 calls mount_mcp(app), which mounts it again. Only the first mount is reachable; the second creates an orphaned session manager. Use uvicorn app.main:app.

• Dead variables in Docker. MODEL and LANGUAGE are set in the Dockerfile and the compose file, but the application never reads them: model and language are chosen per request. The whisper_models volume declared in the compose file is not used either.

• No rate limiting or quota. An authorised client can saturate the queue.

• No automatic expiry. Jobs and their audio occupy disk until explicitly deleted.

• No CI. The tests require a live service and a model weight download, which makes them a poor fit for continuous integration without a cache.

License

GNU Affero General Public License v3.0. The full text is in LICENSE.

AGPL v3.0 has a network clause (section 13): if you modify this software and make it available to users who access it over a network, you must offer those users the source code of your modified version. This is the essential difference from the classic GPL, which only covers distribution.

Copyright (C) 2026 Laurent Lemercier.