Skip to main content
Glama

Nemotron Transcribe MCP

CI License: MIT Python 3.12+ MCP uv

An MCP server for NVIDIA's Nemotron 3.5 Transcribe. It gives an agent tools for transcription, speaker labels on multi-hour recordings, translation, language identification, and questions about audio.

It talks to a vLLM endpoint you already run. Audio stays on your machine or LAN.

You:   Transcribe ~/audio/team-offsite.m4a and label who's talking.
Agent: transcribe(path, speakers=2) -> job id -> get_job_status -> get_job_result

You:   Between minutes 40 and 55, what did they decide about the launch date?
Agent: ask_about_audio(path, question, start=2400, end=3300)

Why a server and not a script

Nemotron 3.5 Transcribe is a language model with speech and speaker encoders attached (34B parameters, about 4.3B active). One chat endpoint handles every task, and the prompt picks which one. Each task needs different prompts, sampling, and token limits, and a request can hold only about an hour of audio. This server handles that so the agent doesn't have to.

Task

Model constraint

What the server does

Transcription

Prompt wording sets the style

Four styles from versioned prompt templates

Speaker labels

Tags up to 8 speakers, but numbering restarts every request

Matches labels across chunks

Long audio

About 1 hour per request

Splits at pauses into ≤55 min pieces, then joins them

Translation

English must be one side

Checks the pair; detects the source language if not given

Language ID

Fixed label set

Samples 10 s of speech, skipping leading silence

Audio Q&A

Reasoning mode, 16,384-token budget

Uses the card's reasoning settings; rejects spans over ~22 min

Related MCP server: transcriber-mcp

Architecture

flowchart LR
  subgraph server["nemo-transcribe-mcp"]
    Tools["Tools"] --> Jobs["Job queue (SQLite)"]
    Jobs --> Planner["ffmpeg + chunking"]
    Planner --> Client["Requests"]
    Client --> Stitcher["Stitching + speaker matching"]
  end
  Agent["MCP client"] -->|"stdio or HTTP"| Tools
  Client -->|"chat/completions"| Model["vLLM: Nemotron 3.5 Transcribe"]

How it works

Long files. Audio is converted to 16 kHz mono PCM. Files over 55 minutes are cut at the longest pause near the end of each window. If there is no pause, the next piece starts 5 seconds early so no words are lost. Repeated words at each join are removed.

Speakers across chunks. The model tells up to eight speakers apart on its own and tags each turn in the transcript:

<spk:0> Okay, let's start with the launch date.
<spk:1> I'd push it a week. QA needs more time.
<spk:0> Agreed. Let's say the fourteenth.

It numbers speakers from 0 on every request, though. With speaker labels on, each piece overlaps the previous one by 10 seconds. The overlapping words show which new label matches which old one. If the match is weak (under 60% agreement) or ambiguous, the transcript gets a [speaker-join-uncertain] marker instead of a guess.

Token limits. Audio costs 12.5 tokens per second, so max_tokens is set to whatever the 65,536-token context has left. If the output is still cut off, that span is split in half and retried.

Jobs. Long work runs in the background. Each tool waits up to wait_seconds (default 25), then returns a job id and a note telling the agent what to do next. Jobs are stored in SQLite. Repeat requests return the cached result, an interrupted job picks up where it stopped when asked again, and results can be read in pages.

Also: stdio and HTTP transports (HTTP uses a bearer token), JSON logs with secrets redacted, init and doctor setup commands, and 112 tests that run without a GPU.

Results

NVFP4 checkpoint on a DGX Spark (GB10), vLLM 0.32.0:

  • A 2 h 47 min podcast with speaker labels took about 18 minutes (about 9× real time).

  • The same sentence from a 5-minute business-news clip:

Call

Output

transcribe, verbatim

That's according to earnings calls this week at both UPS and Fed Ex, some of the biggest players in this space

transcribe, raw_dictation

that's according to earnings calls this week at both ups and fedex some of the biggest players in this space

translate to French

Selon les conférences téléphoniques sur les résultats de cette semaine chez UPS et FedEx, deux des acteurs les plus importants dans ce secteur

identify_language

en_us

Tools

Tool

Inputs

Returns

transcribe

path, speakers (0 or 2–8), style

Transcript or job id

translate

path, target_language, optional source_language

Translation or job id

ask_about_audio

path, question, optional start/end

Answer or job id

identify_language

path, optional candidates

Label such as en_us

list_languages

Language matrix

get_job_status

job_id

Status

get_job_result

job_id, optional offset/limit

Result text

list_jobs

optional state

Recent jobs

cancel_job

job_id

Status

path can be a local file or a YouTube link. Any speakers value from 2 to 8 turns on speaker matching across pieces; the model finds the actual speakers itself.

Styles: verbatim (default) keeps fillers. clean drops fillers and repeats. technical keeps emails, URLs, paths, and IDs exact. raw_dictation has no punctuation and spells out numbers.

Setup

You need a running Nemotron 3.5 Transcribe endpoint (see the model card for vllm serve). Settings go in .env; .env.example documents each one.

Local (stdio)

Requires Python 3.12+, uv, and ffmpeg.

uv sync
cp .env.example .env               # set NEMOTRON_BASE_URL
uv run nemo-transcribe-mcp doctor  # checks endpoint, model, ffmpeg

Cursor config, run from this directory:

{
  "mcpServers": {
    "nemo-transcribe": { "command": "uv", "args": ["run", "nemo-transcribe-mcp"] }
  }
}

Docker (HTTP)

cp .env.example .env   # set NEMOTRON_BASE_URL and MCP_AUTH_TOKEN
docker compose up -d --build
curl -s http://127.0.0.1:8083/healthz

The MCP endpoint is http://127.0.0.1:8083/mcp with Authorization: Bearer <MCP_AUTH_TOKEN>. Put audio in ./audio and pass it as /data/audio/<file>. For a model on the same host, use host.docker.internal in NEMOTRON_BASE_URL, not localhost.

Hermes example (~/.hermes/config.yaml):

mcp_servers:
  nemo-transcribe:
    url: "http://127.0.0.1:8083/mcp"
    headers:
      Authorization: "Bearer YOUR_TOKEN_FROM_ENV"
    timeout: 300

Languages

The model card rates 40 languages. Norwegian is also listed below; the card tested it but didn't rate it. Any of them can be transcribed. Translation needs English on one side.

Leading

Language

Code

Translate

Czech

🇨🇿

cs

✅

Dutch

🇳🇱

nl

✅

English

🇺🇸

en

✅

French

🇫🇷

fr

✅

German

🇩🇪

de

✅

Hindi

🇮🇳

hi

✅

Italian

🇮🇹

it

✅

Korean

🇰🇷

ko

✅

Polish

🇵🇱

pl

✅

Portuguese

🇧🇷

pt

✅

Russian

🇷🇺

ru

✅

Slovak

🇸🇰

sk

✅

Spanish

🇪🇸

es

✅

Production-ready

Language

Code

Translate

Bulgarian

🇧🇬

bg

✅

Croatian

🇭🇷

hr

✅

Danish

🇩🇰

da

✅

Estonian

🇪🇪

et

✅

Finnish

🇫🇮

fi

✅

Hungarian

🇭🇺

hu

✅

Japanese

🇯🇵

ja

✅

Latvian

🇱🇻

lv

✅

Lithuanian

🇱🇹

lt

✅

Romanian

🇷🇴

ro

✅

Slovenian

🇸🇮

sl

✅

Swedish

🇸🇪

sv

✅

Thai

🇹🇭

th

✅

Turkish

🇹🇷

tr

❌

Ukrainian

🇺🇦

uk

✅

Supported (may need fine-tuning)

Language

Code

Translate

Arabic

🇪🇬

ar

✅

Bengali

🇧🇩

bn

❌

Chinese

🇨🇳

zh

✅

Greek

🇬🇷

el

✅

Hebrew

🇮🇱

he

✅

Indonesian

🇮🇩

id

❌

Maltese

🇲🇹

mt

✅

Marathi

🇮🇳

mr

❌

Norwegian

🇳🇴

no

❌

Tagalog

🇵🇭

tl

❌

Tamil

🇮🇳

ta

❌

Telugu

🇮🇳

te

❌

Vietnamese

🇻🇳

vi

❌

Language ID is weaker for Latvian and Slovenian and unavailable for Bengali, Marathi, Norwegian, Tamil, and Telugu. Audio Q&A was only evaluated in English.

Limitations

  • No word timestamps yet, so no SRT/VTT. The model can return per-word times and diarization segments, but this server doesn't request them yet.

  • Not yet exposed: vocabulary hints, nearest-speaker-only transcription, PII tagging.

  • Inputs are local files and YouTube links.

  • One job runs at a time.

Development

uv run pytest
uv run ruff check . && uv run ruff format --check .
npm install && npm run check:yaml

Tests need ffmpeg but not a model. CI runs the same checks on every pull request.

Available Tools

9 tools
ask_about_audioA

Answer a question about one audio file or one YouTube link. The model was checked on English speech. A recording longer than the reasoning budget is refused. Pass start and end, in seconds, for the part you want. show_reasoning includes the model's reasoning before the answer. Waits wait_seconds (default 25). If it is still running, returns a job id. Tell the user, then call get_job_status. Do not poll in a loop.

Inputs: path (required), question (required), end, start, +2 more.

cancel_jobA

Stop a transcription job. Pieces that already finished stay readable through get_job_result.

Inputs: job_id (required).

get_job_resultB

The words from a finished job. offset and limit return a slice so a long transcript does not flood the chat. Only text is stored.

Inputs: job_id (required), limit, format, offset.

get_job_statusA

How far a transcription job has got. Pass wait_seconds from 0 to 45 to wait for it to finish. Tell the user the status. Do not poll in a loop.

Inputs: job_id (required), wait_seconds.

identify_languageA

Say which language is spoken in one audio file or one YouTube link. Returns one label, such as en_us. Listens to about 10 seconds of speech, skipping a pause at the start. Less reliable for Latvian and Slovenian. Not a transcript.

Inputs: path (required), label_set, candidates, clip_seconds, +1 more.

list_jobsB

Jobs on this machine, newest first. Use this to recover a job id.

Inputs: limit, state.

list_languagesA

Languages this model can handle, and how strong that support is. level is leading, production_ready, or supported. translate is yes when that language can be one side of an English translation. identify is yes, low, or no. Latvian and Slovenian are low. Norwegian is listed, but the card only scored transcription. It did not mark it supported.

Inputs: no named parameters.

transcribeA

Speech to text for one audio file or one YouTube link. style verbatim, the default, keeps every word, including fillers. style clean drops fillers and accidental repetitions and keeps the other words. style technical keeps emails, URLs, file paths, commands, and IDs exactly. Do not pass speakers with style technical. style raw_dictation drops punctuation and writes numbers as spoken words. A long file is cut at a pause and sent one piece at a time. Pass speakers from 2 to 8 to repeat 10 seconds at each join. Speaker numbers are matched across those joins. Waits wait_seconds (default 25). If it is still running, returns a job id. Tell the user, then call get_job_status. Do not poll in a loop.

Inputs: path (required), style, speakers, wait_seconds.

translateA

Write the speech in one audio file, or one YouTube link, in another language. source_language is the language spoken in the file, such as Polish or pl. target_language is the language the words should be written in, such as English or en. Pass both when you know them. A language passed here overrides NEMOTRON_SOURCE_LANGUAGE and NEMOTRON_TARGET_LANGUAGE. If source_language is empty, the language heard in the file overrides NEMOTRON_SOURCE_LANGUAGE. Leave target_language empty to use NEMOTRON_TARGET_LANGUAGE. One of the two languages must be English. A long file is cut at a pause and sent one piece at a time. Waits wait_seconds (default 25). If it is still running, returns a job id. Tell the user, then call get_job_status. Do not poll in a loop.

Inputs: path (required), wait_seconds, source_language, target_language.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    TDQS

    A3.8/5.0

    Scored across 9 tools

    Disambiguation5/5

    Each tool has a clearly distinct action: transcribe, translate, ask_about_audio, identify_language, and list_languages operate on audio but produce different outputs, while get_job_result, get_job_status, list_jobs, and cancel_job cover distinct job-management roles. The descriptions make the boundaries explicit, so an agent should not confuse them.

    Naming Consistency4/5

    All names use snake_case and start with a verb, with get_job_result, get_job_status, list_languages, list_jobs, cancel_job, ask_about_audio, and identify_language following a verb_noun pattern. transcribe and translate are bare verbs rather than verb_noun, a minor inconsistency.

    Tool Count5/5

    Nine tools is a well-scoped set for transcription, translation, audio QA, language identification, and job management. Each tool has a clear role and none feels redundant.

    Completeness4/5

    The surface covers the core lifecycle: submit jobs (transcribe/translate), inspect status, retrieve results, list/cancel jobs, and query language support. Minor gaps include no job deletion or audio metadata tool, but these are not blocking.

    Maintenance

    ActivityMaintained
    ResponsivenessNo recent community issues

    Related MCP Connectors

    Related MCP Servers

    • A
      license
      Not graded
      quality
      C
      maintenance
      Enables local transcription of audio/video files and YouTube URLs, generation of SRT/VTT subtitles, and analysis of speech pacing and audience retention risk using faster-whisper.
      MIT
    • A
      license
      Not graded
      quality
      B
      maintenance
      Enables AI clients to transcribe YouTube links and uploaded audio or video files, then retrieve the resulting transcripts as text, SRT, VTT, or JSON with timestamps and optional speaker labels. Also supports paging through long transcripts, checking upload progress, listing or deleting jobs, and viewing remaining account minutes.
      126 npm
      1
      MIT