Nvidia Debuts Nemotron 3.5 Transcribe for Multilingual Audio and Diarization
Nvidia has launched Nemotron 3.5 Transcribe, an open 30B speech model handling transcription, translation, and speaker identification with 4B active parameters.
Nvidia has introduced Nemotron 3.5 Transcribe, an open-weight speech recognition model designed to process natural language instructions alongside raw audio. Rather than requiring distinct components for separate speech tasks, the system merges transcription, translation, speaker identification, and audio-based question answering into a single architecture. It can process audio recordings lasting up to one hour and containing up to eight distinct speakers across 40 languages.
The model accepts simple text instructions to dictate its output. Given a single audio file, developers can prompt the system to generate a transcript with speaker tags, output an English translation, or summarize key conclusions. Prompts can also supply domain-specific vocabulary—such as pharmaceutical names—as hints, which the model uses only if it detects them in the speech.
High Accuracy on Modest Hardware
Nemotron 3.5 Transcribe features 30 billion total parameters, but its architecture activates only about 4 billion parameters per step. Because of this sparse activation, the model can run on a single Nvidia GPU, ranging from the DGX Spark to high-end enterprise hardware. When tested on an RTX PRO 6000 Blackwell Server Edition using NVFP4 precision, it transcribed 100 hours of recorded audio in under three minutes.
Independent evaluations demonstrate strong performance against existing open solutions. On Hugging Face's English Open ASR Leaderboard, the model ranks first among open-weight systems, posting a 5.33 percent average word error rate (WER). In benchmark tests conducted by Artificial Analysis, Transcribe achieved a 2.81 percent WER, performing just behind Mistral's Voxtral Small (2.77 percent) while utilizing roughly one-sixth of the active parameters. According to Nvidia, its results also surpass several commercial speech services as well as open offerings from IBM and Alibaba.
The model demonstrated notable strength in handling multi-speaker overlap. In evaluations by VoiceArena, which tests conversations featuring simultaneous talk, Transcribe achieved the lowest word omission rate at 8.32 percent, well below the benchmark median of approximately 31 percent. It also took the top spot on David AI's benchmark, which tracks speaker attribution across 21 languages.
Granary v2 Dataset and Domain Customization
To accompany the release, Nvidia published the Granary v2 dataset, which comprises around 844,000 hours of speech across 48 languages—an expansion from the 643,000 hours in 25 languages found in the initial version. The audio samples are provided in multiple formats, including verbatim transcripts containing filler words and cleaned-up versions that substitute written numbers with digits. The dataset creation tooling has also been made public.
Nvidia demonstrated the model's domain adaptation using recordings of Indian medical students and doctors. Using specific prompting alone, the baseline error rate decreased from 30.8 percent to 23.8 percent. Following targeted fine-tuning on mostly synthetic data—a process that took approximately 14 GPU hours on an Nvidia GB300—the error rate fell to 11.8 percent, outperforming a dedicated open medical speech model that scored 13.79 percent. However, Nvidia noted trade-offs: fine-tuning slightly degraded performance on standard English speech, and general accuracy on specific brand-name pharmaceuticals remained higher in the specialized medical baseline.
What it means for developers
For engineering teams handling post-call processing, meeting summaries, and multilingual analysis, Nemotron 3.5 Transcribe consolidates multiple pipeline stages. Previously, teams running Nvidia software needed separate systems like Nemotron 3 Diarization alongside transcription engines such as Parakeet. Combining diarization, translation across 31 languages, and transcription into one prompt-driven model simplifies operational infrastructure.
However, teams should be aware of several practical constraints:
- Batch processing only: Unlike Mistral's Voxtral 3, which targets real-time voice agents, Nemotron 3.5 Transcribe is restricted to pre-recorded audio files.
- Language coverage limits: While transcription covers 40 languages, translation to or from English is limited to 31, speaker recognition functions across 13, and audio question answering was trained purely in English.
- Human oversight required: Nvidia advises spot-checking outputs for speaker attribution, specific entity names, and answers, as well as holding back at least 100 verified audio samples when evaluating fine-tuned models.
As developer stacks integrate complex audio processing alongside broader language and reasoning workloads, managing model access remains critical. Developers exploring broader multimodal setups can test and integrate top models like Claude, GPT, Gemini, and DeepSeek cheaply through one API at https://apixoai.online.
Nemotron 3.5 Transcribe, Granary v2, and related fine-tuning guides are currently available on Hugging Face.
Source: Nvidia's open Nemotron 3.5 Transcribe combines transcription, translation, and speaker recognition — The Decoder. Written by the Apixo team from that report.
One key for Claude, GPT, GLM, DeepSeek and more. Pay per token with crypto.
Get your API key
