Nvidia Ships Open Transcription Model That Runs on One Chip

Nvidia released Nemotron 3.5 Transcribe on Wednesday, an open-weight speech model it says handles 40 languages and recordings up to an hour long, all served from a single GPU. The model card lists a 34-billion-parameter mixture-of-experts design that activates roughly 4.3 billion parameters per token, with BF16, FP8 and NVFP4 builds published under the OpenMDW-1.1 license.

The release arrives alongside Granary-v2, Nvidia's updated open speech dataset, and slots into the same NeMo Speech toolchain the company already ships for fine-tuning and serving.

What Nvidia actually shipped

This is not just an ASR checkpoint. Nemotron 3.5 Transcribe is a speech-augmented language model: audio and a text instruction go in, text comes out. That means the same weights handle transcription, speaker-attributed transcription with up to eight speakers separated, speech translation between 31 languages and English, language identification, word-level timestamps, audio question answering and PII tagging for redaction.

Nvidia also ships a model card full of numbers. Against the Open ASR Leaderboard's nine public English sets it reports a mean word error rate of 4.71, ahead of Qwen3-ASR-1.7B at 4.94. On long-form audio it claims 10.98 against Parakeet-TDT-0.6B-v3 at 13.37, and on multilingual evaluation 4.62 against 6.49. Multi-speaker sessions land at a mean cpWER of 15.80 versus 16.83 for the best published pipeline.

Why it matters

Open weights change the deployment conversation. A team that wants meeting or clinical transcription can now run the model inside its own perimeter instead of streaming audio to a vendor API, and the reference deployment is one H100 with 80 GB, with quantized variants cutting that further. Nvidia explicitly points at the Jetson AGX Thor robotics controller and DGX Spark as viable hosts, which is a hint about where it expects voice workloads to land next: on the device, not in the cloud.

The timing is deliberate. Mistral shipped Voxtral 3 this month, ElevenLabs launched Realtime for voice agents, and Salesforce released Launa-Omni. Voice is the current competitive front, and Nvidia's play is to own the substrate rather than the product.

The catch

Transcribe works on recorded audio, not live streams. For a real-time voice agent you still need a streaming pipeline or a different model, so this is a batch transcription engine first. The benchmarks are also Nvidia's own, captured on October 12 and published by Nvidia, so independent leaderboard runs are the thing to watch.

What builders should do

Treat this as the cheap default for post-call processing: recording in, structured transcript out, with speaker labels and timestamps already in the response. Fine-tune on your own jargon through NeMo Speech, quantize to FP8 or NVFP4, and keep a streaming model alongside it for anything conversational. The interesting part is not the error rate. It is that a 34B speech model now fits on hardware you can put in a closet.