Skip to content

Source paper

Moonshine: Speech Recognition for Live Transcription and Voice Commands

Nat Jeffries, Evan King, Manjunath Kudlur and 3 more

Nat Jeffries, Evan King, Manjunath Kudlur, Guy Nicholson, James Wang, Pete Warden

arXiv 2410.15608 · 2024

Research ideas from this paper (4)

Generated by AI. The paper’s authors did not write these.

AI summary

This paper introduces Moonshine, a family of lightweight speech recognition models designed to overcome the fixed-latency limitations of OpenAI's Whisper by using variable-length encoding. By replacing absolute position embeddings with Rotary Position Embeddings (RoPE) and eliminating zero-padding, Moonshine achieves up to 5x faster inference on short audio segments while maintaining comparable Word Error Rates (WER) to Whisper tiny.en.

How it works (AI summary)

The authors utilize an Encoder-Decoder Transformer architecture modified with Rotary Position Embeddings (RoPE) to handle variable input lengths without padding. The model is trained on a massive composite dataset of 200,000 hours, comprising open ASR datasets and web-scraped speech, which was pseudo-labeled using Whisper Large v3 and filtered for hallucinations based on log-probability thresholds.