Mitigating Repetition Loops in Sub-Second Utterances
Fine-tune speech models on short audio datasets and apply a repetition penalty during decoding to eliminate repetition loops on sub-second utterances.
Source paper
Nat Jeffries, Evan King, Manjunath Kudlur, Guy Nicholson, James Wang, Pete Warden
Generated by AI. The paper’s authors did not write these.
Fine-tune speech models on short audio datasets and apply a repetition penalty during decoding to eliminate repetition loops on sub-second utterances.
Pass past decoder key-value caches across sequential segments to maintain conversational state and context during continuous speech transcription.
Replace AdamW with a second-order SOAP optimizer to accelerate ASR model training convergence and improve generalization across variable-length constraints.
Extend maximum sequence length capabilities and reduce hallucinations by fine-tuning models on progressively longer audio clips using a curriculum learning strategy.
This paper introduces Moonshine, a family of lightweight speech recognition models designed to overcome the fixed-latency limitations of OpenAI's Whisper by using variable-length encoding. By replacing absolute position embeddings with Rotary Position Embeddings (RoPE) and eliminating zero-padding, Moonshine achieves up to 5x faster inference on short audio segments while maintaining comparable Word Error Rates (WER) to Whisper tiny.en.
The authors utilize an Encoder-Decoder Transformer architecture modified with Rotary Position Embeddings (RoPE) to handle variable input lengths without padding. The model is trained on a massive composite dataset of 200,000 hours, comprising open ASR datasets and web-scraped speech, which was pseudo-labeled using Whisper Large v3 and filtered for hallucinations based on log-probability thresholds.