Accelerating Video VAE Decoding via Distilled Quantization
Apply W8A8 quantization and task-specific distillation to the 3D Video VAE decoder to eliminate the dominant decoding latency bottleneck.
Source paper
Jintao Zhang, Kaiwen Zheng, Kai Jiang, Haoxu Wang, Ion Stoica, Joseph E. Gonzalez, Jianfei Chen, Jun Zhu
Generated by AI. The paper’s authors did not write these.
Apply W8A8 quantization and task-specific distillation to the 3D Video VAE decoder to eliminate the dominant decoding latency bottleneck.
Develop a causal Sparse-Linear Attention kernel and apply speculative decoding to enable autoregressive video generation.
Implement temporal-aware INT8 quantization across 3D blocks to eliminate precision errors and flickering artifacts in video generation.
The paper introduces TurboDiffusion, a framework designed to accelerate Wan video diffusion models by 100-200x on NVIDIA RTX 5090 GPUs. It achieves this by combining four optimizations: low-bit attention (SageAttention), sparse-linear attention (SLA), step distillation via regularized Consistency Models (rCM), and W8A8 quantization of linear layers.
The authors integrate a specialized sparse attention mechanism (SLA) with integer quantization (INT8 weights and activations) and algorithmic distillation (reducing sampling steps from ~100 to 3-4). They utilize custom CUDA kernels (SageAttention) and Triton implementations to maximize hardware efficiency on the Blackwell architecture.