Information-Theoretic Quantification of CFG-Induced Path Collapse
Implement a spectral analysis of the generator Jacobian and use Fisher Information metrics to track entropy reduction and latent manifold collapse during guided distillation.
Source paper
Dongyang Liu, Peng Gao, David Liu, Ruoyi Du, Zhen Li, Qilong Wu, Xin Jin, Sihan Cao, Shifeng Zhang, Hongsheng Li, Steven Hoi
Generated by AI. The paper’s authors did not write these.
Implement a spectral analysis of the generator Jacobian and use Fisher Information metrics to track entropy reduction and latent manifold collapse during guided distillation.
Replace standard GAN discriminators with a multi-scale feature-matching objective using frozen vision encoders to provide a stable adversarial regularization shield.
Implement an active learning-based sampler that monitors gradient magnitudes across noise levels to dynamically adjust re-noising schedules during multi-step distillation.
This paper reveals that the success of Distribution Matching Distillation (DMD) in few-step generation is driven by two decoupled mechanisms: CFG Augmentation (CA), which acts as the 'engine' for conversion, and Distribution Matching (DM), which acts as a stabilizing 'regularizer'. By identifying this division of labor, the authors propose a Decoupled-Hybrid re-noising schedule that focuses the CA engine on unresolved noise levels while keeping the DM regularizer global, leading to state-of-the-art results in 4-step and 8-step image synthesis.
The researchers perform a mathematical decomposition of the DMD gradient into two distinct terms: a CA term (the difference between conditional and unconditional scores) and a DM term (the difference between real teacher and auxiliary student scores). This decomposition allows them to implement independent re-noising noise schedules (tau) for each component, optimizing the distillation process by tailoring the noise level at which each 'force' is applied to the student model.