Continuous-Time Distribution Matching for Few-Step Diffusion Distillation
Researchers have introduced Continuous-Time Distribution Matching (CDM), a novel technique designed to accelerate diffusion models while maintaining high visual fidelity. Traditional step distillation methods, such as vanilla Distribution Matching Distillation (DMD), often suffer from visual artifacts and over-smoothed outputs due to their reliance on sparse supervision at discrete timesteps. CDM addresses these limitations by migrating the framework from discrete anchoring to continuous optimization. The method employs two key designs: a dynamic continuous schedule of random length that enforces distribution matching at arbitrary points along sampling trajectories, and a continuous-time alignment objective that performs active off-trajectory matching using the student's velocity field. Extensive experiments conducted on architectures like SD3-Medium and Longcat-Image demonstrate that CDM achieves competitive visual quality in few-step image generation without requiring complex auxiliary modules such as GANs or reward models. This advancement represents a significant improvement in efficiency and output quality for generative AI models, offering a streamlined approach to diffusion distillation. The associated code has been made publicly available to facilitate further research and application in the field of computer vision and artificial intelligence.
Wire timeline
Continuous-Time Distribution Matching for Few-Step Diffusion Distillation
Researchers have introduced Continuous-Time Distribution Matching (CDM), a novel technique designed to accelerate diffusion models while maintaining high visual fidelity. Traditional step distillation methods, such as vanilla Distribution Matching Distillation (DMD), often suffer from visual artifacts and over-smoothed outputs due to their reliance on sparse supervision at discrete timesteps. CDM addresses these limitations by migrating the framework from discrete anchoring to continuous optimization. The method employs two key designs: a dynamic continuous schedule of random length that enforces distribution matching at arbitrary points along sampling trajectories, and a continuous-time alignment objective that performs active off-trajectory matching using the student's velocity field. Extensive experiments conducted on architectures like SD3-Medium and Longcat-Image demonstrate that CDM achieves competitive visual quality in few-step image generation without requiring complex auxiliary modules such as GANs or reward models. This advancement represents a significant improvement in efficiency and output quality for generative AI models, offering a streamlined approach to diffusion distillation. The associated code has been made publicly available to facilitate further research and application in the field of computer vision and artificial intelligence.
cs.AI updates on arXiv.org