June 05, 2024
Diffusion models have recently revolutionized the field of image synthesis due to their ability to generate photorealistic images. However, one of the major drawbacks of diffusion models is that the image generation process is costly. A large image-to-image network has to be applied many times to iteratively refine an image from random noise. While many recent works propose techniques to reduce the number of required steps, they generally treat the underlying denoising network as a black box. In this work, we investigate the behavior of the layers within the network and find that 1) the layers' output changes smoothly over time, 2) the layers show distinct patterns of change, and 3) the change from step to step is often very small. We hypothesize that many layer computations in the denoising network are redundant. Leveraging this, we introduce block caching, in which we reuse outputs from layer blocks of previous steps to speed up inference. Furthermore, we propose a technique to automatically determine caching schedules based on each block's changes over timesteps. In our experiments, we show through FID, human evaluation and qualitative analysis that Block Caching allows to generate images with higher visual quality at the same computational cost. We demonstrate this for different state-of-the-art models (LDM and EMU) and solvers (DDIM and DPM).
Written by
Felix Wimbauer
Bichen Wu
Edgar Schoenfeld
Zijian He
Artsiom Sanakoyeu
Peizhao Zhang
Sam Tsai
Jonas Kohler
Christian Rupprecht
Daniel Cramers
Peter Vajda
Publisher
CVPR
Research Topics
June 20, 2024
Weiyao Wang, Pierre Gleize, Hao Tang, Xingyu Chen, Kevin Liang, Matt Feiszli
June 20, 2024
June 17, 2024
Jiawei Ren, Frost Xu, Jerry Wu, Ziwei Liu, Tao Xiang, Antoine Toisoul
June 17, 2024
June 14, 2024
Abhishek Sureddy, Dishant Padalia, Nandhinee Periyakaruppa, Oindrila Saha, Adina Williams, Adriana Romero Soriano, Megan Richards, Polina Kirichenko, Melissa Hall
June 14, 2024
May 06, 2024
Haoyue Tang, Tian Xie
May 06, 2024
Product experiences
Foundational models
Product experiences
Latest news
Foundational models