Diffusion models have become the dominant approach for generative tasks in images, videos, and other domains. However, diverse data properties in generation requests, which are critical for efficient serving, remain underexploited. To address this issue, we propose a diffusion model serving system ChituDiffusion. ChituDiffusion leverages the locality of data properties to recompose a diffusion pipeline into dGraphs with shared optimization opportunities, enabling thorough compile-time and runtime co-optimizations. During compilation, ChituDiffusion compiles each dGraph into multiple execution engines optimized for specific data properties. At runtime, heterogeneous requests are elaborately reorganized into fine-grained batching tasks with similar properties and then efficiently executed by matched engines. Evaluation on five diffusion applications shows that ChituDiffusion improves the throughput by up to 2.13× (1.58× on average) on A100 and 2.19× (1.51× on average) on H100 compared with existing frameworks. The code for ChituDiffusion and the production traces have been made open-source at https://github.com/thu-pacman/chitu/tree/Diffusion.

Convolutional Neural Networks (CNNs) have achieved remarkable performance at a huge computational cost. By improving the model sparsity, it can effectively reduce the complexity. However, with deepening of sparsity, the problems of unbalanced workloads, computing fragmentation and mapping access conflict caused by irregular sparsity have become more and more remarkable. These problems pose great challenges for efficient computation of sparse CNN s. In order to make full use of two side of sparsity introduced by activations and weights, and overcome the above problems, this paper proposes an efficient sparse CNN s accelerator on FPGA to achieve the inference acceleration. We designed and implemented the accelerator on the Zynq UltraScale+ MPSoC ZCU102 evaluation board. By running AlexNet, VGG16 and ResNet50 networks on the accelerator to evaluated the peeformance. Experimental results show that the method proposed in this paper can achieve more than 97% reduction in collision rate and 2.35x improvement in computing performance and 9.37x improvement in energy efficiency.