Diffusion models (DMs) have attracted attention in generative modeling due to their ability to produce high-quality, diverse outputs by progressively adding noise to data and then denoising it. However, DMs are computationally intensive due to their iterative nature, requiring numerous forward passes and high-precision operations, making them less efficient for resource-constrained environments. Recent efforts to reduce these computational demands using quantization show promise by converting high-precision parameters to lower precision, but they face challenges unique to DMs, particularly in addressing cross-timestep error propagation in the iterative process. In this paper, we analyze cross-timestep error propagation in quantized DMs, revealing that previous methods focusing only on reducing noise estimation discrepancies are insufficient. Instead, we introduce Cross-Timestep Error Correction (CTEC), where the quantized model not only approximates the full-precision model but also corrects errors from the previous timestep. A distillation method is applied to learn this correction process effectively. We conduct extensive experiments on unconditional image generation with LSUN-Churches and LSUN-Bedrooms, as well as conditional image generation with ImageNet. Our findings demonstrate the effectiveness of our method in significantly reducing accumulated quantization errors across timesteps within the quantized diffusion process. This enhancement enables the generation of high-quality images, even when constrained by reduced bitwidths.

Deep Neural Networks (DNNs) have demonstrated remarkable accuracy in vision classification tasks. However, they exhibit vulnerability to additional noises known as adversarial attacks. Previous studies hypothesize that this vulnerability might stem from the fact that high-accuracy DNNs heavily rely on irrelevant and non-robust features, such as textures and the background. In this work, we reveal that edge information extracted from images can provide relevant and robust features related to shapes and the foreground. These features assist pretrained DNNs in achieving improved adversarial robustness without compromising their accuracy on clean images. A lightweight and plug-and-play EdgeNet is proposed, which can be seamlessly integrated into existing pretrained DNNs, including Vision Transformers, a recent family of state-of-the-art models for vision classification. Our EdgeNet can process edges derived from either clean nature images or noisy adversarial images, yielding robust features which can be injected into the intermediate layers of the frozen backbone DNNs. The cost of obtaining such edges using conventional edge detection algorithms (e.g., Canny edge detector) is marginal, and the cost of training the EdgeNet is equivalent to that of fine-tuning the backbone network with techniques such as Adapter.

Forgery of facial images and videos has increased the concern about digital security. It has led to the significant development of detecting forgery data recently. However, the data, especially the videos published on the Internet, are usually compressed with lossy compression algorithms such as H.264. The compressed data could significantly degrade the performance of recent detection algorithms. The existing anti-compression algorithms focus on enhancing the performance in detecting heavily compressed data but less consider the compression adaption to the data from various compression levels. We believe creating a forgery detection capable of handling data compressed with unknown levels is important. To enhance the performance of such models, we consider the weak compressed and strong compressed data as two views of the original data and they should have similar representation and relationships with other samples. We propose a novel anti-compression forgery detection framework by maintaining closer relations within data under different compression levels. Specifically, our algorithm measures the pair-wise similarity within data as the relations, ensuring that relationships between weakly and strongly compressed data remain consistent. This enhances the discriminative power for detecting highly compressed data. To achieve a better strong compressed data relation guided by the less compressed one, we apply video-level contrastive learning for weak compressed data, which forces the model to produce similar representations within the same video and far from the negative samples. The experiment results show that the proposed algorithm could boost performance for strong compressed data while improving the accuracy rate when detecting clean data.

Recently, text-to-image models have been thriving. Despite their powerful generative capacity, our research has uncovered a lack of robustness in this generation process. Specifically, the introduction of small perturbations to the text prompts can result in the blending of primary subjects with other categories or their complete disappearance in the generated images. In this paper, we propose **Auto-attack on Text-to-image Models (ATM)**, a gradient-based approach, to effectively and efficiently generate such perturbations. By learning a Gumbel Softmax distribution, we can make the discrete process of word replacement or extension continuous, thus ensuring the differentiability of the perturbation generation. Once the distribution is learned, ATM can sample multiple attack samples simultaneously. These attack samples can prevent the generative model from generating the desired subjects without tampering with the category keywords in the prompt. ATM has achieved a 91.1\% success rate in short-text attacks and an 81.2\% success rate in long-text attacks. Further empirical analysis revealed three attack patterns based on: 1) variability in generation speed, 2) similarity of coarse-grained characteristics, and 3) polysemy of words. The code is available at https://github.com/duchengbin8/Stable_Diffusion_is_Unstable