Diffusion-based personalization brings convenience to users in text-to-image generation but it also poses risks of rights infringement and content misuse. To address this issue, researchers have proposed several proactive defense methods by adversarial attacks. However, most of these methods directly attack the noise prediction results during the fine-tuning process, overlooking the unique characteristics of diffusion-based personalization, which results in limited defense performance. Therefore, this paper summarizes the two core tasks of personalized fine-tuning as identity aggregation and identity binding, and proposes a defense method named IdentityGuard to specifically attack these two core tasks. The IdentityGuard designs a training sample decorrelation (TSD) attack and a text-image decoupling (TID) attack respectively for the two core tasks. The TSD attack disrupts the learning of common features by reducing the correlations among training samples. The TID attack targets all tokens by using the value-inverted attention map of each token as its adaptive target, aiming to suppress high-attention regions and strengthen low-attention regions. In addition, a token-level adaptive weighting strategy is designed to dynamically allocate attack weights across different tokens during fine-tuning. Experimental results demonstrate that the IdentityGuard effectively enhances proactive defense performance against diffusion-based personalization, achieving an average improvement of 22.26% in terms of Identity Score Matching (ISM) metric compared to the state-of-the-art (SOTA) methods. The source code is available at https://github.com/imagecbj/IdentityGuard.

The advent of deep learning has accelerated the development of facial manipulation techniques, particularly face-swapping and face attribute editing, raising serious concerns about privacy and identity-related misuse. Existing proactive defense methods predominantly target attribute editing and often generalize poorly to face-swapping models, making it difficult to provide effective protection across both tasks within a unified framework. To bridge this gap, we propose a generalized defense framework, Epoch-Adaptive Adversarial Perturbation Optimization (EA-APO). Specifically, EA-APO introduces a proactive defense mechanism that establishes optimal adversarial paths by optimizing perturbations on a white-box surrogate model to enhance adversarial transferability, and applies the resulting perturbations to source face images to disrupt both face swapping and face attribute editing, even against previously unseen target models in black-box settings. This approach mitigates identity feature tampering while adapting to changes in visual attributes and preserving high-quality adversarial examples. Experimental results show the generalization of our method across multiple face-swapping and attribute-editing models, including commercial ones, while also maintaining strong defense under various common post-processing operations and real-world social media transmission conditions, underscoring its potential for real-world deployment.

The rapid advancement of Generative Adversarial Networks (GANs) and diffusion models has enabled the creation of highly realistic synthetic images, presenting significant societal risks, such as misinformation and deception. As a result, detecting AI-generated images has emerged as a critical challenge. Existing research emphasizes extracting fine-grained features to enhance detector generalization, yet they often lack consideration for the importance and interdependencies of internal elements within local regions and are limited to a single frequency domain, hindering the capture of general forgery traces. To overcome the aforementioned limitations, we first utilize a sliding window to restrict the attention mechanism to a local window, and reconstruct the features within the window to model the relationships between neighboring internal elements within the local region. Then, we design a dual frequency domain branch framework consisting of four frequency domain subbands of DWT and the phase part of FFT to enrich the extraction of local forgery features from different perspectives. Through feature enrichment of dual frequency domain branches and fine-grained feature extraction of reconstruction sliding window attention, our method achieves superior generalization detection capabilities on both GAN and diffusion model-based generative images. Evaluated on diverse datasets comprising images from 65 distinct generative models, our approach achieves a 2.13% improvement in detection accuracy over state-of-the-art methods.

Image steganalysis is a detection task to distinguish whether a secret message is embedded in a digital image. Due to the domain inconsistency caused by Cover Source Mismatch(CSM) and Steganographic Algorithm Mismatch (SAM), most of them suffer from significant performance degradation. Recent mismatched steganalysis focused on extracting domain invariant features by domain adversarial training or feature alignment. However these schemes are limited to unstable performance in diverse domain mismatch scenarios, and are even ineffective in some cases. In this paper, we propose a Universal Mismatched Steganalysis PCD-UMS via pair-wise confidence difference-based pseudo-label selection from the perspective of optimizing target training data. Specifically, we reveal a strong positive correlation commonality between pair-wise confidence difference and the detection performance of steganalysis among various mismatch scenarios. Based on this, a novel pseudo-label selection strategy consisting of maximum confidence difference first (MCDF) rule and pair-wise label differential storage (PLDS) rule is designed to select and filter the reliable target pseudo-labels. Furthermore, a multi-perspective pair-wise feature alignment loss is designed to initially transfer the classification ability of source steganalysis, thus solving the problem that source steganalysis fails completely under some domain mismatch scenarios. Comprehensive experiments show that our PCD-UMS outperforms the existing mismatched steganalysis by 12.07% and 3.40% in terms of detection performance under CSM and SAM scenarios.

Proactive defense against face forgery seeks to disrupt the output of forgery models by embedding imperceptible adversarial perturbations into face images to be protected. However, existing methods predominantly focus on deepfakes, often neglecting traditional image manipulations. It limits their practical applicability, as attackers may resort to traditional manipulations when deepfake attempts fail. To bridge this gap, a Dual-Forgery Proactive Defense (DFPD) method is proposed for combating both deepfakes and traditional image manipulations. For deepfake resistance, the DFPD designs a gradient-based ensemble adversarial attack that effectively disrupts outputs from multiple deepfake models. To defeat traditional manipulations, it also designs a fragile watermarking algorithm based on Invertible Neural Network (INN), enabling accurate localization of tampered regions. Furthermore, to mitigate the mutual interference between perturbation injection and watermark embedding, on the one hand, the DFPD adopts a serial pipeline starting with watermark embedding and then perturbation injection, which ensures that the injected perturbations are not displaced into residual image during INN-based embedding. On the other hand, a morphological post-processing module is introduced to eliminate adversarial noise in the tampering localization results. Extensive experiments validate the effectiveness of DFPD, demonstrating a 20.25% improvement in deepfake disruption over the best baseline in terms of PSNR and a 9.67% increase in traditional tampering localization in terms of ACC, while preserving high perceptual quality (32.75 dB PSNR).

Forged videos are often subjected to double compression. When a forger maliciously or unintentionally increases the video's bitrate during re-encoding, the resulting videos are termed fake bitrate videos. Detecting these videos offers a generalized approach for efficiently identifying potentially forged content within large datasets. However, previous research has largely focused on video-level detection of fully fake bitrate videos, where an entire video is re-encoded at a higher bitrate after content modification or the creation of fake high-definition (HD) footage. In practice, a skilled forger may adjust the bitrate of only specific video segments, generating partial fake bitrate videos-a common manipulation in tampering processes like video splicing. Existing methods face difficulties in detecting such partial modifications at the frame level and in pinpointing the manipulated segments. Our study addresses this gap by introducing a novel frame-level detection approach, which significantly enhances forensic precision. We simultaneously account for two types of abnormal frames arising from re-encoding and bitrate escalation and, for the first time, define fake bitrate video detection as a triple classification problem. To meet the challenges of this task, we extract anomalous bitrate-compression traces that capture subtle differences among the three frame types. Additionally, we propose the Trident Transformer Network (TTNet), a model designed to effectively integrate and learn high-frequency information within the encoding domain. Our approach achieves substantial improvements in accuracy, surpassing state-of-the-art methods by 3.62% and 11.95% in video-level and frame-level detection scenarios, respectively.

The rapid advancement of generative models, such as GANs and Diffusion models, has enabled the creation of highly realistic synthetic images, raising serious concerns about misinformation, deepfakes, and copyright infringement. Although numerous Artificial Intelligence Generated Image (AIGI) detectors have been proposed, often reporting high accuracy, their effectiveness in real-world scenarios remains questionable. To bridge this gap, we introduce AIGIBench, a comprehensive benchmark designed to rigorously evaluate the robustness and generalization capabilities of state-of-the-art AIGI detectors. AIGIBench simulates real-world challenges through four core tasks: multi-source generalization, robustness to image degradation, sensitivity to data augmentation, and impact of test-time pre-processing. It includes 23 diverse fake image subsets that span both advanced and widely adopted image generation techniques, along with real-world samples collected from social media and AI art platforms. Extensive experiments on 11 advanced detectors demonstrate that, despite their high reported accuracy in controlled settings, these detectors suffer significant performance drops on real-world data, limited benefits from common augmentations, and nuanced effects of pre-processing, highlighting the need for more robust detection strategies. By providing a unified and realistic evaluation framework, AIGIBench offers valuable insights to guide future research toward dependable and generalizable AIGI detection.

Few-shot image generation aims to train generative models using a small number of training images. When there are few images available for training (e.g. 10 images), Learning From Scratch (LFS) methods often generate images that closely resemble the training data while Transfer Learning (TL) methods try to improve performance by leveraging prior knowledge from GANs pre-trained on large-scale datasets. However, current TL methods may not allow for sufficient control over the degree of knowledge preservation from the source model, making them unsuitable for setups where the source and target domains are not closely related. To address this, we propose a novel pipeline called Peer is your Pillar (PIP), which combines a target few-shot dataset with a peer dataset to create a data-unbalanced conditional generation. Our approach includes a class embedding method that separates the class space from the latent space, and we use a direction loss based on pre-trained CLIP to improve image diversity. Experiments on various few-shot datasets demonstrate the advancement of the proposed PIP, especially reduces the training requirements of few-shot image generation.

Deep Neural Networks (DNNs) have achieved remarkable success across a wide range of tasks; however, their susceptibility to backdoor attacks remains a significant concern. Existing methods predominantly focus on optimizing the construction phase of backdoor attacks, aiming to reduce the detectability of trigger patterns and enhance stealth. In contrast, the selection phase—specifically the identification of appropriate benign samples for poisoning—has received limited attention. Recent studies have explored efficient poisoning sample selection to improve attack stealth. However, the underlying factors that determine the informativeness or effectiveness of a sample for backdoor learning remain poorly understood. To address this gap, we investigate the role of forgettable event and loss landscape curvature in enhancing poisoning sample efficiency. Our findings reveal that samples most likely to be forgotten during the poisoning process are crucial for effective attacks, and that low-curvature regions of the loss surface correlate with higher poisoning efficiency. Based on these insights, we introduce the Improved Filtering and Updating Strategy (FUS++), which significantly outperforms traditional selection methods in terms of efficiency. Our contributions provide new perspectives on sample selection for backdoor attacks and propose a novel strategy to improve poisoning efficacy.

Recent deep neural networks (DNNs) have came to rely on vast amounts of training data, providing an opportunity for malicious attackers to exploit and contaminate the data to carry out backdoor attacks. However, existing backdoor attack methods make unrealistic assumptions, assuming that all training data comes from a single source and that attackers have full access to the training data. In this paper, we introduce a more realistic attack scenario where victims collect data from multiple sources, and attackers cannot access the complete training data. We refer to this scenario as $\textbf{data-constrained backdoor attacks}$. In such cases, previous attack methods suffer from severe efficiency degradation due to the $\textbf{entanglement}$ between benign and poisoning features during the backdoor injection process. To tackle this problem, we introduce three CLIP-based technologies from two distinct streams: $\textit{Clean Feature Suppression}$ and $\textit{Poisoning Feature Augmentation}$. The results demonstrate remarkable improvements, with some settings achieving over $\textbf{100}$% improvement compared to existing attacks in data-constrained scenarios.

Poisoning efficiency is crucial in poisoning-based backdoor attacks, as attackers aim to minimize the number of poisoning samples while maximizing attack efficacy. Recent studies have sought to enhance poisoning efficiency by selecting effective samples. However, these studies typically rely on a proxy backdoor injection task to identify an efficient set of poisoning samples. This proxy attack-based approach can lead to performance degradation if the proxy attack settings differ from those of the actual victims, due to the shortcut nature of backdoor learning. Furthermore, proxy attack-based methods are extremely time-consuming, as they require numerous complete backdoor injection processes for sample selection. To address these concerns, we present a Proxy attack-Free Strategy (PFS) designed to identify efficient poisoning samples based on the similarity between clean samples and their corresponding poisoning samples, as well as the diversity of the poisoning set. The proposed PFS is motivated by the observation that selecting samples with high similarity between clean and corresponding poisoning samples results in significantly higher attack success rates compared to using samples with low similarity. Additionally, we provide theoretical foundations to explain the proposed PFS. We comprehensively evaluate the proposed strategy across various datasets, triggers, poisoning rates, architectures, and training hyperparameters. Our experimental results demonstrate that PFS enhances backdoor attack efficiency while also offering a remarkable speed advantage over previous proxy attack-based selection methodologies.

In this study, we delve into the task of few-shot Generative Domain Adaptation (GDA), which involves transferring a pre-trained generator from one domain to a new domain using only a few reference images. Inspired by the way human brains acquire knowledge in new domains, we present an innovative generator structure called $\textbf{Domain Re-Modulation (DoRM)}$. DoRM not only meets the criteria of $\textit{high quality}$, $\textit{large synthesis diversity}$, and $\textit{cross-domain consistency}$, which were achieved by previous research in GDA, but also incorporates $\textit{memory}$ and $\textit{domain association}$, akin to how human brains operate. Specifically, DoRM freezes the source generator and introduces new mapping and affine modules (M\&A modules) to capture the attributes of the target domain during GDA. This process resembles the formation of new synapses in human brains. Consequently, a linearly combinable domain shift occurs in the style space. By incorporating multiple new M\&A modules, the generator gains the capability to perform high-fidelity multi-domain and hybrid-domain generation. Moreover, to maintain cross-domain consistency more effectively, we introduce a similarity-based structure loss. This loss aligns the auto-correlation map of the target image with its corresponding auto-correlation map of the source image during training. Through extensive experiments, we demonstrate the superior performance of our DoRM and similarity-based structure loss in few-shot GDA, both quantitatively and qualitatively. Code will be available at https://github.com/wuyi2020/DoRM.

While Deep Neural Networks (DNNs) excel in many tasks, the huge training resources they require become an obstacle for practitioners to develop their own models. It has become common to collect data from the Internet or hire a third party to train models. Unfortunately, recent studies have shown that these operations provide a viable pathway for maliciously injecting hidden backdoors into DNNs. Several defense methods have been developed to detect malicious samples, with the common assumption that the latent representations of benign and malicious samples extracted by the infected model exhibit different distributions. However, it is still an open question whether this assumption holds up. In this article, we investigate such differences thoroughly via answering three questions: 1) What are the characteristics of the distributional differences? 2) How can they be effectively reduced? 3) What impact does this reduction have on difference-based defense methods? First, the distributional differences of multi-level representations on the regularly trained backdoored models are verified to be significant by adopting Maximum Mean Discrepancy (MMD), Energy Distance (ED), and Sliced Wasserstein Distance (SWD) as the metrics. Then, ML-MMDR, a difference reduction method that adds multi-level MMD regularization into the loss, is proposed, and its effectiveness is testified on three typical difference-based defense methods. Across all the experimental settings, the F1 scores of these methods drop from 90%-100% on the regularly trained backdoored models to 60%-70% on the models trained with ML-MMDR. These results indicate that the proposed MMD regularization can enhance the stealthiness of existing backdoor attack methods. The prototype code of our method is now available athttps://github.com/xpf/Multi-Level-MMD-Regularization.

Generative Adversarial Networks (GANs) have the ability to generate images that are visually indistinguishable from real images. However, recent studies have revealed that generated and real images share significant differences in the frequency domain. In this article, we argue that the frequency gap is caused by the high-frequency sensitivity of the discriminator. According to our observation, during the training of most GANs, severe high-frequency differences make the discriminator focus on high-frequency components excessively, which hinders the generator from fitting the low-frequency components that are important for learning images’ content. Then, we propose two simple yet effective image pre-processing operations in the frequency domain for eliminating the side effects caused by high-frequency differences in GANs training: High-frequency Confusion (HFC) and High-frequency Filter (HFF). The proposed operations are general and can be applied to most existing GANs at a fraction of the cost. The advanced performance of the proposed operations is verified on multiple loss functions, network architectures, and datasets. Specifically, the proposed HFF achieves significant improvements of 42.5% FID on CelebA (128*128) unconditional generation based on SNGAN, 30.2% FID on CelebA unconditional generation based on SSGAN, and 69.3% FID on CelebA unconditional generation based on InfoMAXGAN. Furthermore, we also adopt HFF as the first attempt at data augmentation in the frequency domain for contrastive learning, achieving state-of-the-art performance on unconditional generation. Code is available at https://github.com/iceli1007/HFC-and-HFF .

Recent studies have proven that deep neural networks are vulnerable to backdoor attacks. Specifically, by mixing a small number of poisoned samples into the training set, the behavior of the trained model can be maliciously controlled. Existing attack methods construct such adversaries by randomly selecting some clean data from the benign set and then embedding a trigger into them. However, this selection strategy ignores the fact that each poisoned sample contributes inequally to the backdoor injection, which reduces the efficiency of poisoning. In this paper, we formulate improving the poisoned data efficiency by the selection as an optimization problem and propose a Filtering-and-Updating Strategy (FUS) to solve it. The experimental results on CIFAR-10 and ImageNet-10 indicate that the proposed method is effective: the same attack success rate can be achieved with only 47% to 75% of the poisoned sample volume compared to the random selection strategy. More importantly, the adversaries selected according to one setting can generalize well to other settings, exhibiting strong transferability. The prototype code of our method is now available at https://github.com/xpf/Data-Efficient-Backdoor-Attacks.

Generative adversarial nets (GANs) have been successfully applied in many fields like image generation, inpainting, super-resolution, and drug discovery, etc. By now, the inner process of GANs is far from being understood. To get a deeper insight into the intrinsic mechanism of GANs, in this paper, a method for interpreting the latent space of GANs by analyzing the correlation between latent variables and the corresponding semantic contents in generated images is proposed. Unlike previous methods that focus on dissecting models via feature visualization, the emphasis of this work is put on the variables in latent space, i.e. how the latent variables affect the quantitative analysis of generated results. Given a pre-trained GAN model with weights fixed, the latent variables are intervened to analyze their effect on the semantic content in generated images. A set of controlling latent variables can be derived for specific content generation, and the controllable semantic content manipulation is achieved. The proposed method is testified on the datasets Fashion-MNIST and UT Zappos50K, experiment results show its effectiveness.

Automatic medical image segmentation has wide applications for disease diagnosing. However, it is much more challenging than natural optical image segmentation due to the high-resolution of medical images and the corresponding huge computation cost. The sliding window is a commonly used technique for whole slide image (WSI) segmentation, however, for these methods based on the sliding window, the main drawback is lacking global contextual information for supervision. In this paper, we propose a dual-inputs attention network (denoted as DA-RefineNet) for WSI segmentation, where both local fine-grained information and global coarse information can be efficiently utilized. Sufficient comparative experiments are conducted to evaluate the effectiveness of the proposed method, the results prove that the proposed method can achieve better performance on WSI segmentation compared to methods relying on single-input.