Attributing synthetic images to their source generative models is critical for digital forensics and security. While most existing attribution methods can distinguish images produced by known models and reject those from unknown ones, they are unable to verify whether a given image was produced by a specific, previously unseen model. To address this limitation, we formulate an open-set verification problem: determining whether a given image was generated by a specific model. Our key insight is that synthetic images from different models show consistent, content-independent fingerprints in their amplitude spectrum. Based on this insight, we design a dynamic fingerprint simulator capable of simulating over 1.6 trillion generative model architectures. We further train an extractor to capture model-specific fingerprint representations with supervised contrastive learning, enabling accurate attribution of synthetic images, even from previously unseen models. Our method does not rely on any synthetic images, instead, it is trained solely on real images. On DMDetection and AIGCBenchmark, which comprises dozens of state-of-the-art and in-the-wild generative models, our method improves the attribution performance (AUC) of the prior method from random level to 94.05% and 83.05%, respectively. On GenImage and OSMA datasets, we obtain 85.08%, and 88.48% OSCR, outperforming the SOTA methods by 4.30% and 9.37% under the same settings.

Recent advances in deepfake forensics have primarily focused on improving the classification accuracy and generalization performance. Despite enormous progress in detection accuracy across a wide variety of forgery algorithms, existing algorithms lack intuitive interpretability and identity traceability to help with forensic investigation. In this paper, we introduce a novel DeepFake Identity Recovery scheme (DFREC) to fill this gap. DFREC aims to recover the pair of source and target faces from a deepfake image to facilitate deepfake identity tracing and reduce the risk of deepfake attacks. It comprises three key components: an Identity Segmentation Module (ISM), a Source Identity Reconstruction Module (SIRM), and a Target Identity Reconstruction Module (TIRM). The ISM segments the input face into distinct source and target face information, and the SIRM reconstructs the source face and extracts latent target identity features with the segmented source information. The background context and latent target identity features are synergetically fused by a Masked Autoencoder in the TIRM to reconstruct the target face. We evaluate DFREC on different high-fidelity face-swapping attacks on FaceForensics++, CelebaMegaFS, FFHQ-E4S, and Celeb-DFv2 datasets, which demonstrate its superior recovery performance over state-of-the-art deepfake recovery algorithms. In addition, DFREC is the only scheme that can recover both pristine source and target faces directly from the forgery image with high fidelity.

We introduce OmniMark, a novel and efficient fingerprinting method for Latent Diffusion Models (LDM). OmniMark can encode user-specific fingerprints across diverse dimensions of the weights of the LDM, including kernels, filters, channels, and spatial domains. The LDM is fine-tuned to encode the invisible fingerprint into generated images, which can be decoded by a decoder. By altering fingerprints and re-encoding the weights, OmniMark supports efficient and scalable ad-hoc generation (

As AI generative models evolve, face swap technology has become increasingly accessible, raising concerns over potential misuse. Celebrities may be manipulated without consent, and ordinary individuals may fall victim to identity fraud. To address these threats, we propose Secure Swap, a method that protects persons of interest (POI) from face-swapping abuse and embeds a unique, invisible watermark into nonPOI swapped images for traceability. By introducing an ID Passport layer, Secure Swap redacts POI faces and generates watermarked outputs for nonPOI. A detachable watermark encoder and decoder are trained with the model to ensure provenance tracing. Experimental results demonstrate that Secure Swap not only preserves face swap functionality but also effectively prevents unauthorized swaps of POI and detects different embedded model’s watermarks with high accuracy. Specifically, our method achieves a 100% success rate in protecting POI and over 99% watermark extraction accuracy for nonPOI. Besides fidelity and effectiveness, the robustness of protected models against image-level and model-level attacks in both online and offline application scenarios is also experimentally demonstrated.

Current Large Vision-Language Models (LVLMs) have demonstrated remarkable capabilities in understanding multimodal data, but their potential remains underexplored for deepfake detection due to the misalignment of their knowledge and forensics patterns. To this end, we present a novel framework that unlocks LVLMs’ potential capabilities for deepfake detection. Our framework includes a Knowledge-guided Forgery Detector (KFD), a Forgery Prompt Learner (FPL), and a Large Language Model (LLM). The KFD is used to calculate correlations between image features and pristine/deepfake image description embeddings, enabling forgery classification and localization. The outputs of the KFD are subsequently processed by the Forgery Prompt Learner to construct fine-grained forgery prompt embeddings. These embeddings, along with visual and question prompt embeddings, are fed into the LLM to generate textual detection responses. Extensive experiments on multiple benchmarks, including FF++, CDF2, DFD, DFDCP, DFDC, and DF40, demonstrate that our scheme surpasses state-of-the-art methods in generalization performance, while also supporting multi-turn dialogue capabilities.

In real-world applications, screen capturing represents a significant scenario where this process can induce substantial distortion to the original image. Previous methods for simulating screen-shooting distortion often involved combining different formulas. We found that these simulation methods still have a significant gap compared to real distortions, making it urgently necessary to develop a realistic and credible comprehensive noise layer to achieve robustness against screen-shooting distortion. This paper presents a watermarking scheme capable of withstanding severe screen-shooting distortion. First, a dataset is constructed to train a screen-shooting distortion simulation network based on style transfer. Subsequently, a comprehensive noise layer is built upon this network to achieve robustness against severe screen-shooting distortion. Additionally, this paper incorporates structural re-parameterization techniques into the traditional U-shaped encoder to improve the quality of encoded images. Extensive experiments demonstrate the proposed scheme’s superior performance in terms of robustness and generalization, especially under severe screen-shooting distortion conditions.

With the rise of digital media, protecting image property has become a critical issue. Visible watermarks, once a key tool for copyright protection, have become increasingly vulnerable to removal methods using deep neural networks (DNNs). This poses a significant threat to the ability of visible watermarks to protect image ownership and copyright. To address this increasingly severe challenge, we propose a novel visible watermark protection mechanism based on information hiding. Unlike traditional methods of directly adding perturbations to protected images, we hide adversarial perturbations in watermarked images through a specially designed reversible information exchange (RIE) module, which includes multiple discrete wavelet transform (DWT) and affine coupling blocks. This design can concentrate the perturbations on textured areas of the watermarked images, making them less visually noticeable. Meanwhile, theoretical analysis indicates that the difference between the adversarial image (i.e., the watermarked image after embedding the adversarial perturbation) generated by our method and the watermarked image is completely controllable. To evaluate the proposed mechanism in various scenarios, based on several widely used datasets (i.e., LOGO-Gray, LOGO-H, and LOGO-L), we further synthesize two new datasets, namely LOGO-Multi and LOGO-Full. LOGO-Multi contains images embedded with multiple watermarks, and LOGO-Full contains images embedded with a watermark covering the whole image. Extensive testing on five datasets demonstrates that, compared to the baseline methods, the proposed scheme can greatly improve the visual quality of adversarial images and enhance their capability to resist various watermark removal techniques.

In this work, we propose IDGuard, a novel proactive defense method from the perspective of developers, to protect Persons-of-Interest (POI) such as national leaders from face editing abuse. We build a bridge between identities and model behavior, safeguarding POI identities rather than merely certain face images. Given a face editing model, IDGuard enables it to reject editing any image containing POI identities while retaining its editing functionality for regular use. Specifically, we insert an ID Normalization Layer into the original face editing model and introduce an ID Extractor to extract the identities of input images. To differentiate the editing behavior between POI and nonPOI, we use a transformer-based ID Encoder to encode extracted POI identities as parameters of the ID Normalization Layer. Our method supports the simultaneous protection of multiple POI and allows for the addition of new POI in the inference stage, without the need for retraining. Extensive experiments show that our method achieves 100% protection accuracy on POI images even if they are neither included in the training set nor subject to any preprocessing. Notably, our method exhibits excellent robustness against image and model attacks and maintains 100% protection performance when generalized to various face editing models, further demonstrating its practicality.

We propose a novel multi-bit box-free watermarking method for the protection of Intellectual Property Rights (IPR) of GANs with improved robustness against white-box model-level attacks like fine-tuning, pruning, quantization, and surrogate model attacks. The watermark is embedded by adding an extra watermarking loss term during GAN training, ensuring that the images generated by the GAN contain an invisible watermark that can be retrieved by a pre-trained watermark decoder. In order to improve the robustness against white-box model-level attacks, we make sure that the model converges to a wide flat minimum of the watermarking loss term, in such a way that any modification of the model parameters does not erase the watermark. To do so, we add random noise vectors to the parameters of the generator and require that the watermarking loss term is as invariant as possible with respect to the presence of noise. This procedure forces the generator to converge to a wide flat minimum of the watermarking loss. The proposed method is architecture- and dataset-agnostic, thus being applicable to many different generation tasks and models, as well as to CNN-based image processing architectures. We present the results of extensive experiments showing that the presence of the watermark has a negligible impact on the quality of the generated images, and proving the superior robustness of the watermark against model modification and surrogate model attacks.

With the development of the Internet of Things (IoT) and cloud computing, various multimedia data such as audio, video, and images have experienced explosive growth, ushering in the era of big data. Large-scale computing tasks in the Multimedia Internet of Things (M-IoT), such as mathematical optimization problems, have begun to be outsourced from IoT devices with limited computing power to cloud servers for execution. However, outsourcing computation brings security concerns, because the behaviors of clouds are invisible to users. The leakage of privacy data in outsourced optimization problems leads to immeasurable losses. The mutual distrust between clouds and users causes that the correctness of the optimal decisions and the fairness of the payment activities are not guaranteed. Blockchain technology has the characteristic of immutability and has become a new security paradigm for eliminating multi-party trust concerns. In this article, we propose a Bitcoin-based secure outsourcing scheme to address the aforementioned security concerns. To prevent confidential data leakage, the proposed scheme designs a computable privacy-preserving method for the outsourced optimization problems. To judge the correctness of the optimal decision and reduce verification costs, the proposed scheme designs a low-cost two-layer verification mechanism based on dual theory and blockchain technology. Blockchain nodes reach a consensus on the problem solutions and trigger an automatic fair payment protocol-based Bitcoin. Security analysis and experimental results demonstrate that our scheme guarantees privacy, fairness, and computational efficiency.

In this work, we investigate improving the generalizability of GAN-generated image detectors by performing data augmentation in the fingerprint domain. Specifically, we first separate the fingerprints and contents of the GAN-generated images using an autoencoder based GAN fingerprint extractor, followed by random perturbations of the fingerprints. Then the original fingerprints are substituted with the perturbed fingerprints and added to the original contents, to produce images that are visually invariant but with distinct fingerprints. The perturbed images can successfully imitate images generated by different GANs to improve the generalization of the detectors, which is demonstrated by the spectra visualization. To our knowledge, we are the first to conduct data augmentation in the fingerprint domain. Our work explores a novel prospect that is distinct from previous works on spatial and frequency domains augmentation. Extensive cross-GAN experiments demonstrate the effectiveness of our method compared to the state-of-the-art methods in detecting fake images generated by unknown GANs.

Abstract With the advent of the screen‐reading era, the confidential documents displayed on the screen can be easily captured by a camera without leaving any traces. Thus, this paper proposes a novel screen‐shooting resilient watermarking scheme for document image using deep neural network. By applying this scheme, when the watermarked image is displayed on the screen and captured by a camera, the watermark can be still extracted from the captured photographs. Specifically, the scheme is an end‐to‐end neural network with an encoder to embed watermark and a decoder to extract watermark. During the training process, a distortion layer between encoder and decoder is added to simulate the distortions introduced by screen‐shooting process in real scenes, such as camera distortion, shooting distortion, and light source distortion. Furthermore, a background sensitive loss and a lpips loss are used to improve visual quality of the watermarked document images in the training process. Besides, a strength factor adjustment strategy is also designed to improve the visual quality with little loss of bit extraction accuracy. The experimental results show that the proposed scheme has higher visual quality and robustness than the other two recent state‐of‐the‐art methods.

In this work, we propose a novel method to improve the generalization ability of CNN-based face forgery detectors. Our method considers the feature anomalies of forged faces caused by the prevalent blending operations in face forgery algorithms. Specifically, we propose a weakly supervised Second Order Local Anomaly (SOLA) learning module to mine anomalies in local regions using deep feature maps. SOLA first decomposes the neighborhood of local features by different directions and distances and then calculates the first and second order local anomaly maps which provide more general forgery traces for the classifier. We also propose a Local Enhancement Module (LEM) to improve the discrimination between local features of real and forged regions, so as to ensure accuracy in calculating anomalies. Besides, an improved Adaptive Spatial Rich Model (ASRM) is introduced to help mine subtle noise features via learnable high pass filters. With neither pixel level annotations nor external synthetic data, our method using a simple ResNet18 backbone achieves competitive performances compared with state-of-the-art works when evaluated on unseen forgeries.

This paper proposes a commonality learning strategy for face video forgery detection to improve the generalization. Considering various face forgery methods could leave certain similar forgery traces in videos, we attempt to learn the common forgery features from different forgery databases, so as to achieve better generalization in the detection of unknown forgery methods. Firstly, the Specific Forgery Feature Extractors (SFFExtractors) are trained separately for each of given forgery methods. We utilize the U-net structure and consider the triplet loss, location loss, classification loss, and automatic weighted loss to ensure the detection ability of SFFExtractors on the corresponding forgery methods. Next, the Common Forgery Feature Extractor (CFFExtractor) is trained under the supervision of SFFExtractors to explore the commonality of the forgery traces caused by different forgery methods. The extracted common forgery feature is expected to have a good generalization. The experimental results on FaceForensic++ show that the SFFExtractors outperform many state-of-the-arts in face forgery detection. The generalization performance of the CFFExtractor is verified on FaceForensic++, DFDC, and CelebDF. It is proved that commonality learning can be an effective strategy to improve generalization.

Recent progress in deep learning, in particular the generative models, makes it easier to synthesize sophisticated forged faces in videos, leading to severe threats on social media about personal privacy and reputation. It is therefore highly necessary to develop forensics approaches to distinguish those forged videos from the authentic. Existing works are absorbed in exploring frame-level cues but insufficient in leveraging affluent temporal information. Although some approaches identify forgeries from the perspective of motion inconsistency, there is so far not a promising spatiotemporal feature fusion strategy. Towards this end, we propose the Channel-Wise Spatiotemporal Aggregation (CWSA) module to fuse deep features of continuous video frames without any recurrent units. Our approach starts by cropping the face region with some background remained, which transforms the learning objective from manipulations to the difference between pristine and manipulated pixels. A deep convolutional neural network (CNN) with skip connections that are conducive to the preservation of detection-helpful low-level features is then utilized to extract frame-level features. The CWSA module finally makes the real or fake decision by aggregating deep features of the frame sequence. Evaluation against a list of large facial video manipulation benchmarks has illustrated its effectiveness. On all three datasets, FaceForensics++, Celeb-DF, and DeepFake Detection Challenge Preview, the proposed approach outperforms the state-of-the-art methods with significant advantages.