Deep learning-based watermarking models play a crucial role in copyright protection across various applications. However, many high-performance models are limited in practical deployment due to their large number of parameters. Meanwhile, the robustness and invisibility performance of existing lightweight models are unsatisfactory. This presents a pressing need for a watermarking model that combines lightweight capacity with satisfactory performance. Our research identifies a key reason that limits the performance of existing watermarking frameworks: a mismatch between commonly used decoding losses (e.g., mean squared error and binary cross-entropy loss) and the actual decoding goal, leading to parameter redundancy. We propose two innovative solutions: (1) Decoding-oriented surrogate loss (DO), which redesigns the loss function to mitigate the influence of decoding-irrelevant optimization directions; and (2) Detachable projection head (PH), which incorporates a detachable redundant module during training to handle these irrelevant directions and is discarded during inference. Additionally, we propose a novel watermarking framework comprising five submodules, allowing for independent parameter reduction in each component. Our proposed model achieves better efficiency, invisibility, and robustness while utilizing only 2.2% of the parameters compared to the state-of-the-art frameworks. By improving efficiency while maintaining robust copyright protection, our model is well suited for practical applications in resource-constrained environments. The DO and PH methods are designed to be plug-and-play, facilitating seamless integration into future lightweight models.

The effective denoising demonstrated by the latent diffusion model poses a new threat to image watermarking, as attackers can erase the watermark by performing a forward diffusion, followed by backward denoising. While such denoising might introduce large distortion in the pixel domain, the image semantics remain similar. Unfortunately, most existing robust watermarking methods fail to tackle such an erasure attack since they are primarily designed for traditional channel distortions. To address such issue, this paper proposed DERO, a diffusion-model-erasure robust watermarking framework. Based on the frequency domain analysis of the diffusion model's denoising process, we designed a destruction and compensation noise layer (DCNL) to approximate the distortion effects caused by latent diffusion model erasure (LDE). In detail, DCNL consists of a multi-scale low-pass filtering and a white noise compensation process, where the high-frequency components of the image are first obliterated, and then full-frequency components are enriched with white noise. Such a process broadly simulates the LDE distortions. Besides, on the extraction side, we cascaded a pre-trained variational autoencoder before the decoder to extract the watermark in the latent domain, which closely adapts to the operation domain of the LDE process. Meanwhile, to improve the robustness of the decoder, we also design a latent feature augmentation (LFA) operation on the latent feature. Throughout the end-to-end training with the DCNL and LFA, DERO can successfully achieve robustness against LDE. Our experimental results demonstrate the effectiveness and the generalizability of the proposed framework. The LDE robustness is significantly improved from 75% with SOTA methods to an impressive 96% with DERO.

Abstract With the advent of the digital era, millions of facial images are shared online daily, posing severe privacy threats. Generative face anonymization (GFA) methods generate virtual faces to conceal original identities, protecting sensitive information while preserving utility. However, deep learning based user identity linkage (UIL) methods can link similar faces to the same identity and leverage the linked profiles for malicious purposes, including localization and behaviour prediction. These UIL methods pose a significant challenge to the diversity of virtual faces, a challenge that existing GFA methods have not adequately addressed. To address this research gap, we propose Style Diversification‐based Generative Face Anonymization (SD‐GFA), a framework that generates virtual faces with diverse identities and high visual quality. SD‐GFA features an equalized control module to balance input faces and user‐specified keys, a face generation module with a re‐connection strategy for high‐quality synthesis, and a maximum probability simulation module to enhance diversity. Our experiments demonstrate that SD‐GFA effectively mitigates linkage risk by improving the diversity of virtual faces, while also enhancing their utility and visual quality. This study provides a robust solution to enhance the security of anonymized faces shared on the internet.

A high-value dataset is the key for accurate deep learning models, therefore, protecting the dataset is particularly important. Once the dataset is stolen, the attacker can easily train a surrogate model with similar performance to the original model. One possible solution to address such threat is data poisoning, whereby the performance of the surrogate model could be greatly influenced if trained with poisoned dataset. This paper focuses on an advanced scenario where the attacker might be an experienced malicious employee who has the white-box access to the dataset and black-box access (can only query) to original business model (e.g.MLaaS model). In order to re-train a surrogate model, he may first judge whether the dataset is poisoned and then try to erase potential perturbations to restore the original dataset. Under this condition, three main requirements must be satisfied: 1.Imperceptibility, which ensures that the poisoned data is not easily identified by human eyes; 2.Robustness, which ensures that the perturbation is not easily erased. 3.Stealthiness, which ensures that the poisoned data will not be recognized by the original business model i.e. produce abnormal output. In this paper, we propose a noveldataprotection method bydatapoisoning dubbed DP$^{2}$to meet the requirements. To achieve imperceptibility and robustness, we propose a poisoning mechanism that consists of a poisoning process and a balancing process. The poisoning process is conducted by a designed dual-U-Net-based poisoning network, by training with the reference mapping strategy and the corresponding noise layer, the imperceptibility and robustness can be both achieved. Then the balancing process is performed to balance the imperceptibility and poisoning performance. As for stealthiness, we propose a recover-net to eliminate the perturbation, so that the business model with black-box access could be an enclose version of the recover-net and the original business model. Besides, based on the recover-net, the poisoned dataset could be re-applied for the normal use. Various experiments indicate superior performance of the proposed scheme in the view of imperceptibility and robustness compared with other schemes. The solution which makes the poisoned data recoverable greatly ensures the stealthiness, and the derived recoverability of poisoned data could be utilized in other scenarios.

Deep learning-based digital watermarking frameworks have been widely studied recently. Most existing methods adopt an ``encoder-noise layer-decoder''-based architecture where the embedding and extraction processes are accomplished separately by the encoder and the decoder. However, one potential drawback of such a framework is that the encoder and the decoder may not be well coupled, resulting in the fact that the encoder may embed some redundant features into the host image thus influencing the invisibility and robustness of the whole algorithm. To address this limitation, this paper proposes a flow-based robust watermarking framework. The basic component of such framework is an invertible up-down-sampling neural block that can realize the embedding and extraction simultaneously. As a consequence, the encoded feature could keep high consistency with the feature that the decoder needed, which effectively avoids the embedding of redundant features. In addition, to ensure the robustness of black-box distortion, an invertible noise layer (INL) is designed to simulate the distortion and is served as a noise layer in the training stage. Benefiting from its reversibility, INL is also applied as a preprocessing before extraction to eliminate the distortion, which further improves the robustness of the algorithm. Extensive experiments demonstrate the superiority of the proposed framework in terms of visual quality and robustness. Compared with the state-of-the-art architecture, the visual quality (measured by PSNR) of the proposed framework improves by 2dB and the extraction accuracy after JPEG compression (QF=50) improves by more than 4%. Besides, the robustness against black-box distortions can be greatly achieved with more than 95% extraction accuracy.

Deep neural networks are vulnerable to adversarial attacks. In this paper, we take the role of investigators who want to trace the attack and identify the source, that is, the particular model which the adversarial examples are generated from. Techniques derived would aid forensic investigation of attack incidents and serve as deterrence to potential attacks. We consider the buyers-seller setting where a machine learning model is to be distributed to various buyers and each buyer receives a slightly different copy with the same functionality. A malicious buyer generates adversarial examples from a particular copy ${\mathcal{M}_i}$ and uses them to attack other copies. From these adversarial examples, the investigator wants to identify the source ${\mathcal{M}_i}$. To address this problem, we propose a two-stage separate-and-trace framework. The model separation stage generates multiple copies of a model for the same classification task. This process injects unique features into each copy so that adversarial examples generated have distinct and traceable features. We give a parallel structure which pairs a unique tracer with the original classification model in each copy and a variational autoencoder (VAE)-based training method to achieve this goal. The tracing stage takes in adversarial examples and a few candidate models, and identifies the likely source. Based on the unique features induced by the tracer, we could effectively trace the potential adversarial copy by considering the output logits from each tracer. Empirical results show that it is possible to trace the origin of the adversarial example and the mechanism can be applied to a wide range of architectures and datasets.

Cross-channel (e.g. Screen-to-Camera) robustness is an urgent requirement for modern watermarking systems. To realize such robustness, training a network that can precisely simulate the cross-channel distortion as the noise layer for deep watermarking training is an effective way. However, network training requires massive data, and generating the data is laborious. Meanwhile, directly using limited data to train may lead to an over-fitting issue. To address such limitation, we proposed DeNoL, a decoupling noise layer for cross-channel simulation which only needs few-shot samples. We believe the overfitting issue comes from the overlearning of the training image content rather than only simulating the distortion style. Consequently, we design a network that can decouple the image content and the distortion style into different components. Thus, by fixing the content representation component and fine-tuning a new style component accordingly, the network can efficiently learn and only learn the distortion style. Such learning can be done with only few-shot samples. Besides, in order to enhance adaptability, we also proposed a diversification operation to cooperate with DeNoL. Experimental results show that DeNoL can effectively simulate cross-channel distortion with only 20 image pairs and assist in training a general and robust watermarking network.

Deep-learning-based watermarking technique is being extensively studied. Most existing approaches adopt a similar encoder-driven scheme which we name END (Encoder-NoiseLayer-Decoder) architecture. In this paper, we revamp the architecture and creatively design a decoder-driven watermarking network dubbed De-END which greatly outperforms the existing END-based methods. The motivation for designing De-END originated from the potential drawback we discovered in END architecture: The encoder may embed redundant features that are not necessary for decoding, limiting the performance of the whole network. We conducted a detailed analysis and found that such limitations are caused by unsatisfactory coupling between the encoder and decoder in END. De-END addresses such drawbacks by adopting a Decoder -Encoder-Noiselayer-Decoder architecture. In De-END, the host image is firstly processed by the decoder to generate a latent feature map instead of being directly fed into the encoder. This latent feature map is concatenated to the original watermark message and then processed by the encoder. This change in design is crucial as it makes the feature of encoder and decoder directly shared thus the encoder and decoder are better coupled. We conducted extensive experiments and the results show that this framework outperforms the existing state-of-the-art (SOTA) END-based deep learning watermarking both in visual quality and robustness. On the premise of the same decoder structure, the visual quality (measured by PSNR) of De-END improves by 1.6dB (45.16dB to 46.84dB), and extraction accuracy after JPEG compression (QF=50) distortion outperforms more than 4% (94.9% to 99.1%).