Embedding watermarks in the diffusion latent space improves robustness but often alters visual content due to the distribution shift between watermarked and clean latent variables. To address this issue, stable latent integration watermark (SLIM) is proposed in this paper, in which watermarks are integrated into the features output by the noise prediction network of a diffusion model, while ensuring that the perturbation introduced in the diffusion latent space remains negligible. Specifically, a watermark encoder–decoder is first trained to convert binary watermark sequences into watermark latent variables that are dimensionally compatible with the diffusion latent variables, enabling flexible and reliable embedding and extraction. The watermark latent variables are processed through the first down-sampling block of the denoising U-Net, and the resulting watermark features are fused with the block output to minimize interference with image semantics. To counteract the perturbations in diffusion features induced by watermark embedding and to ensure accurate watermark extraction, the denoising U-Net is efficiently fine-tuned using a low-rank adaptation module. Experimental results demonstrate that SLIM achieves superior generation quality while exhibiting exceptional robustness against diverse attacks compared with baseline methods. Code will be available at https://github.com/XiaoxiKong/SLIM.

WebP, as an image format increasingly adopted across the internet, is becoming an important cover for steganography, yet its steganographic capacity remains largely unexplored. The distinctive structure of lossy WebP compression, particularly its predictive coding and uniform quantization, poses significant challenges to conventional embedding cost modeling. These include nonlinear relationship between Discrete Cosine Transform (DCT) coefficients and reconstruction pixels, quantization uniformity obscuring frequency sensitivity, and distortion propagation across blocks. To address these challenges, we propose WebP-Adaptive Residual-reconstruction-united and Predictive-aware steganography (WARP), a novel embedding framework specifically designed for WebP images. WARP introduces a unified cost modeling paradigm that integrates reconstruction-domain texture complexity, residual-frequency quantization behavior, and predictive coding order. Specifically, it precisely identifies secure embedding regions by calculating reconstruction-domain texture-adaptive costs and then refining them with residual-domain frequency sensitivity. Furthermore, to mitigate inter-block distortion propagation inherent in predictive coding, WARP incorporates a novel predictive-order-aware cost decay mechanism that adjusts embedding costs based on block coordinates. To the best of our knowledge, this work represents the first comprehensive study on secure adaptive WebP steganography. Extensive experiments demonstrate that WARP achieves superior security performance against state-of-the-art steganalysis while maintaining high visual quality.

Video inpainting aims to reconstruct missing or corrupted regions in video frames, with applications in video editing, restoration, and special effects. Current deep video inpainting methods rely on optical flow to guide the propagation of effective features and spatiotemporal attention mechanisms to model relationships between frames. However, as an explicit motion representation, the optical flow extracted offline in preceding steps often suffers from instability and errors during estimation. These errors accumulate during subsequent content hallucination, resulting in artifacts and blurring. Meanwhile, although traditional spatiotemporal attention effectively captures frame relationships, its dense computational nature introduces redundant information, disrupting inpainting tasks and reducing efficiency. To address these issues, we propose an implicit motion-guided approach for efficient video inpainting. Instead of relying on optical flow, our method uses implicit motion in the latent feature space to guide the dual-domain propagation of images and features end-to-end, avoiding error accumulation from the independent optical flow estimation process. Additionally, we introduce a self-correcting module that enables feedback between image and feature propagation, reducing errors during propagation. Furthermore, we design an adaptive sparse video attention mechanism to focus on highly relevant regions, minimizing the impact of irrelevant information. Experimental results demonstrate that the proposed method outperforms state-of-the-art approaches both qualitatively and quantitatively, while also delivering superior efficiency.

Document images are vulnerable to tampering attacks from image editing tools and deep models. Therefore, the Document Tampering Localization (DTL) task has received increasing attention in recent years. However, given the wide variety of document types (e.g., contracts, certificates, ID cards), our analysis shows that existing DTL methods struggle with document images containing diverse background colors and varying semantic contents. Further analysis and experiments verify that the varying background color and semantic contents interfere with the forensic feature extraction process in the existing DTL methods. To address this issue, we propose two disentanglement modules to mitigate such interference and improve the ability of forgery trace detection. First, we design a Color Disentanglement (CD) module that applies disentangled learning representation to forensic features. The CD module, grounded in real-world prior knowledge, effectively decouples color information from forensic features, thereby improving robustness against varying background colors. Second, we propose the Semantic Disentanglement (SD) module, which performs image-level clustering on the tampering probability map during the inference process. The SD module focuses on tampering probabilities for each pixel, while discarding local semantic information (e.g., font, location, and shape). It leads to strong robustness against variations in document content. The evaluations demonstrate that our CD-SD method outperforms existing methods by 45.12% or 0.162 on the F1 metric in cross-dataset tests. Ablation studies show that the CD and SD modules improve the F1 score by 7.98% and 13.38%, respectively, across different backbones. Our method delivers consistent and stable improvements across various experimental protocols. Moreover, it is compatible with many DTL methods in a plug-and-play fashion.

In the advertising and media industries, image editing often involves multiple manipulation techniques to meet creative and technical requirements. Detecting tampered regions is crucial in scenarios like legal disputes or media integrity assessments. However, existing forensic methods often target single manipulation types or treat all manipulations as one, and many deep learning approaches lack flexibility in frequency and edge extraction, limiting their effectiveness. To address these challenges, this paper proposes an ALL-IN-ONE framework for comprehensive image forensic analysis, which adopts a divide-and-conquer strategy for multi-manipulation image classification and localization. Specifically, we introduce a Multi-Frequency Band Extraction Module (MBEM) to capture richer artifact information in the frequency domain. This is complemented by an Attention Window-based Fusion Module, which fuses same-frequency features across different scales and enhances the discriminative features more effectively. To improve the localization of copy-move manipulation, we design a Copy-Move Accurate Detection Module (CADM), which leverages the visual consistency between source and target regions. Furthermore, we propose a Precise Edge Generator (PEG) as part of the Edge-Guided Progressive Fine-Tune Module (EPFM), which can generate more accurate edge to enhance edge localization. To address the issue of insufficient labeled data, we construct a publicly available dataset, the Multi-Manipulation Image Dataset (MMID), consisting of 2,000 multi-manipulation images, each containing at least two types of forgeries. Extensive experiments are conducted, comparing our method with state-of-the-art approaches on MMID, as well as on single-manipulation datasets such as CASIA, CoMoFoD, and NIST. The results demonstrate that MMID is effective for training discriminative models and validate that our proposed method significantly outperforms existing approaches in terms of accuracy and robustness for simultaneous forgery localization and manipulation classification.

The quantization step is a crucial parameter in the JPEG compression process, and provides prior knowledge for JPEG image steganography and forensics. Existing neural network-based methods typically estimate the quantization steps for all discrete cosine transform (DCT) subbands jointly, by treating the entire quantization table as a unified input and leveraging the inter-subband relationships. However, subband relationships vary across different quantization tables, leading to poor generalization for methods that rely heavily on such relationships. To address the above issues, we depart from the strategy that relies on inter-subband relationships and instead train the model on a specific single subband. To compensate for the possible decrease in accuracy due to the lack of relationships between subbands, we extract the ranking features and histogram features from the DCT coefficient histograms of the subbands. Ranking features capture local patterns in DCT histograms by modeling the relative relationships between neighboring coefficients, thereby compensating for the absence of local detail. On the other hand, histogram features represent the overall distribution pattern of the DCT coefficient histograms and capture the global trends and statistical properties in the subbands. We subsequently employ convolutional groups and multilayer perceptron (MLP) structures to extract compression artifacts from these two features. Finally, we introduce a comprehensive evaluation metric, called GenAQt, to quantify the algorithm’s generalization ability across quantization tables. The experimental results demonstrate that our method maintains high accuracy across quantization tables, with RelGenAQt (relative accuracy decrease) exceeding 81% and AbsGenAQt (absolute accuracy decrease) being less than 0.38.

Deep learning-based image forgery localization models are increasingly deployed in real-world forensic services, yet their robustness against black-box adversarial manipulation remains insufficiently understood, calling for practical anti-forensics techniques to expose potential security weaknesses. Prior adversarial anti-forensics studies for forgery localization mainly assume white-box access, which limits their applicability to deployed systems where only hard, mask-like outputs are available and queries are tightly constrained. To bridge this gap, we propose AdvFor, a query-efficient black-box attack frame-work tailored for forgery localization with hard-label, mask-only, spatially dense binary feedback. AdvFor formulates the attacker–model interaction as a finite-horizon Markov Decision Process and learns a transferable attack policy from hard-mask feedback. Once trained, AdvFor can be deployed via fixed-length policy execution with onlyT=7 queries per image, avoiding per-image boundary refinement or query-dense direction/gradient estimation. The learned policy optimizes a structured objective—progressively suppressing forgery responses in the predicted localization mask so that the masks of forgery images approach an authentic-like (near-zero) output—while maintaining visual fidelity. Extensive experiments on six benchmark datasets and multiple modern forgery localization models demonstrate that AdvFor consistently achieves stronger attack performance than representative baselines under the same perturbation constraints, while operating in an ultra-low-query regime.We further validate AdvFor under common deployment-style defenses, showing its notable effectiveness in realistic settings.

The widespread application of AIGC contents has brought not only unprecedented opportunities, but also potential security concerns, e.g., audio-visual deepfakes. Therefore, it is of great importance to develop an effective and generalizable method for multi-modal deepfake detection. Typically, the audio-visual correlation learning could expose subtle cross-modal inconsistencies, e.g., audio-visual misalignment, which serve as crucial clues in deepfake detection. In this paper, we reformulate the correlation learning with variational Bayesian estimation, where audio-visual correlation is approximated as a Gaussian distributed latent variable, and thus develop a novel framework for deepfake detection, i.e., Forgery-aware Audio-Visual Adaptation with Variational Bayes (FoVB). Specifically, given the prior knowledge of pre-trained backbones, we adopt two core designs to estimate audio-visual correlations effectively. First, we exploit various difference convolutions and a high-pass filter to discern local and global forgery traces from both modalities. Second, with the extracted forgery-aware features, we estimate the latent Gaussian variable of audio-visual correlation via variational Bayes. Then, we factorize the variable into modality-specific and correlation-specific ones with orthogonality constraint, allowing them to better learn intra-modal and cross-modal forgery traces with less entanglement. Extensive experiments demonstrate that our FoVB outperforms other state-of-the-art methods in various benchmarks.

Generator-based adversarial attack methods aim to fool deep neural networks (DNNs) by training a generator for crafting adversarial examples (AEs). However, as DNNs evolve from Convolutional Neural Networks (CNNs) to Transformers, the existing generator-based methods can hardly achieve satisfactory attack performance against different target model architectures in semi-whitebox attack scenarios. In addition, the generated AEs are susceptible to various distortions (especially for JPEG compression with low quality factors), which deteriorate the attack ability and increase the unreliability. To address these issues, we propose a dual-branch guided generative model called Heterogeneous Encoding Network (HENet) to form a robust generator-based adversarial attack framework. Specifically, our HENet introduces an Adaptive Feature Fusion Module (AFFM) to solve the dimensions and representativeness contradictions between CNNs and Transformers, which steers the perturbation generation based on a richer latent space and achieves better general attack ability. To further improve the robustness against JPEG compression, we design and integrate a Dynamic Differentiable JPEG Simulator (DDJS), which introduces an adaptive quantization mask to determine the flow of the gradient backpropagation in each frequency position. Extensive experiments prove the proposed method achieves a better attack success rate, lower perturbation magnitude, and higher robustness for various target network architectures under compressed, distorted, and lossless scenarios. Our codes will be made publicly available.

The proliferation of deepfake faces poses huge potential negative impacts on our daily lives. Despite substantial advancements in deepfake detection over these years, the generalizability of existing methods against forgeries from unseen datasets or created by emerging generative models remains constrained. In this paper, inspired by the zero-shot advantages of Vision-Language Models (VLMs), we propose a novel approach that repurposes a well-trained VLM for general deepfake detection. Motivated by the model reprogramming paradigm that manipulates the model prediction via input perturbations, our method can reprogram a pre-trained VLM model (e.g., CLIP) solely based on manipulating its input without tuning the inner parameters. First, learnable visual perturbations are used to refine feature extraction for deepfake detection. Then, we exploit information of face embedding to create sample-level adaptative text prompts, improving the performance. Extensive experiments on several popular benchmark datasets demonstrate that (1) the cross dataset and cross-manipulation performances of deepfake detection can be significantly and consistently improved (e.g., over 88% AUC in cross-dataset setting from FF++ to Wild-Deepfake); (2) the superior performances are achieved with fewer trainable parameters, making it a promising approach for real-world applications.

Recently, image inpainting has become a common tool for manipulating nature images in a malicious manner, which has led to the rapid advancement of inpainting forensics. Although current forensics methods have shown precise location of inpainting regions and reliable robustness against image post-processing operations, it remains unclear whether they can effectively resist the possible attacks in real-world scenarios. To identify potential flaws, we propose a novel black-box anti-forensics framework to attack inpainting forensics methods, which employs reinforcement learning to generate a query-efficient countermeasure, named RLGC. To this end, we define reinforcement learning paradigm to model the Markov Decision Process of query-based black-box anti-forensics scenario. Specifically, pixel-wise agents are used to modulate anti-forensics images based on action selection and query forensics methods to obtain corresponding outputs. Later, reward function evaluates attack effect and image distortion with these outputs. To maximize the cumulative reward, policy and value networks are integrated and trained by Asynchronous Advantage Actor-Critic algorithm. Experimental results demonstrate that, without visually detectable distortion on anti-forensics images, RLGC achieves remarkable attack effects in a highly query-effcient way against various black-box inpainting forensics methods, even outperforming the most representative white-box attack method.

With the rapid advancement of deep neural networks (DNNs), model watermarking has emerged as a widely adopted technique for safeguarding model copyrights. A prevalent method involves utilizing a watermark decoder to retrieve watermark bits from generated outputs, but such methods are often vulnerable to model fine-tuning attacks. Traditionally, this challenge is mitigated through adversarial training or data augmentation, both of which significantly increase the computational burden. In this paper, we present a solution employing Flat Minima Aware (FMA) optimizers to bolster the robustness of model watermarking without requiring additional training data. By optimizing the watermark loss with flat minima awareness, our approaches significantly enhance the robustness of watermarks against the model fine-tuning attack. Comprehensive experiments have demonstrated our method’s superior ability to preserve watermark integrity. These findings suggest that this innovative optimization strategy offers a robust and efficient pathway for protecting models, thereby contributing to more secure and reliable model copyright protection mechanisms.

Recapture detection is critical in forensic tasks, especially for authentication of face and document images in electronic Know Your Customer (e-KYC) processes. While deep learning has advanced face anti-spoofing (FAS) and document presentation attack detection (DPAD), weak forensic cues still hinder reliability. We propose the Forensicability Assessment Network (FANet) to assess sample forensicability and reject low-forensicability samples before recapture detection. This enhances the overall performance and reliability of e-KYC systems. FANet operates independently, without relying on real training data or actual forensic features, ensuring strong generalization across various scenarios. It combines image quality and forensic task cues, defining three forensicability classes based on domain knowledge. FANet is trained with cross-entropy loss, updating centers using a momentum-based approach. Experimental results demonstrate significant improvements in reliability by filtering out low-forensicability samples. This work introduces the first comprehensive approach to assessing forensicability, ensuring more reliable recapture detection in face and document images. The source codes are available at https://github.com/chenlewis/FANet.

Recent advances in face forgery detection have shown strong in-domain performance but often fail to generalize to out-of-distribution data, especially when confronted with unseen manipulation techniques or domain shifts (e.g., lighting conditions, camera noise). We propose a novel Mixture-of-Experts framework, termed GM-DF, that decouples domain-specific and domain-invariant features to tackle cross-domain face forgery detection. Our method builds upon a foundation model (CLIP) and incorporates three key modules: (1) Dataset-Embedding Generator that leverages lightweight expert layers and database-aware feature normalization to adaptively modulate features at a per-domain level, capturing idiosyncratic cues without overfitting; (2) Multi-Dataset Representation mechanism that fuses these expert embeddings using scaled dot-product attention and integrates a mask image modeling (MIM) task to amplify local forgery artifacts; (3) Meta-Domain-Embedding Optimizer, inspired by MAML, which alternates between domain-specific (inner-loop) and domain-invariant (outer-loop) updates to facilitate rapid adaptation on new domains. Additionally, inspired by [13] (Yossi Gandelsman, Alexei A Efros, and Jacob Steinhardt. 2024. Interpreting the second-order effects of neurons in clip. arXiv preprint arXiv:2406.04341 (2024)) we introduce second-order feature propagation in the intermediate layers of CLIP to enhance fine-grained artifact cues and propose domain-class disentangled prompts to flexibly encode multi-domain text representations. Together, these strategies enable GM-DF to learn robust, shared forgery cues while preserving essential domain nuances. Our extensive experiments on multiple cross-domain benchmarks demonstrate that GM-DF significantly outperforms state-of-the-art approaches in both detection accuracy and domain transferability, reducing reliance on superficial artifacts and improving generalization to unseen forgeries. Importantly, our design requires minimal overhead beyond standard CLIP, making GM-DF both effective and computationally efficient for real-world face forgery detection.

Document Image Tampering Localization (DITL) advances considerably, yet achieving robust cross-dataset generalization remains a formidable challenge for practical applications. Expanding existing document datasets for training is labor-intensive, making it appealing to incorporate data from non-document domains such as natural scene images. However, domain-specific variations, including differences in color distribution and texture, compromise the performance of joint training. To address this issue, we propose DITL2, a Dual-stage Invariance Transfer Learning framework for Document Image Tampering Localization that consists of Cross-Domain Invariance Pre-training (CDIP) and Frequency Decoupling Parameter Adaptation (FDPA). In the pre-training stage, CDIP employs style transfer and texture consistency learning to suppress domain-specific influences from tampered natural scene images, and tampering trace commonality learning to acquire domain-invariant features. In the fine-tuning stage, FDPA adapts the parameters of the pre-trained model, leveraging the general knowledge from the pre-trained model to address DITL tasks while reducing the risk of overfitting. Experiments show that this approach effectively leverages external data resources to boost model performance, achieving state-of-the-art results across a variety of cross-dataset settings.

A significant challenge in audio deepfake detection (ADD) is to improve model generalization against unseen vocoders and other unknown factors, as existing methods often overfit to specific vocoder patterns or synthetic-irrelevant factors. To overcome this challenge, by focusing on vocoder-agnostic features and synthetic traces for generalizable ADD, we propose a novel dual-level disentanglement with meta-learning (ALDEN ) framework. Specifically, we first introduce an adversarial-training-based disentanglement learning (ADL) module to explicitly learn vocoder-agnostic and vocoder-specific features, effectively disentangling audio signals in terms of low-level characteristics. To suppress synthetic-irrelevant information, such as semantics and speaker identities, we simultaneously employ a reconstruction-based disentanglement learning (RDL) module, which further disentangles both synthetic-relevant and synthetic-irrelevant features from vocoder-agnostic features at a high-level of semantics. Additionally, as low-level non-semantic features are more critical in ADD, a vocoder-agnostic meta-learning (VAML) module is proposed to simulate cross-vocoder scenarios so as to further boost generalization performance. Extensive experiments demonstrate that ALDEN outperforms state-of-the-art methods in cross-vocoder and in-the-wild scenarios. The code, model, and supplementary materials will be released on the project page: https://beyond0814.github.io/ALDEN/.

Securing personal identity against deepfake attacks is increasingly critical in the digital age, especially for celebrities and political figures whose faces are easily accessible and frequently targeted. Most existing deepfake detection methods focus on general-purpose scenarios and often ignore the valuable prior knowledge of known facial identities, e.g., "VIP individuals" whose authentic facial data are already available. In this paper, we propose **VIPGuard**, a unified multimodal framework designed to capture fine-grained and comprehensive facial representations of a given identity, compare them against potentially fake or similar-looking faces, and reason over these comparisons to make accurate and explainable predictions. Specifically, our framework consists of three main stages. First, we fine-tune a multimodal large language model (MLLM) to learn detailed and structural facial attributes. Second, we perform identity-level discriminative learning to enable the model to distinguish subtle differences between highly similar faces, including real and fake variations. Finally, we introduce user-specific customization, where we model the unique characteristics of the target face identity and perform semantic reasoning via MLLM to enable personalized and explainable deepfake detection. Our framework shows clear advantages over previous detection works, where traditional detectors mainly rely on low-level visual cues and provide no human-understandable explanations, while other MLLM-based models often lack a detailed understanding of specific face identities. To facilitate the evaluation of our method, we build a comprehensive identity-aware benchmark called **VIPBench** for personalized deepfake detection, involving the latest 7 face-swapping and 7 entire face synthesis techniques for generation. Extensive experiments show that our model outperforms existing methods in both detection and explanation. The code is available at https://github.com/KQL11/VIPGuard .

In audio spoofing detection, most studies rely on clean datasets, making models susceptible to real-world post-processing attacks, such as channel compression and noise. To overcome this challenge, we propose the Adaptive MixtUre Low-rank ExperTs (AMULET) framework, which enhances resilience by leveraging attack-specific knowledge and dynamically adapting to varied attack conditions. Specifically, AMULET employs Attack-Specific Experts (ASEs) fine-tuned with Low-Rank Adaptation (LoRA), allowing each expert to focus on distinct post-processing patterns using just 1.13% of the parameters required for full fine-tuning. Furthermore, we introduce Adaptive Expert Fusion (AEF), which adaptively selects and integrates expert knowledge to enhance the robustness of spoofing detection. Experimental results demonstrate that AMULET significantly enhances robustness by improving noise resilience and exhibiting greater adaptability to unseen post-processing methods compared to models trained with full fine-tuning. Additionally, our framework outperforms both single expert and other expert aggregation strategies under various mixed attacks, demonstrating its superior robustness and adaptability in managing complex real-world scenarios. Both our data processing pipeline and model will be released athttps://github.com/foolishcqx/AMULET-AudioSpoof.

Document Presentation Attack is an anti-forensic operation that conceals the forgery traces of image manipulation in the digital domain. Existing document presentation attack detection (DPAD) methods show unsatisfactory performance under samples with different contents and qualities. In this work, we focus on the DPAD task on screen-recapturing channel and exploit the prior knowledge of distortion (i.e., moire pattern) in the spectral domain to address these limitations. We propose a frequency-domain moir ´ e´ augmentation (FMAG) strategy that enhances the spectral components contributed to the moire distortion, improving the generalization ´ performance under different document contents. We devise the mask moire frequency modeling (M ´ 2FM) scheme to reconstruct the moire-related spectral components in low-quality samples under the guidance of the spectral distortion model and a pre-trained DPAD ´ classifier. To evaluate the generalization performance, we collect the diverse Screen Recaptured Document Image Dataset with 162 different document contents (SRDID162) consisting of 162 genuine document images, as well as 2592 low and high-quality recaptured document images, respectively. Our experimental protocol involves training with high-quality ID images and testing with SRDID162 dataset of diverse contents and image qualities. Compared to a SOTA data augmentation approach for recaptured natural images, our FMAG & M2FM approach achieves a significant improvement of 49.15% or 22.50 percentage points in average EER on the generic deep learning backbones. The data and code of this work will be available at Github

Due to the widespread use and proliferation of Deep Neural Networks (DNNs), safeguarding their Intellectual Property Rights (IPR) has become increasingly important. This article proposes a method for watermarking a cyclic Generative Adversarial Network (GAN), specifically CycleGAN, to address the gap between the watermarking of conventional GAN models and cyclic GAN watermarking. The proposed method involves training a watermark decoder, which is then frozen and used to extract the watermark bits during the training of the CycleGAN model. The model is trained using specific loss functions that are optimized to achieve excellent performance on both the Image-to-Image Translation (I2IT) task and watermark embedding. Besides, a comprehensive theoretical and practical statistical analysis to verify the ownership of the model from the extracted watermark bits is given. At last, the model's robustness is evaluated against image post-processing, and further improved by fine-tuning the watermark decoder by applying data augmentation to the generated images before extracting the watermark bits. We also verify the robustness of the watermark to surrogate model attacks, carried out by accessing the watermarked model in a black-box modality. The experimental results demonstrate that the proposed method is effective and robust against image post-processing and can resist surrogate model attacks.

Anti-forensics seeks to eliminate or conceal traces of tampering artifacts. Typically, anti-forensic methods are designed to deceive binary detectors and persuade them to misjudge the authenticity of an image. However, to the best of our knowledge, no attempts have been made to deceive forgery detectors at the pixel level and mis-locate forged regions. Traditional adversarial attack methods cannot be directly used against forgery localization due to the following defects: 1) they tend to just naively induce the target forensic models to flip their pixel-level pristine or forged decisions; 2) their anti-forensics performance tends to be severely degraded when faced with the unseen forensic models; 3) they lose validity once the target forensic models are retrained with the anti-forensics images generated by them. To tackle the three defects, we propose SEAR (Self-supErvised Anti-foRensics), a novel self-supervised and adversarial training algorithm that effectively trains deep-learning anti-forensic models against forgery localization. SEAR sets a pretext task to reconstruct perturbation for self-supervised learning. In adversarial training, SEAR employs a forgery localization model as a supervisor to explore tampering features and constructs a deep-learning concealer to erase corresponding traces. We have conducted large-scale experiments across diverse datasets. The experimental results demonstrate that, through the combination of self-supervised learning and adversarial learning, SEAR successfully deceives the state-of-the-art forgery localization methods, as well as tackle the three defects regarding traditional adversarial attack methods mentioned above.

Recent advances in deep learning algorithms have shown impressive progress in image copy-move forgery detection (CMFD). However, these algorithms lack generalizability in practical scenarios where the copied regions are not present in the training images, or the cloned regions are part of the background. Additionally, these algorithms utilize convolution operations to distinguish source and target regions, leading to unsatisfactory results when the target regions blend well with the background. To address these limitations, this study proposes a novel end-to-end CMFD framework that integrates the strengths of conventional and deep learning methods. Specifically, the study develops a deep cross-scale PatchMatch (PM) method that is customized for CMFD to locate copy-move regions. Unlike existing deep models, our approach utilizes features extracted from high-resolution scales to seek explicit and reliable point-to-point matching between source and target regions. Furthermore, we propose a novel pairwise rank learning framework to separate source and target regions. By leveraging the strong prior of point-to-point matches, the framework can identify subtle differences and effectively discriminate between source and target regions, even when the target regions blend well with the background. Our framework is fully differentiable and can be trained end-to-end. Comprehensive experimental results highlight the remarkable generalizability of our scheme across various copy-move scenarios, significantly outperforming existing methods.

The rebroadcasting of screen-recaptured document images introduces a significant risk to the confidential docu-ments processed in government departments and commer-cial companies. However, detecting recaptured document images subjected to distortions from online social networks (OSNs) is challenging since the common forensics cues, such as moiré pattern, are weakened during transmission. In this work, we first devise a pixel-level distortion model of the screen-recaptured document image to identify the robust features of color artifacts. Then, we extract a chromaticity map from the recaptured image to highlight the presence of color artifacts even under low-quality samples. Based on the prior understanding, we design a chromaticity map adapter (CMA) to efficiently extract the chromaticity map, and feed it into the transformer backbone as multi-modal prompt tokens. To evaluate the performance of the pro-posed method, we collect a recaptured office document im-age dataset with over 10K diverse samples. Experimental results demonstrate that the proposed CMA method outper-forms a SOTA approach (with RGB modality only), reducing the average EER from 26.82% to 16.78%. Robustness eval-uation shows that our method achieves 0.8688 and 0.7554 AUCs under samples with JPEG compression$(QF=70)$and resolution as low as$534\times 503$pixels.

As manipulating images may lead to misinterpretation of the visual content, addressing the image forgery detection and localization (IFDL) problem has drawn serious public concerns. In this work, we propose a simple assumption that the effective forensic method should focus on the mesoscopic properties of images. Base on the assumption, a novel two-stage self-supervised framework leveraging the diffusion model for IFDL task, i.e., DiffForensics, is proposed in this paper. The DiffForensics begins with self-supervised denoising diffusion paradigm equipped with the module of encoder-decoder structure, by freezing the pre-trained encoder (e.g., in ADE-20K) to inherit macroscopic features for general image characteristics, while encour-aging the decoder to learn microscopic feature represen-tation of images, enforcing the whole model to focus the mesoscopic representations. The pre-trained model as a prior, is then further fine-tuned for IFDL task with the customized Edge Cue Enhancement Module (ECEM), which progressively highlights the boundary features within the manipulated regions, thereby refining tampered area local-ization with better precision. Extensive experiments on several public challenging datasets demonstrate the effectiveness of the proposed method compared with other state-of-the-art methods. The proposed DiffForensics could significantly improve the model's capabilities for both accurate tamper detection and precise tamper localization while con-currently elevating its generalization and robustness.

Image-within-image technique aims to establish covert communication by concealing a secret image within a cover image. Compared with traditional steganography algorithms, the security of image-within-image technique has not been rigorously evaluated by steganalysis. Existing attack methods just brutally destroy the container image, resulting in the secret image cannot be revealed by the original decryption model (key). This paper introduces a novel keyless extraction framework, carrying out steganalysis on the container image without destroying it. Our approach utilizes collected pairs of container and revealed images to construct a master key, enabling us to extract secret image from container image without relying on the original key. Remarkably, the master key remains effective for multiple image-within-image techniques simultaneously, even when their encryption and decryption models are re-trained. In addition, we propose a patch-based data augmentation technique to adapt to scenarios with limited training samples, and we design a weighted loss function with three components to further enhance the visual quality of the extracted secret image. All the experiments are conducted on datasets derived from ImageNet, COCO and DIV2k. The results demonstrate that our approach can extract secret images with comparable visual quality to the original ones.

Most previous deepfake detection methods bent their efforts to discriminate artifacts by end-to-end training. However, the learned networks often fail to mine the generic face forgery information efficiently due to ignoring data diversity. In this work, we propose to introduce sample hardness into the training of deepfake detectors via a curriculum learning paradigm. Specifically, we present a novel simple yet effective strategy, named Dynamic Facial Forensic Curriculum (DFFC), which makes the model gradually focus on hard samples during the training. To this end, we propose Dynamic Forensic Hardness (DFH) which integrates the facial quality score and instantaneous instance loss to dynamically measure sample hardness during training. Besides, we present a pacing function to construct data subsets from easy to hard throughout the training process based on DFH. Comprehensive experiments show that DFFC can improve both within- and cross-dataset performance of various kinds of end-to-end deepfake detectors in a plug-and-play manner. It indicates that DFFC can help deepfake detectors learn generic forgery discriminative features more efficiently by exploiting the information from hard samples.

The steganalysis of JPEG images is a crucial area of research. Deep-learning based steganalysis methods have achieved superior detection performance. All methods for JPEG steganalysis rely on residual networks. Although the incorporation of residual connections has enhanced detection performance, it has also led to a notable increase in computational complexity. Furthermore, most of these methods are not complete end-to-end models. In their approaches, traditional hand-crafted filters are employed for image preprocessing. To avoid relying on residual connections and prior knowledge, we propose an end-to-end VGG-style ConvNet. During training, the model utilizes a multi-branch architecture, while it is transformed into a VGG-style ConvNet through structural reparameterization during inference. We conduct extensive experiments on ALASKA KAGGLE dataset and ALASKA II dataset, demonstrating that the proposed method achieves state-of-the-art results in the JPEG domain comparable to other CNN-based steganalyzers such as UCNet and EfficientNet, with clearly better convergence capacity and lower model complexity.

Generating synthetic fake faces, known as pseudo-fake faces, is an effective way to improve the generalization of DeepFake detection. Existing methods typically generate these faces by blending real or fake faces in spatial domain. While these methods have shown promise, they overlook the simulation of frequency distribution in pseudo-fake faces, limiting the learning of generic forgery traces in-depth. To address this, this paper introduces {\em FreqBlender}, a new method that can generate pseudo-fake faces by blending frequency knowledge. Concretely, we investigate the major frequency components and propose a Frequency Parsing Network to adaptively partition frequency components related to forgery traces. Then we blend this frequency knowledge from fake faces into real faces to generate pseudo-fake faces. Since there is no ground truth for frequency components, we describe a dedicated training strategy by leveraging the inner correlations among different frequency knowledge to instruct the learning process. Experimental results demonstrate the effectiveness of our method in enhancing DeepFake detection, making it a potential plug-and-play strategy for other methods.

The trace of double compression can serve as a crucial evidence of image manipulation for forensic investigation. With the ever-increasing popularity of WebP format, a new type of double compression case, WebP-JPEG transcoding, has emerged. However, distinguishing it from two common compression cases, single JPEG (SJPEG) and double JPEG (DJPEG) has not yet been studied. In this paper, we propose a specialized method for the new task. Firstly, a detailed analysis is conducted to reveal the differences in compression artifacts between WebP-JPEG and SJPEG/DJPEG, which manifests in the distributions of$4\times 4$/$8\times 8$DCT coefficients and the high-frequency portions of image spectrum. Then, multi-modality DCT histograms (MMDH) and high-pass-filtered image residuals (HPFIR) are proposed as front-end dual-domain forensic features to expose the above differences. An indispensable part of these features are extracted through a novel frequency-isolation module (FIM), offering additional information based on the derived relationship between$4\times 4$and$8\times 8$DCT coefficients. Finally, a CNN-ViT (Convolutional Neural Network-Vision Transformer) dual-stream network is designed to learn back-end deep features for a reliable detection, where a CNN stream is used to process statistical features in MMDH while a ViT stream to learn spatial correlations in HPFIR. Extensive experimental results demonstrate that the proposed method significantly outperforms state-of-the-art double compression detection methods in distinguishing WebP-JPEG from SJPEG/DJPEG and is more effective in tampering localization. In specific, the proposed method achieves an average detection accuracy of 0.942 for small images of size$128\times 128$.

Recent research has shown that architectures utilizing reinforcement learning (RL) are effective in cost-based image steganography. However, these architectures only learn embedding probabilities rather than costs, and are trained for a specific embedding payload, making it difficult to extend the trained model to serve other payloads. In this paper, we propose a payload-independent cost learning framework using RL called PICO-RL. This framework directly learns universal costs that can be applied to any payload. PICO-RL incorporates an optimal probability approximation (OPA) module that can calculate the required probability map for embedding simulation directly from a learned cost map for any payload, eliminating the need for time-consuming searches for a valid probability scaling parameter. Additionally, PICO-RL uses an advanced steganalysis environment network to provide more effective reward feedback for learning. During RL training, the learned cost maps of different payloads converge and eventually become similar under the OPA constraint, resulting in payload independence. Experimental results demonstrate that a well-trained PICO-RL model, which acts as a universal cost function, defines costs with superior security performance against steganalysis and has better coding compatibility when encoding with practical steganographic codes.

Detection of aligned double Joint Photographic Experts Group (JPEG) compressed images is a crucial area of research within the field of digital image forensics. The detection tasks for aligned double JPEG compression can be categorized into two sub-tasks, namely detecting double JPEG images with the same quantization matrix (DJSQM) or double JPEG images with different quantization matrices (DJDQM). Existing methods for one of these sub-tasks may not be effective for the other. To address this issue, a novel approach is proposed by recompressing both DJDQM and DJSQM using modified quantization coefficients. The perturbation in the recompression process results in a perturbed error image, which is valid for both DJDQM and DJSQM. Subsequently, the relative change rate is used to combine the perturbed error image, the original error image, and the quantization error to derive the interference error and the interference quantization error. The interference error and interference quantization error further expand the difference between single and double compressed images by preserving the general validity of the original image information. Furthermore, the recompression process of DJDQM and DJSQM results in the conversion of truncation and rounding errors at the pixel level, which can be represented by the pixel state map. The pixel state map characterizes the differing transformation relationships between single and double compressed images and provides additional valid features, thereby enhancing the performance of the proposed method. The empirical results demonstrate that the proposed method outperforms existing methods on detecting aligned double JPEG compressed images.

Digital watermarking is a widely adopted authentication technique and one of its primary concerns in practical usage is robustness. However, existing audio watermarking methods face challenges in countering desynchronization attacks and replay attacks, which can easily lead to watermark extraction failure. In this paper, we introduce a learning-based scheme, named DRAW (Dual-decoder-based Robust Audio Watermarking), to overcome the robustness issue. Specifically, a watermark encoder embeds payloads together with synch codes into audio frames with high imperceptibility. For reliable watermark extraction, two separate decoders are designed, one for Fixed Length Synchronization Decoding (FLSD) and the other for Variable Length Payload Decoding (VLPD). The dual decoders are trained with the encoder with different weights in the loss function by considering their different roles for watermark extraction. To better resist attacks, a distortion layer is incorporated in-between the encoder and the decoders to simulate distortion and facilitate end-to-end learning. For the more challenging replay attacks, a pre-trained Replay Attack Simulation Network (RASN) is applied to simplify the simulation of re-recording with background noise and reverberation. Extensive experimental results show that the proposed method can be applied to variable-length audio clips with better auditory quality, and it outperforms state-of-the-art methods in robustness against various kinds of attacks.

The effectiveness of deep learning-based steganalyzers is significantly compromised by adversarial steganography. In response to this challenge, recent efforts have been devoted to identifying distinct traces of adversarial perturbations, yet they have overlooked the inherently adversarial robustness required in steganalyzers. This paper aims to develop a steganalytic model that defends against adversarial steganography by increasing the difficulty of generating adversarial stego images. To achieve this objective, the techniques of learning neighboring feature relationships and self-adversarial adjustment are proposed with three essential modules. The first one, named K-times Dropout Neighboring Feature Transformer (KDNFT), is designed to accept a set of neighboring features obtained by dropout as input. Based on the finding that K-times dropout neighboring features have different distributions for covers and adversarial stegos, KDNFT effectively learns to exploit the relationships among these features for adversarial steganalysis. To facilitate adversarial training, which is an effective way to improve intrinsic robustness, the second module called Pseudo Adversarial Stego Generator (PASG) is proposed to synthesize samples for training. The third module is a Test-time Active Perturbation (TAP) module that adjusts the results of adversarial stego samples close to the decision boundary in a self-adversarial way. Extensive experiments demonstrate that our method achieves improvements in steganalyzing various kinds of adversarial steganographic methods.

The prominent progress in generative models has significantly improved the authenticity of generated faces, raising serious concerns in society. To combat GAN-generated faces, many countermeasures based on Convolutional Neural Networks (CNNs) have been spawned due to their strong learning capabilities. In this paper, we rethink this problem and explore a new approach based on forest models instead of CNNs. Concretely, we describe a simple and effective forest-based method set, termed ForensicsForest Family, to detect GAN-generate faces. The ForensicsForest family is composed of three variants: ForensicsForest, Hybrid ForensicsForest and Divide-and-Conquer ForensicsForest. ForenscisForest is a novel Multi-scale Hierarchical Cascade Forest that takes appearance, frequency, and biological features as input, hierarchically cascades different levels of features for authenticity prediction, and employs a multi-scale ensemble scheme to consider different levels of information comprehensively for further performance improvement. Building upon ForensicsForest, we create Hybrid ForensicsForest, an extended version that integrates the CNN layers into models, to further enhance the efficacy of augmented features. Furthermore, to reduce memory usage during training, we introduce Divide-and-Conquer ForensicsForest, which can construct a forest model using only a portion of training samplings. In the training stage, we train several candidate forest models using the subsets of training samples. Then, a ForensicsForest is assembled by selecting suitable components from these candidate forest models. Our method is validated on state-of-the-art GAN-generated face datasets and compared with several CNN models, demonstrating the surprising effectiveness of our method in detecting GAN-generated faces.

The widespread misuse of advanced image editing tools and deep generative techniques has led to a proliferation of images with altered content in real-life scenarios, often without any discernible traces of tampering. This has created a potential threat to security and credibility of images. Image forgery localization is an urgent technique. In this paper, we propose a novel reinforcement learning-based framework CoDE (Construct Decision-making Environment) that can provide reliable localization result of tampered area in forged images. We model the forgery localization task as a Markov Decision Process (MDP), where each pixel is equipped with an agent that performs Gaussian distribution-based continuous action to iteratively update the respective forgery probability, so as to achieve pixel-level image forgery localization. In order to construct the state transitions within MDP, we propose a twin-flow state encoder to handle the updated state, which consists of the forged image and its corresponding forgery probability map. What’s more, considering that the tampered area is often sparse in practical image tampering scenarios, we design a reward function specifically for these sparse tampered area. This reward function can guide the agent to more effectively learn the optimal strategy for maximizing the cumulative reward. Extensive experiments conducted on a variety of benchmark datasets demonstrate CoDE’s superior localization accuracy and robustness against image degradation caused by transmission through Online Social Networks (OSNs) and various post-processing attacks.

The misuse of Deepfakes to create unauthorized fake facial images and videos poses a growing threat to personal privacy and social stability. Proactive defense algorithms have been proposed to prevent this fraud by injecting adversarial perturbations into facial images. However, these perturbations are sensitive to the lossy compression on online social networks (OSNs). Recent studies have attempted to produce compression resistance by modeling compression at the pixel level. However, accurate modeling is challenging due to the customization of proprietary compression mechanisms by different OSNs. In this paper, we propose a Robust Adversarial Perturbation (DF-RAP) that provides persistent protection for facial images under OSN compression. Specifically, a novel Compression Approximation GAN (ComGAN) is designed to explicitly model OSN compression. The well-trained ComGAN is then incorporated as a sub-module of the target Deepfake model to derive DF-RAP. Furthermore, we reveal a commonality among various OSNs, i.e., that the lossy compression employed tends to destroy perturbations. Based on this, a novel objective-level destruction-aware constraint (DAC) is introduced during ComGAN training. The extensive experimental results show that DF-RAP can effectively protect facial images from Deepfakes under complex OSN compression, especially for OSNs employing more stringent compression. We also investigate the lossy operation mechanisms employed by widely used OSN platforms and build an OSN-transmission dataset based on the CelebA to facilitate future research.

Recent advances in content-adaptive Audio Steganography in Temporal Domain (ASTD) suggest that modification of micro-amplitude samples may compromise its security. To prevent the micro-amplitude samples from being modified, a targeted Large Amplitude First (LAF) rule was adopted in some audio steganographic schemes, e.g., DFR. However, it is observed that the results with LAF rule are often unstable across different datasets, we thus propose a new Micro-Amplitude Suppression (MAS) rule in this paper following the design philosophy of wet paper coding. Unlike DFR where the audio steganographic performance heavily depends on the adopted heuristic filters, we propose to evaluate the embedding cost of cover audio with the Generalized Audio Intrinsic Energy (GAIE), which is obtained by calculating the weighted sum of squared DCT coefficients for each segmented audio clip with carefully designed weights. Extensive experimental results demonstrate that the proposed MAS rule tends to be more general and consistent than the LAF rule, and the proposed GAIE also shows better empirical security performance and audio quality compared to the advanced AAC and DFR_res (a variant of DFR). In addition, by preventing the micro-amplitude samples from being modified, the proposed GAIE_MAS can not only outperform other hand-crafted audio steganographic schemes but also the recently emerged deep learning-based schemes, e.g., IAA.

In recent years, although cost learning methods have made great progress in single-image steganography, its development in batch steganography is relatively slower, which is a more practical communication scenario in the real world. The difficulties are capturing the full view of the image batch and building connections between cost learning and payload allocation by neural networks. To address the issues, this paper proposes a cost learning framework for batch steganography called JoCoP (Joint Cost Learning and Payload Allocation), wherein the policy network is designed to learn the optimal embedding policies for a batch of images via the collaboration between a cost learning module and a payload allocation module. In specific layers of the policy network, in the cost learning module, the intermediate feature maps of embedding costs are extracted for different images independently, which are sent to the payload allocation module. In the payload allocation module, to implement implicit payload allocation, the feature maps corresponding to different images within the same batch are adjusted by an image-wise attention mechanism. Afterwards, these adjusted feature maps are returned to the cost learning module for subsequent feature extraction in the next layer. Owing to the collaboration between the two modules and the batch-level receptive field in the image-wise attention mechanism, the embedding costs and the payload allocation can be jointly optimized in an end-to-end manner. Experimental results show that the proposed JoCoP outperforms existing methods against both single-image steganalyzers and pooled steganalyzers based on feature extraction and convolutional neural networks.

In recent years, audio spoofing detection has received widespread attention for protecting personal privacy and social security. Despite the significant progress achieved in audio single-view spoofing detection, challenges remain with regard to addressing unknown spoofing attacks in realistic scenarios. To solve these challenging problems, in this paper, we introduce a novel audio multi-view spoofing detection framework (AMSDF), whose goal is to capture both intra-view and inter-view cues by measuring correlations within audio multi-view features (i.e., audio-emotion-text) for audio spoofing detection. In general, different view features are inherently interconnected in the real patterns, while they may present unnatural correlations in the spoofing patterns. Therefore, more discriminative cues can be mined by utilizing their complex interactions, which is beneficial to the audio spoofing detection task. To this end, an intra-view graph attention mechanism (IGAM) is first utilized to aggregate each intra-view node within the same view. Subsequently, a heterogeneous graph fusion module (HGFM) is applied to measure correlations within inter-view nodes, which are enhanced with a master node for comprehensive analysis purposes. Finally, a group-based readout scheme (GRS) is designed to capture and preserve the most distinctive cues by leveraging the strengths of different feature sets, thereby effectively distinguishing subtle differences between real and spoofing audio. The experimental results show that our proposed framework can achieve better performance than that of the state-of-the-art methods, especially in realistic scenarios. The code and pre-trained models are available athttps://github.com/ItzJuny/AMSDF.

Recently, deep learning has been widely used in forensics tools to detect and localize forgery images. However, its susceptibility to adversarial attacks highlights the need for the exploration of anti-forensics research. To achieve this, we introduce an innovative and query-efficient black-box anti-forensics framework tailored for the generation of adversarial forgery images. This framework is designed to simulate the query dynamics of online forensic services, utilizing a Markov Decision Process formulation within the paradigm of reinforcement learning. We further introduce a novel reward function, which evaluates the efficacy of attacks based on the disjunction between query results and attack targets. To improve the query efficiency of these attacks, an actor-critic algorithm is employed to maximize cumulative rewards. Empirical findings substantiate the efficacy of our proposed methodology. Specifically, it demonstrates pronounced adversarial effects on a range of prevailing image forgery detectors, while ensuring negligible visually perceptible distortions in the resultant anti-forensics images.

The current state-of-the-art(SOTA) blind watermark embedding method MBRS based on deep learning is less robust to Crop, and additional diffusion layers need to be added for optimization. However, the diffusion layer will make the model less robust to noise other than Crop. Therefore, MBRS which needs to add or delete components is not a practical watermarking framework. Not only that, MBRS is easy to generate chessboard artifacts, resulting in the generated watermark being easy to be detected by the human eye. Therefore, we construct a more generalized watermarking framework and propose an improved blind watermarking method. The method addresses the shortcomings of MBRS by using an improved message processor with sub-pixel convolution layers and low-frequency features and incorporating double discriminators to improve the performance of the network. Extensive experiments show that our method demonstrates superior results compared to the SOTA method.

In this paper, we present the Screenshot Text Forgery Dataset (STFD), which is the first public dataset for the smartphone screenshot text forgery localization task. To address such a task, we propose a novel Screenshot Text Forgery Localization Network (STFL-Net). Specifically, we introduce the OCR (Optical Character Recognition) stream as the complementary of the RGB stream, and propose a novel dual-stream Y-net architecture to collaboratively learn the representations focused on the traces on text regions of the image. Considering the text forgery is often subtle and local, we introduce a multi-teacher knowledge distillation learning strategy for training the STFL-Net, which makes the model less prone to over-fit one specific forgery trace. Comprehensive experimental results on STFD show that our method outperforms several previous methods designed for image forgery localization. We believe that, with our STFD dataset and STFL-Net, more advanced countermeasures against screenshot text forgeries can be developed in the future.

We describe a simple and effective method called ForensicsForest to detect GAN-generate faces. Instead of using the commonly used CNN models, we describe a novel multi-scale hierarchical cascade forest, which takes semantic and frequency features as input, and hierarchically cascades different levels of features for authenticity prediction. We then propose a multi-scale ensemble, which comprehensively considers different levels of information to improve the performance further. Our method is validated on state-of-the-art GAN-generated face datasets in comparison with several CNN models, which demonstrates the surprising effectiveness of our method in detecting GAN-generated faces.

Most of existing deep learning models for image forgery localization rely on a large number of high-quality labeled samples for training. The training procedures are performed off-line and without adaptation to the image under scrutiny. In this letter, we propose to perform run-time learning of the forgery traces from the suspicious image itself. To this aim, a Variational Auto-Encoder (VAE) model is learned to reconstruct small cliques of the suspicious image, and those cliques with anomalous larger reconstruction errors are therein identified as forged. To further enhance performance, Vision Transform (ViT) is employed as the VAE encoder, and multi-modal input information is explored by considering noise inconsistency, high-pass residual inconsistency, and edge discontinuity. Evaluation on widely used benchmark datasets shows our method outperforms existing blind methods by a large margin, and is competitive against approaches that use ground-truth for supervised training.

Public concerns about deepfake face forgery are continually rising in recent years. Most deepfake detection approaches attempt to learn discriminative features between real and fake faces through end-to-end trained deep neural networks. However, the majorities of them suffer from the problem of poor generalization across different data sources, forgery methods, and/or post-processing operations. In this paper, following the simple but effective principle in discriminative representation learning, i.e., towards learning features of intra-consistency within classes and inter-diversity between classes, we leverage a novel transformer-based self-supervised learning method and an effective data augmentation strategy towards generalizable deepfake detection. Considering the differences between the real and fake images are often subtle and local, the proposed method firstly utilizes Self Prediction Learning (SPL) to learn rich hidden representations by predicting masked patches at a pre-training stage. Intra-class consistency clues in images can be mined without deepfake labels. After pre-training, the discrimination model is then fine-tuned via multi-task learning, including a deepfake classification task and a forgery mask estimation task. It is facilitated by our new data augmentation method called Adjustable Forgery Synthesizer (AFS), which can conveniently simulate the process of synthesizing deepfake images with various levels of visual reality in an explicit manner. AFS greatly prevents overfitting due to insufficient diversity in training data. Comprehensive experiments demonstrate that our method outperforms the state-of-the-art competitors on several popular benchmark datasets in terms of generalization to unseen forgery methods and untrained datasets.

We exploit the computational capability of deep convolutional neural network (CNN) architecture and the natural interpretability of the co-occurrence matrix (CM) to learn deep co-occurrence features (DCOFs). The DCOFs represent the statistics of the co-occurrences of pixels thus overcoming the black box nature of traditional deep representation learning while at the same time solving the inherent computational difficulty of CM. We propose a parametric co-occurrence matrix (PCM) model to approximate the CM with multivariate Gaussian functions, and have developed three approaches to decomposing the PCM model into linear and nonlinear operations such that the model can be easily implemented using standard CNN operations and to learn the DCOFs of arbitrary shapes. The CNN implementation of the PCM model, termed PCMCNN, can be used as a standard plugin module of a deep learning system and adaptively learns the DCOFs for downstream applications. We demonstrate the broad applicability of the DCOFs and their effectiveness in fine-grained image classification tasks such as texture classification and GAN (generative adversarial network) image detection. The introduction of the PCMCNN module makes it much more compact and efficient than conventional implementations of deep learning models, achieving comparable classification performances to state of the art methods on a variety of benchmarking datasets with models that are more than 30 folds smaller and 11 times less complex. The small model size for learning the DCOFs makes the new method particularly effective for few shot classification of large number of texture categories where the small number of training samples can easily cause traditional deep learning models to overfit. This work shows the potential benefits of combining the principles of traditional handcrafted features and deep representation learning to take advantage of both for advancing state of the art.

Steganalysis aims to reveal covert communication established via steganography. In the arm race with steganography, steganalysis has evolved from the old-style hand-crafted features set to deep-learning architectures. However, recent studies show that the majority of existing deep steganalysis models have a large amount of redundancy, which leads to a huge waste of storage and computing resources. The existing model compression method cannot flexibly compress the convolutional layer in residual shortcut block so that a satisfactory shrinking rate cannot be obtained. In this paper, we propose STD-NET, an unsupervised deep-learning architecture search approach via hierarchical tensor decomposition for image steganalysis. Our proposed strategy will not be restricted by various residual connections, since this strategy does not change the number of input and output channels of the convolution block. We propose a normalized distortion threshold to evaluate the sensitivity of each involved convolutional layer of the base model to guide STD-NET to compress target network in an efficient and unsupervised approach, and obtain two network structures of different shapes with low computation cost and similar performance compared with the original one. Extensive experiments have confirmed that, on one hand, our model can achieve comparable or even better detection performance in various steganalytic scenarios due to the great adaptivity of the obtained network architecture. On the other hand, the experimental results also demonstrate that our proposed strategy is more efficient and can remove more redundancy compared with previous steganalytic network compression methods.

Recent studies have shown that side-information estimation (SIE) via JPEG image deblocking/restoration is effective in enhancing steganographic security when the side-information of JPEG rounding errors is unavailable. The polarity map of deblocking errors can work well in modulating handcrafted embedding costs. However, it may not be easy to design an optimal deblocking method that is universal to enhance security for all images, and it is unclear how to better utilize the polarity map in modulating statistical model-based embedding costs. To circumvent the difficulty of deblocking method design, we propose an optimal polarity map selection (OPMS) method leveraging existing well-performed deblocking methods. OPMS is designed based on polarity-oriented synthetic stego and minimum feature distance, so that the selected optimal polarity map (OPM) ensures a high security performance to each image. Besides, we propose a statistical model-based modulation method to better exploiting OPM in a quaternary quantized Gaussian embedding (QQGE) model. Through shifting the mean of the distribution, QQGE can derive effective modulated embedding probabilities and reduce the number of modified coefficients without increasing coding complexity. Experimental results demonstrate that the proposed overall steganographic method, called OPMS-QQGE, greatly surpasses existing state-of-the-art SIE-based methods in resisting both CNN-based and feature-based steganalyzers.

Recently, the success of non-additive steganography has demonstrated that asymmetric distortion can remarkably improve security performance compared with symmetric cost functions. However, most of current existing additive steganographic methods are still based on symmetric distortion. In this paper, for the first time we optimize asymmetric distortion for additive steganography and propose an A3C (Asynchronous Advantage Actor-Critic) based steganographic framework, called ReLOAD. ReLOAD is composed of an actor and a critic, where the former guides action selection for pixel-wise distortion modulation, and the latter evaluates the performance of modulated distortion. Meanwhile, a reward function that considers embedding effects is proposed to unify the goal of steganography and reinforcement learning, so that the minimization of embedding effects can be achieved by learning secure policy to maximize total rewards. Statistical analysis shows that compared with non-additive steganography, ReLOAD achieves lower change rates and makes embedding traces more consistent with cover image textures. Comprehensive experiments conducted on both hand-crafted feature-based and deep learning-based steganalyzers show that ReLOAD significantly promotes the state-of-the-art security performance of current additive methods and even outperforms non-additive steganography when the modification distribution gets sparser.

With the rapid development of face forgery techniques, the existing frame-based deepfake video detection methods have fell into a dilemma that frame-based methods may fail when encountering extremely realistic images. To overcome the above problem, many approaches attempted to model the spatio-temporal inconsistency of videos to distinguish real and fake videos. However, current works model spatio-temporal inconsistency by combining intra-frame and inter-frame information, but ignore the disturbance caused by facial motions that would limit further improvement in detection performance. To address this issue, we investigate into long and short range inter-frame motions and propose a novel dynamic difference learning method to distinguish between the inter-frame differences caused by face manipulation and the inter-frame differences caused by facial motions in order to model precise spatio-temporal inconsistency for deepfake video detection. Moreover, we elaborately design a dynamic fine-grained difference capture module (DFDC-module) and a multi-scale spatio-temporal aggregation module (MSA-module) to collaboratively model spatio-temporal inconsistency. Specifically, the DFDC-module applies self-attention mechanism and fine-grained denoising operation to eliminate the differences caused by facial motions and generates long range difference attention maps. The MSA-module is devised to aggregate multi-direction and multi-scale temporal information to model spatio-temporal inconsistency. The existing 2D CNNs can be extended into dynamic spatio-temporal inconsistency capture networks by integrating the proposed two modules. Extensive experimental results demonstrate that our proposed algorithm steadily outperforms state-of-the-art methods by a clear margin in different benchmark datasets.

The diffusion of AI tools capable of generating realistic DeepFakes (DF) videos raises serious threats to face-based biometric recognition systems. For this reason, several detectors based on Deep Neural Networks (DNNs) have been developed to distinguish between real and DF videos. Despite their good performance, these methods suffer from vulnerability to adversarial attacks. In this paper, we argue that it is possible to increase the resilience of DNN-based DF detectors against black-box adversarial attacks by exploiting the temporal information contained in the video. By using such information, in fact, the transferability of adversarial examples from a source to a target model is significantly decreased, making it difficult to launch an attack without accessing the target network. To back this claim, we trained two convolutional neural networks (CNNs) to detect DF videos, and measured their robustness against black-box, transfer-based, attacks. We also trained two detectors by adding to the CNNs a long short-term memory (LSTM) layer to extract temporal information. Then, we measured the transferability of adversarial examples to-wards the LSTM-networks. The results we got suggest that the methods based on temporal information are less prone to black-box attacks.

Existing methods for deepfake face forgery detection have already achieved tremendous progress in well-controlled laboratory conditions. However, under wild scenarios where the training and testing forgeries are synthesized by different algorithms and when labeled data are insufficient, the performance always drops greatly. In this work, we present a Semi-supervised Contrastive Learning and Knowledge Distillation-based framework (SCL-KD) for deepfake detection to reduce the aforementioned performance gap. Our proposed framework contains three stages: self-supervised pre-training, supervised training, and knowledge distillation. Specifically, a feature encoder is firstly trained in a self-supervised manner with a large number of unlabeled samples through a momentum contrastive mechanism. Secondly, a fully-connected classifier on top of the feature encoder is trained in a supervised manner with a small amount of labeled samples to build a teacher model. Finally, a compact student model is trained with the help of the teacher model using knowledge distillation, in order to avoid overfitting to labeled data and have better generalizability on mismatched datasets. Evaluations on several benchmark datasets corroborate the good performance of our approach in cross-dataset situations and few labeled data scenarios. It reveals the potential of our proposed method for real-world deepfake detection.

Swapping faces with deep learning technology to generate realistic fake videos/images (a.k.a, deepfakes) has drawn great public con-cerns recently. Numerous approaches have been proposed to iden-tify fake contents; however, less work has been dedicated to pro-tecting the source faces in an active way. In this paper, we stand for a legitimate faceswap service provider and present an approach called Source-ID- Tracker (SIDT), which aims to protect the identity of source faces in deepfakes from malicious uses. As a plug-in, the encoder of SIDT implicitly embeds a source face image into a deep-fake image while ensuring the resultant encoded image is visually indistinguishable from the deepfake image. After sharing through social media, the embedded source face and its identity can still be recovered with a decoder. Experimental results show that the pro-posed model achieves a promising performance, in terms of reconstruction quality and attribution inference accuracy, in revealing the hidden source face.

Collaborative learning is used in multi-media applications to distribute computing tasks and data storage over multiple sites. Recent studies found that private data information can be derived from model updates between the server and clients. Yet, previous methods are limited by their capabilities of privacy inference in more general and practical situations. In this paper, we propose a novel property inference method in the deep feature space to overcome those limitations. In particular, our method can make inference decisions on the level of individual examples instead of a batch of examples. We can simultaneously perform multiple property inference attacks without the need of image reconstruction. The proposed method is evaluated on several image benchmark datasets, which demonstrates significant improvement of inference accuracy even in the presence of privacy protection schemes.

Gaussian mixture model (GMM) is a powerful statistical tool in data modeling, especially for unsupervised learning tasks. Traditional learning methods for GMM such as expectation maximization (EM) require the covariance of the Gaussian components to be non-singular, a condition that is often not satisfied in real-world applications. This paper presents a new learning method called G$^2$M$^2$ (General Gaussian Mixture Model) by fitting an unnormalized Gaussian mixture function (UGMF) to a data distribution. At the core of G$^2$M$^2$ is the introduction of an integral cosine similarity (ICS) function for comparing the UGMF and the unknown data density distribution without having to explicitly estimate it. By maximizing the ICS through Monte Carlo sampling, the UGMF can be made to overlap with the unknown data density distribution such that the two only differ by a constant scalar, and the UGMF can be normalized to obtain the data density distribution. A Siamese convolutional neural network is also designed for optimizing the ICS function. Experimental results show that our method is more competitive in modeling data having correlations that may lead to singular covariance matrices in GMM, and it outperforms state-of-the-art methods in unsupervised anomaly detection.

Abstract Most of the existing edge aware filters have a number of parameters. The optimal settings of these parameters guarantee the best performance but they depend on the input image. It would be difficult for inexperienced users to empirically get the optimal parameters. This paper proposes a new metric for choosing the optimal parameter settings of edge aware filtering, which is called metric of edge aware filtering (MEAF). MEAF evaluates the quality of filtered images from three aspects: the color distance, the distance of the prominent structure, and smoothness. The colour distance is calculated by rooted mean square error. The distance of the prominent structure is calculated by the proposed SSIM map masked by Sobel edges (MASKED‐SSIM). In MASKED‐SSIM, Sobel detector is used to detect the prominent structure from the input image and the calculation of structure distance is constrained on the prominent structures. Number of gradients and relative total variation are further defined to measure the smoothness of the filtered image. MEAF is an objective metric, which is specially designed for choosing the optimal parameters of edge aware filtering and can be used universally for arbitrary input image. Experiments on 12 state‐of‐the‐art edge aware filters show the effectiveness of MEAF.

Passive video forensics has become an active topic in recent years. Generally, a pristine video obtained from surveillance cameras or other video recording devices is single compressed. At the same time, double lossy compression will certainly be introduced in tampered videos since it needs to go through recompression to perform tampering on a commonly used compressed video. In the video’s double compression, some traces are left due to the intrinsic effects of recompression. Traditional supervised learning is inefficient because it requires two classes (the pristine and the manipulated) that occur in the video to be exhaustively assigned labels. Actually, compared with pristine videos, manipulated videos with labels are more difficult to obtain. To address this problem, in this paper, one-class classification, which is often used for anomaly detection and only needs the target class, is introduced. We first treat all decompressed video frames as still images and extract subtractive pixel adjacency matrix (SPAM) steganalysis features to detect traces left in the double compression process. Then we adopt a Gaussian density-based one-class classifier since SPAM features extracted from pristine video frames approximately subject to the Gaussian distribution. Furthermore, we improve the robustness of the classifier by using ensemble strategy. Experimental results indicate that our proposed method exceeds other more complex one-class classification methods, and outperforms fully-supervised learning methods only by feeding features from single compressed video frames to our one-class classifier.

Compared to element-wise embedding, vector-wise embedding based on CPV (color pixel vector) shows its superiority in color image steganography. However, when working with an adversarial embedding scheme for introducing adversarial perturbations, its success rate of deceiving a target CNN (convolutional neural network) steganalyzer dramatically drops. In this paper, inspired by the I-FGSM (iterative fast gradient sign method), we present an effective steganography for color images. Specifically, after decomposing an image into several non-overlapped sub-images, we iteratively and gradually increase the possibilities of generating adversarial perturbations for the CPVs in each sub-image by changing their adversarial costs. The costs are incrementally adjusted with a small step so that their maximum relative variation is minimized. Leveraging a new designed cost adjustment criterion, more modification patterns of CPV can participate in producing effective adversarial perturbations. Extensive experiments demonstrate that the proposed method achieves a high success rate in deceiving the target CNN steganalyzer and stably defending against the detection of other non-target steganalytic schemes for color images.

Although significant progress has been achieved recently in automatic learning of steganographic cost, the existing methods designed for spatial images cannot be directly applied to JPEG images which are more common media in daily life. The difficulties of migration are mainly caused by the characteristics of the$8\times 8$DCT mode structure. To address the issue, in this paper we extend an existing automatic cost learning scheme to JPEG, where the proposed scheme called JEC-RL (JPEG Embedding Cost with Reinforcement Learning) is explicitly designed to tailor the JPEG DCT structure. It works with the embedding action sampling mechanism under reinforcement learning, where a policy network learns the optimal embedding policies via maximizing the rewards provided by an environment network. Following a domain-transition design paradigm, the policy network is composed of three modules, i.e., pixel-level texture complexity evaluation module, DCT feature extraction module, and mode-wise rearrangement module. These modules operate in serial, gradually extracting useful features from a decompressed JPEG image and converting them into embedding policies for DCT elements, while considering JPEG characteristics including inter-block and intra-block correlations simultaneously. The environment network is designed in a gradient-oriented way to provide stable reward values by using a wide architecture equipped with a fixed preprocessing layer with$8\times 8$DCT basis filters. Extensive experiments and ablation studies demonstrate that the proposed method can achieve good security performance for JPEG images against both advanced feature-based and modern CNN-based steganalyzers.

Image editing techniques enable people to modify the content of an image without leaving visual traces and thus may cause serious security risks. Hence the detection and localization of these forgeries become quite necessary and challenging. Furthermore, unlike other tasks with extensive data, there is usually a lack of annotated forged images for training due to annotation difficulties. In this paper, we propose a self-adversarial training strategy and a reliable coarse-to-fine network that utilizes a self-attention mechanism to localize forged regions in forgery images. The self-attention module is based on a Channel-Wise High Pass Filter block (CW-HPF). CW-HPF leverages inter-channel relationships of features and extracts noise features by high pass filters. Based on the CW-HPF, a self-attention mechanism, calledforgery attention, is proposed to capture rich contextual dependencies of intrinsic inconsistency extracted from tampered regions. Specifically, we append two types of attention modules on top of CW-HPF respectively to model internal interdependencies in spatial dimension and external dependencies among channels. We exploit a coarse-to-fine network to enhance the noise inconsistency between original and tampered regions. More importantly, to address the issue of insufficient training data, we design a self-adversarial training strategy that expands training data dynamically to achieve more robust performance. Specifically, in each training iteration, we perform adversarial attacks against our network to generate adversarial examples and train our model on them. The proposed method is based on the assumption of content-changed manipulations. Extensive experimental results demonstrate that our proposed algorithm steadily outperforms state-of-the-art methods by a clear margin in different benchmark datasets.

Nowadays a steganography has to face challenges to both feature-based staganalysis and convolutional neural network (CNN) based steganalysis. In this paper, we present a novel steganographic scheme to incorporate synchronizing modification directions and iterative adversarial perturbations to enhance steganographic performance. Firstly an existing steganographic function is employed to compute initial costs. Then the secret message bits are embedded following clustering modification directions profile. If the target CNN classifier discriminates the resulting stego image as the correct class, we change costs in adversarial manners, and then choose a sub-image to re-embed message with changed costs. Adversarial intensity will be iteratively increased until the adversarial stego image can deceive the target CNN classifier, which guarantees that applied adversarial perturbations are minimal and it is unnecessary to search the optimal adversarial intensity. Experiments demonstrate that the proposed method effectively enhances security to counter both feature-based classifiers and CNN classifiers, no matter they are targeted or non-targeted.

Abstract Estimating the primary quantization matrix of double JPEG compressed images is a problem of relevant importance in image forensics since it allows to infer important information about the past history of an image. In addition, the inconsistencies of the primary quantization matrices across different image regions can be used to localize splicing in double JPEG tampered images. Traditional model-based approaches work under specific assumptions on the relationship between the first and second compression qualities and on the alignment of the JPEG grid. Recently, a deep learning-based estimator capable to work under a wide variety of conditions has been proposed that outperforms tailored existing methods in most of the cases. The method is based on a convolutional neural network (CNN) that is trained to solve the estimation as a standard regression problem. By exploiting the integer nature of the quantization coefficients, in this paper, we propose a deep learning technique that performs the estimation by resorting to a simil-classification architecture. The CNN is trained with a loss function that takes into account both the accuracy and the mean square error (MSE) of the estimation. Results confirm the superior performance of the proposed technique, compared to the state-of-the art methods based on statistical analysis and, in particular, deep learning regression. Moreover, the capability of the method to work under general operative conditions, regarding the alignment of the second compression grid with the one of first compression and the combinations of the JPEG qualities of former and second compression, is very relevant in practical applications, where these information are unknown a priori.

Recent research has shown that non-additive image steganographic frameworks effectively improve security performance through adjusting distortion distribution. However, as far as we know, all of the existing non-additive proposals are based on handcrafted policies, and can only be applied to a specific image domain, which heavily prevent non-additive steganography from releasing its full potentiality. In this paper, we propose an automatic non-additive steganographic distortion learning framework called MCTSteg to remove the above restrictions. Guided by the reinforcement learning paradigm, we combine Monte Carlo Tree Search (MCTS) and steganalyzer-based environmental model to build MCTSteg. MCTS makes sequential decisions to adjust distortion distribution without human intervention. Our proposed environmental model is used to obtain feedbacks from each decision. Due to its self-learning characteristic and domain-independent reward function, MCTSteg has become the first reported universal non-additive steganographic framework which can work in both spatial and JPEG domains. Extensive experimental results show that MCTSteg can effectively withstand the detection of both hand-crafted feature-based and deep-learning-based steganalyzers. In both spatial and JPEG domains, the security performance of MCTSteg steadily outperforms the state of the art by a clear margin under different scenarios.

Over the past few years, detection performance improvements of deep-learning based steganalyzers have been usually achieved through structure expansion. However, excessive expanded structure results in huge computational cost, storage overheads, and consequently difficulty in training and deployment. In this paper we propose CALPA-NET, a ChAnneL-Pruning-Assisted deep residual network architecture search approach to shrink the network structure of existing vast, over-parameterized deep-learning based steganalyzers. We observe that the broad inverted-pyramid structure of existing deep-learning based steganalyzers might contradict the well-established model diversity oriented philosophy, and therefore is not suitable for steganalysis. Then a hybrid criterion combined with two network pruning schemes is introduced to adaptively shrink every involved convolutional layer in a data-driven manner. The resulting network architecture presents a slender bottleneck-like structure. We have conducted extensive experiments on BOSSBase + BOWS2 dataset, more diverse ALASKA dataset and even a large-scale subset extracted from ImageNet CLS-LOC dataset. The experimental results show that the model structure generated by our proposed CALPA-NET can achieve comparative performance with less than two percent of parameters and about one third FLOPs compared to the original steganalytic model. The new model possesses even better adaptivity, transferability, and scalability.

Automatic cost learning for steganography based on deep neural networks is receiving increasing attention. Steganographic methods under such a framework have been shown to achieve better security performance than methods adopting hand-crafted costs. However, they still exhibit some limitations that prevent a full exploitation of their potentiality, including using a function-approximated neural-network-based embedding simulator and a coarse-grained optimization objective without explicitly using pixel-wise information. In this article, we propose a new embedding cost learning framework called SPAR-RL (Steganographic Pixel-wise Actions and Rewards with Reinforcement Learning) that overcomes the above limitations. In SPAR-RL, an agent utilizes a policy network which decomposes the embedding process into pixel-wise actions and aims at maximizing the total rewards from a simulated steganalytic environment, while the environment employs an environment network for pixel-wise reward assignment. A sampling process is utilized to emulate the message embedding of an optimal embedding simulator. Through the iterative interactions between the agent and the environment, the policy network learns a secure embedding policy which can be converted into pixel-wise embedding costs for practical message embedding. Experimental results demonstrate that the proposed framework achieves state-of-the-art security performance against various modern steganalyzers, and outperforms existing cost learning frameworks with regard to learning stability and efficiency.

The emergence of powerful image editing software has substantially facilitated digital image tampering, leading to many security issues. Hence, it is urgent to identify tampered images and localize tampered regions. Although much attention has been devoted to image tampering localization in recent years, it is still challenging to perform tampering localization in practical forensic applications. The reasons include the difficulty of learning discriminative representations of tampering traces and the lack of realistic tampered images for training. Since Photoshop is widely used for image tampering in practice, this paper attempts to address the issue of tampering localization by focusing on the detection of commonly used editing tools and operations in Photoshop. In order to well capture tampering traces, a fully convolutional encoder-decoder architecture is designed, where dense connections and dilated convolutions are adopted for achieving better localization performance. In order to effectively train a model in the case of insufficient tampered images, we design a training data generation strategy by resorting to Photoshop scripting, which can imitate human manipulations and generate large-scale training samples. Extensive experimental results show that the proposed approach outperforms state-of-the-art competitors when the model is trained with only generated images or fine-tuned with a small amount of realistic tampered images. The proposed method also has good robustness against some common post-processing operations.

Bitrate is generally regarded as an important criterion of video quality. However, with sophisticated video editing software, forgers can create fake bitrate videos by up-converting the bitrate of original videos with lower video quality to attract more viewers on video sharing websites. In this work, we first model the generation process of fake bitrate videos and analyze the dominant sources of information loss. It is found that the recompression error generated by the proposed one-step-further recompression operation is an efficient measurement to expose distinguishable quality variation tendencies between true and fake bitrate videos. Based on this analysis, we propose a detection method for fake bitrate videos using a hybrid deep-learning network from recompression error. For an input video, the patch-wise recompression errors are first calculated to increase the learning capability of the network. To learn robust representations of recompression errors in local regions with different degrees of predictability, a hybrid deep-learning network that contains two branches with heterogeneous structures is designed. For noise-like recompression errors, the first branch has a shallow CNN structure initialized with an Inception-like module using multisize convolutional kernels. For zero-element clustered recompression errors, the second branch has a multi-layer perceptron structure equipped with a unique layer that extracts the histogram of zero-element clustered square regions. The output vectors of different branches are concatenated and then jointly optimized to obtain the patch-wise detection results. Finally, the majority voting (local-to-global) strategy is applied to obtain the final detection result. Extensive experiments are conducted to evaluate the detection performance under various coding parameter settings, such as different bitrates, rate-distortion optimization strategies and so on. The experimental results demonstrate the superiority of the proposed method compared with several state-of-the-art methods to provide more fine-grained forensic clues.

In traditional steganographic schemes, RGB three channels payloads are assigned equally in a true color image. In fact, the security of color image steganography relates not only to data-embedding algorithms but also to different payload partition. How to exploit inter-channel correlations to allocate payload for performance enhancement is still an open issue in color image steganography. In this paper, a novel channel-dependent payload partition strategy based on amplifying channel modification probabilities is proposed, so as to adaptively assign the embedding capacity among RGB channels. The modification probabilities of three corresponding pixels in RGB channels are simultaneously increased, and thus the embedding impacts could be clustered, in order to improve the empirical steganographic security against the channel co-occurrences detection. The experimental results show that the new color image steganographic schemes, incorporated with the proposed strategy, can effectively make the embedding changes concentrated mainly in textured regions, and achieve better performance on resisting the modern color image steganalysis.

Identifying whether a phone call comes from VoIP (Voice over Internet Protocol) is a challenging but less-investigated audio forensic issue. As shown in a previous study, existing feature based methods do not work well. In this paper, we propose a robust data-driven approach, called CNN-MLS (convolutional neural network based multi-domain learning scheme), to distinguish VoIP calls from mobile phone calls. To better explore the differences between VoIP and mobile phone calls, we first process data with high-pass filtering, and then extract deep features from both temporal domain and spectral domain. Two CNN architectures are designed for accepting data from respective domains, and some tricks such as auxiliary classifiers and individual subnet training are used for accelerating network convergence. The deep features are finally fused in a classification module for identifying the phone call type. The proposed method is evaluated on VPCID (VoIP Phone Call Identification Database) dataset, under various testing conditions. We pay particular attention to tests on data belonging to a source mismatched with the training sources. Experimental results show that, compared with existing methods, our method can achieve satisfactory and better accuracy on two-second-long inputs, implying that an alert may be activated shortly after a VoIP call is made.

The emergence of the high-efficiency video coding (HEVC) standard enables people to enjoy high definition (HD) video content; meanwhile, HD videos, tamper detection has become a crucial issue and gradually aroused people's attention. The detection of double HEVC compressed videos with the same coding parameters is challenging since the recompression traces are inconspicuous. To deal with this issue, a novel method based on the intra prediction mode is proposed in this paper. First, the quality degradation mechanism is analyzed to facilitate the selection of classification features and the source of error in intra coding is fully considered to establish the equivalent error model. Second, the feature model of double HEVC compression detection, which is mainly based on the statistical feature of intra prediction mode, is proposed. Finally, the experiment is carried out in 720p and 1080p HEVC videos instead of low-resolution (CIF or QCIF) videos. Experimental results have demonstrated better efficiency of the proposed method in comparison to the state-of-the-art methods. Besides, the proposed method is more robust to various encoding configurations.

Embedding costs used in content-adaptive image steganographic schemes can be defined in a heuristic way or with a statistical model. Inspired by previous steganographic methods, i.e., MG (multivariate Gaussian model) and MiPOD (minimizing the power of optimal detector), we propose a model-driven scheme in this paper. Firstly, we model image residuals obtained by high-pass filtering with quantized multivariate Gaussian distribution. Then, we derive the approximated Fisher Information (FI). We show that FI is related to both Gaussian variance and filter coefficients. Lastly, by selecting the maximum FI value derived with various filters as the final FI, we obtain embedding costs. Experimental results show that the proposed scheme is comparable to existing steganographic methods in resisting steganalysis equipped with rich models and selection-channel-aware rich models. It is also computational efficient when compared to MiPOD, which is the state-of-the-art model-driven method.

Steganographic schemes are commonly designed in a way to preserve image statistics or steganalytic features. Since most of the state-of-the-art steganalytic methods employ a machine learning (ML)-based classifier, it is reasonable to consider countering steganalysis by trying to fool the ML classifiers. However, simply applying perturbations on stego images as adversarial examples may lead to the failure of data extraction and introduce unexpected artifacts detectable by other classifiers. In this paper, we present a steganographic scheme with a novel operation called adversarial embedding (ADV-EMB), which achieves the goal of hiding a stego message while at the same time fooling a convolutional neural network (CNN)-based steganalyzer. The proposed method works under the conventional framework of distortion minimization. In particular, ADV-EMB adjusts the costs of image elements modifications according to the gradients back propagated from the target CNN steganalyzer. Therefore, modification direction has a higher probability to be the same as the inverse sign of the gradient. In this way, the so-called adversarial stego images are generated. Experiments demonstrate that the proposed steganographic scheme achieves better security performance against the target adversary-unaware steganalyzer by increasing its missed detection rate. In addition, it deteriorates the performance of other adversary-aware steganalyzers, opening the way to a new class of modern steganographic schemes capable of overcoming powerful CNN-based steganalysis.

Until recently, deep steganalyzers in the spatial domain have been all designed for gray-scale images. In this paper, we propose the wider separate-then-reunion network (WISERNet) for steganalysis of color images. We provide theoretical rationale to claim that the summation in normal convolution is one sort of linear collusion attack which reserves strong correlated patterns while impairs uncorrelated noises. Therefore, in the bottom convolutional layer which aims at suppressing correlated image contents, we adopt separate channel-wise convolution without summation instead. Conversely, in the upper convolutional layers, we believe that the summation in normal convolution is beneficial. Therefore, we adopt united normal convolution in those layers and make them remarkably wider to reinforce the effect of linear collusion attack. As a result, our proposed wide-and-shallow, separate-then-reunion network structure is specifically suitable for color image steganalysis. We have conducted extensive experiments on color image datasets generated from BOSSBase raw images and another large-scale dataset that contains 100, 000 raw images, with different demosaicking algorithms and down-sampling algorithms. The experimental results show that our proposed network outperforms other state-of-the-art color image steganalytic models either hand crafted or learned using deep networks in the literature by a clear margin. Specifically, it is noted that the detection performance gain is achieved with less than half the complexity compared to the most advanced deep-learning steganalyzer as far as we know, which is scarce in the literature.

Recent steganalytic schemes reveal embedding traces in a promising way by using convolutional neural networks (CNNs). However, further improvements, such as exploring complementary data processing operations and using wider structures, were not extensively studied so far. In this letter, we design a new CNN in these aspects in order to better capture embedding artifacts. Specifically, on the one hand, we propose to process information diversely with a module called diverse activation module. On the other hand, we build a wide structure with parallel subnets using several filter groups for preprocessing. To accelerate the training process, we pretrain the subnets independently. Extensive experiments show that the proposed method is effective in detecting content-adaptive steganographic schemes.

Research on adaptive steganography mainly focuses on how to design a reasonable cost function and how to utilize that cost function to achieve embedding in a stego image with the minimal distortion based on syndrome-trellis codes. Because previous adaptive steganographic methods use convolution with filters to obtain the residuals, these methods do not make good use of the textures of the image itself in the design of the cost function. In this letter, we define a new cost function that uses nonnegative matrix factorization to predict the image pixels and utilizes the mutual dependencies among the pixels to calculate the costs. We present a novel cost function in which the residuals are not calculated via convolution with constant filters. Experimental results show that our method outperforms the state-of-the-art MiPOD, spatial universal wavelet relative distortion, wavelet obtained weights, and HUGO-BD methods in resisting steganalysis based on the spatial rich model and is slightly superior to the high-pass, low-pass, low-pass method.

The standard local binary pattern (LBP) operator shows its versatility in performing image classification-related tasks, including texture analysis, object recognition, and steganalysis. However, a conventional well-designed scheme utilizing LBP operator and histogram-based features does not have obvious advantage when compared with the well-known steganalytic scheme spatial rich model (SRM). In this paper, we propose an adapted LBP version, called threshold LBP (TLBP), to reveal the artifacts caused by data embedding. In the proposed steganalytic scheme, the TLBP operation is performed on residual images which are obtained by using a set of high-order derivative filters to capture intricate relationships among pixels. After performing TLBP operation, second order co-occurrence matrix features are formed and then processed with aggregation and non-linear mapping for boosting feature effectiveness. Experimental results show that the proposed TLBP features prevail over SRM features under various steganographic conditions.

Adoption of deep learning in image steganalysis is still in its initial stage. In this paper, we propose a generic hybrid deep-learning framework for JPEG steganalysis incorporating the domain knowledge behind rich steganalytic models. Our proposed framework involves two main stages. The first stage is hand-crafted, corresponding to the convolution phase and the quantization and truncation phase of the rich models. The second stage is a compound deep-neural network containing multiple deep subnets, in which the model parameters are learned in the training procedure. We provided experimental evidence and theoretical reflections to argue that the introduction of threshold quantizers, though disabling the gradient-descent-based learning of the bottom convolution phase, is indeed cost-effective. We have conducted extensive experiments on a large-scale data set extracted from ImageNet. The primary data set used in our experiments contains 500 000 cover images, while our largest data set contains five million cover images. Our experiments show that the integration of quantization and truncation into deep-learning steganalyzers do boost the detection performance by a clear margin. Furthermore, we demonstrate that our framework is insensitive to JPEG blocking artifact alterations, and the learned model can be easily transferred to a different attacking target and even a different data set. These properties are of critical importance in practical applications.

Generative adversarial network has shown to effectively generate artificial samples indiscernible from their real counterparts with a united framework of two subnetworks competing against each other. In this letter, we first propose an automatic steganographic distortion learning framework using a generative adversarial network, which is composed of a steganographic generative subnetwork and a steganalytic discriminative subnetwork. Via alternately training these two oppositional subnetworks, our proposed framework can automatically learn embedding change probabilities for every pixel in a given spatial cover image. The learnt embedding change probabilities can then be converted to embedding distortions, which can be adopted in the existing framework of minimal-distortion embedding. Under this framework, the distortion function is directly related to the undetectability against the oppositional evolving steganalyzer. Experimental results show that with adversarial learning, our proposed framework can effectively evolve from nearly naive random ±1 embedding at the beginning to much more advanced content-adaptive embedding which tries to embed secret bits in textural regions. The security performance is also steadily improved with increasing training iterations.

The adaptive multi-rate (AMR) audio codec adopted by many portable recording devices is widely used in speech compression. The use of AMR speech recordings as evidence in court is growing. Nowadays, it is easy to tamper with digital speech recordings, which makes audio forensics increasingly important. The detection of double compressed audio is one of the key issues in audio forensics. In this paper, we propose a framework for detecting double compressed AMR audio based on the stacked autoencoder (SAE) network and the universal background model-Gaussian mixture model (UBM-GMM). Instead of hand-crafted features, we used the SAE to learn the optimal features automatically from the audio waveforms. Audio frames are used as network input and the last hidden layer's output constitutes the features of a single frame. For an audio clip with many frames, the features of all the frames are aggregated and classified by UBM-GMM. Experimental results show that our method is effective in distinguishing single/double compressed AMR audio and outperforms the existing methods by achieving a detection accuracy of 98% on the TIMIT database. Exhaustive experiments demonstrate the effectiveness and robustness of the proposed method.

This paper deals with the state-of-the-art synchronize-embedding-changes (SECs) steganography. We propose the pixel-decimation-assisted steganalytic feature set, a novel feature set construction protocol that extends upon the recent selection-channel-aware spatial rich model maxSRMd2. Our method is based on pixel decimation, a specific type of image downsampling. Based on theoretical analysis and empirical evaluation, we clearly demonstrate that our method impairs the synchronization of embedding changes in SEC steganography, and improves the accuracy of embedding change probability estimation. Our method significantly improves stego image detection performance when extended from a selection-channel-aware rich-model feature set (maxSRMd2) and is robust to different image downsampling methods. Furthermore, increasing the number of sweeps in SEC steganography has no effect to the performance of our proposed method even though it further strengthens synchronization of embedding changes. It is worth noting that with ensemble classifier, the above-mentioned performance improvements are achieved at a little extra cost.

It is conventionally assumed that steganographic schemes for gray-scale images can be directly applied to color images by embedding messages independently in different color channels. However, the correlation among color channels may be disturbed and it is unclear how to preserve the channel correlation so as to increase empirical security. In this paper, we propose a strategy called CMD-C (clustering modification directions for color components). The basic idea of the strategy is to change different color components from the same pixel location towards a positive or negative direction consistently. To implement the strategy, we decompose an image into several sub-images in which segmented hidden message bits are successively embedded. The embedding costs of a sub-image are computed by considering the correlation both within and among color channels. Experimental results show that the proposed CMD-C strategy has made great improvement over conventional methods in resisting state-of-the-art steganalytic methods.

Passive multimedia forensics has become an active topic in recent years. However, less attention has been paid to video forensics. Research on video forensics, and especially on automatic detection of object-based video forgery, is still in its infancy. In this paper, we develop an approach for automatic identification and forged segment localization of object-based forged video encoded with advanced frameworks. The proposed approach starts with a frame manipulation detector. An automatic algorithm is proposed to identify object-based video forgery based on the frame manipulation detector. Then, a two-stage automatic algorithm is provided to accurately locate the forged video segments in the suspicious video. To construct the proposed frame manipulation detector, motion residuals are generated from the target video frame sequence. We regard the object-based forgery in video frames as image tampering in the motion residuals and employ the feature extractors that are originally built for still image steganalysis to extract forensic features from the motion residuals. The experiments show that the proposed approach achieves excellent results in both forged video identification and automatic forged temporal segment localization.

To identify whether an image has been JPEG compressed is an important issue in forensic practice. The state-of-the-art methods fail to identify high-quality compressed images, which are common on the Internet. In this paper, we provide a novel quantization noise-based solution to reveal the traces of JPEG compression. Based on the analysis of noises in multiple-cycle JPEG compression, we define a quantity called forward quantization noise. We analytically derive that a decompressed JPEG image has a lower variance of forward quantization noise than its uncompressed counterpart. With the conclusion, we develop a simple yet very effective detection algorithm to identify decompressed JPEG images. We show that our method outperforms the state-of-the-art methods by a large margin especially for high-quality compressed images through extensive experiments on various sources of images. We also demonstrate that the proposed method is robust to small image size and chroma subsampling. The proposed algorithm can be applied in some practical applications, such as Internet image classification and forgery detection.

Most of the recently proposed steganographic schemes are based on minimizing an additive distortion function defined as the sum of embedding costs for individual pixels. In such an approach, mutual embedding impacts are often ignored. In this paper, we present an approach that can exploit the interactions among embedding changes in order to reduce the risk of detection by steganalysis. It employs a novel strategy, called clustering modification directions (CMDs), based on the assumption that when embedding modifications in heavily textured regions are locally heading toward the same direction, the steganographic security might be improved. To implement the strategy, a cover image is decomposed into several subimages, in which message segments are embedded with well-known schemes using additive distortion functions. The costs of pixels are updated dynamically to take mutual embedding impacts into account. Specifically, when neighboring pixels are changed toward a positive/negative direction, the cost of the considered pixel is biased toward the same direction. Experimental results show that our proposed CMD strategy, incorporated into existing steganographic schemes, can effectively overcome the challenges posed by the modern steganalyzers with high-dimensional features.

In this paper, we present a statistical analysis of JPEG noises, including the quantization noise and the rounding noise during a JPEG compression cycle. The JPEG noises in the first compression cycle have been well studied; however, so far less attention has been paid on the statistical model of JPEG noises in higher compression cycles. Our analysis reveals that the noise distributions in higher compression cycles are different from those in the first compression cycle, and they are dependent on the quantization parameters used between two successive cycles. To demonstrate the benefits from the analysis, we apply the statistical model in JPEG quantization step estimation. We construct a sufficient statistic by exploiting the derived noise distributions, and justify that the statistic has several special properties to reveal the ground-truth quantization step. Experimental results demonstrate that the proposed estimator can uncover JPEG compression history with a satisfactory performance.

A well defined cost function is crucial to steganography under the scenario of minimizing embedding distortion. In this paper, we present a new cost function for spatial image steganography. The proposed cost function is designed by using a high-pass filter to locate the less predictable parts in an image, and then using two low-pass filters to make the low cost values more clustered. Experiments show that the steganographic method with the proposed cost function makes the embedding changes more concentrated in texture regions, and thus achieves a better performance on resisting the state-of-the-art steganalysis over prior works, including HUGO, WOW, and S-UNIWARD.

Relating the embedding cost in a distortion function to statistical detectability is an open vital problem in modern steganography. In this paper, we take one step forward by formulating the process of cost assignment into two phases: 1) determining a priority profile and 2) specifying a cost-value distribution. We analytically show that the cost-value distribution determines the change rate of cover elements. Furthermore, when the cost-values are specified to follow a uniform distribution, the change rate has a linear relation with the payload, which is a rare property for content-adaptive steganography. In addition, we propose some rules for ranking the priority profile for spatial images. Following such rules, we propose a five-step cost assignment scheme. Previous steganographic schemes, such as HUGO, WOW, S-UNIWARD, and MG, can be integrated into our scheme. Experimental results demonstrate that the proposed scheme is capable of better resisting steganalysis equipped with high-dimensional rich model features.

Histogram shifting (HS) is a useful technique of reversible data hiding (RDH). With HS-based RDH, high capacity and low distortion can be achieved efficiently. In this paper, we revisit the HS technique and present a general framework to construct HS-based RDH. By the proposed framework, one can get a RDH algorithm by simply designing the so-called shifting and embedding functions. Moreover, by taking specific shifting and embedding functions, we show that several RDH algorithms reported in the literature are special cases of this general construction. In addition, two novel and efficient RDH algorithms are also introduced to further demonstrate the universality and applicability of our framework. It is expected that more efficient RDH algorithms can be devised according to the proposed framework by carefully designing the shifting and embedding functions.

The adaptive pixel-value differencing steganography proposed by Luo et al. is a state-of-the-art content-adaptive steganographic method which resists blind steganalytic attacks. In this paper, the authors point out that the combination of rotate operation and ternary embedding units in the adaptive pixel-value differencing steganography introduces intrinsic statistical imbalance which can be used to construct a targeted steganalytic algorithm. Experimental results reveal that the proposed method can obtain excellent results for detecting stego images even when the embedding rate is low.

In this letter, the authors point out that the readjusting phase of edge adaptive image steganography based on LSB matching revisited introduces a pulse distortion to the long exponential tail of the histogram of the absolute difference of the pixel pairs. Making use of this observation, a targeted steganalytic method based on B-Spline fitting is proposed. Experimental results show that the proposed method obtains excellent results for detecting stego images with low embedding rate. The dominant performance of our method compared with state-of-the-art blind steganalyzers, such as SPAM and SRM is apparent. Furthermore, our method can accurately estimate the threshold used in the secret data embedding procedure and can separate the stego images with unit block size from those with block sizes greater than one.

A promising steganographic method-yet another steganography scheme (YASS)-was designed to resist blind steganalysis via embedding data in randomized locations. In addition to a concrete realization which is named the YASS algorithm in this paper, a few strategies were proposed to work with the YASS algorithm in order to enhance the data embedding rate and security. In this work, the YASS algorithm and these strategies, together referred to as YASS, have been analyzed from a warden's perspective. It is observed that the embedding locations chosen by YASS are not randomized enough and the YASS embedding scheme causes detectable artifacts. We present a steganalytic method to attack the YASS algorithm, which is facilitated by a specifically selected steganalytic observation domain (SO-domain), a term to define the domain from which steganalytic features are extracted. The proposed SO-domain is not exactly, but partially accesses, the domain where the YASS algorithm embeds data. Statistical features generated from the SO-domain have demonstrated high effectiveness in detecting the YASS algorithm and identifying some embedding parameters. In addition, we discuss how to defeat the above-mentioned strategies of YASS and demonstrate a countermeasure to a new case in which the randomness of the embedding locations is enhanced. The success of detecting YASS by the proposed method indicates a properly selected SO-domain is beneficial for steganalysis and confirms that the embedding locations are of great importance in designing a secure steganographic scheme.

In this paper, we present a new universal steganalysis scheme to effectively attack some recently proposed JPEG steganography. Different from the other steganalyzers, not only the magnitude but also the sign dependencies existed in intra-block and inter-block quantized DCT (discrete cosine transform) coefficients are exploited by the Markov empirical transition matrices. Moreover, a new microscopic and macroscopic calibration method is proposed to calibrate the local and global distribution of the quantized DCT coefficients of the test image, thus improve the detecting performance. Experimental results demonstrate that our proposed scheme outperforms some existing steganalyzers in attacking the advanced JPEG steganography such as F5, MB1 and Outguess.

In this paper, we utilize the probabilities of the first digits of quantized DCT (Discrete Cosine Transform) coefficients from individual AC (Alternate Current) modes to detect doubly compressed JPEG images. Our proposed features, named by Mode Based First Digit Features (MBFDF), have been shown to outperform all previous methods on discriminating doubly compressed JPEG images from singly compressed JPEG images. Furthermore, combining the MBFDF with a multi-class classification strategy can be exploited to identify the quality factor in the primary JPEG compression, thus successfully revealing the double JPEG compression history of a given JPEG image.

This letter presents a method for attacking multiple-base notational system (MBNS) steganography . In the MBNS steganography, secret data are converted into symbols in a notational system with multiple bases. The pixels of a host image are then modified such that when the pixel values are divided by the bases, their remainders are equal to the symbols. Through analysis, we prove that the amount of small remainders increases due to the modification. Based on this observation, we propose a steganalytic approach which is effective in not only detecting MBNS steganography but also estimating its embedding rate.

In this paper, we propose a new method for attacking the LSB (least significant bit) matching based steganography. Different from the LSB substitution, the least two or more significant bit-planes of the cover image would be changed during the embedding in LSB matching steganography and thus the pairs of values do not exist in stego image. In our proposed method, we get an image by combining the least two significant bit-planes and divide it into 3x3 overlapped subimages. The subimages are grouped into four types, i.e.T1,T2,T3andT4according to the count of gray levels. Via embedding a random sequence by LSB matching and then computing the alteration rate of the number of elements inT1, we find that normally the alteration rate is higher in cover image than in the corresponding stego image. This new finding is used as the discrimination rule in our method. Experimental results demonstrate that the proposed algorithm is efficient to detect the LSB matching stegonagraphy on uncompressed gray scale images.

A recently developed LSB greedy embedding algorithm for JPEG images is capable of resisting the chi-square attack. By carefully studying the quantized DCT (discrete cosine transform) coefficients of the cover and stego images, we find that the embedding algorithm does not preserve the histogram of the DCT coefficients well. In this paper, we define a new chi-square statistic which is used to measure whether the image under scrutiny is like the cover or the stego. Our proposed steganalytic method is based on the symmetry property of the DCT coefficients in JPEG images. It can also be used in the scenario where the cover images are double JPEG compressed. The reliability of this specific steganalytic scheme depends on the embedding rate and it is influenced by the JPEG quality factor. Experimental results show that when the embedding rate exceeds half of the maximal embedding capacity, the steganographic algorithm is detectable with a very low false negative rate, whatever the quality factor is.