Unauthorized screen capturing and dissemination pose severe security threats such as data leakage and information theft. Several studies propose robust watermarking methods to track the copyright of Screen-Camera (SC) images, facilitating post-hoc certification against infringement. These techniques typically employ heuristic mathematical modeling or supervised neural network fitting as the noise layer, to enhance watermarking robustness against SC. However, both strategies cannot fundamentally achieve an effective approximation of SC noise. Mathematical simulation suffers from biased approximations due to the incomplete decomposition of the noise and the absence of interdependence among the noise components. Supervised networks require paired data to train the noise-fitting model, and it is difficult for the model to learn all the features of the noise. To address the above issues, we propose Simulation-to-Real (S2R). Specifically, an unsupervised noise layer employs unpaired data to learn the discrepancy between the modeled simulated noise distribution and the real-world SC noise distribution, rather than directly learning the mapping from sharp images to real-world images. Learning this transformation from simulation to reality is inherently simpler, as it primarily involves bridging the gap in noise distributions, instead of the complex task of reconstructing fine-grained image details. Extensive experimental results validate the efficacy of the proposed method, demonstrating superior watermark robustness and generalization compared to state-of-the-art methods.

Reversible data hiding in JPEG images is critical for secure multimedia applications, while, existing histogram modification schemes are confined to fixed 1D or 2D dimensions. These methods fail to adapt to the characteristics of cover images and the demands of embedded data, resulting in suboptimal trade-offs among embedding capacity, image quality, and file size. To address these issues, this study proposes a Multi-Dimensional Histogram Modification Framework that advances histograms from "fixed dimension" to "variable dimension" and extends them to high-dimensional structures. Our key contributions are as follows: first, we propose an adaptive dimension selection strategy that dynamically determines the optimal histogram dimension by evaluating the capacity and distortion for each cover image and embedding task; second, we develop a high-dimensional histogram construction method that combines AC coefficients with absolute values of 1 and 2 at the same frequency to enhance the utilization of coefficient correlation; third, we design an optimized mapping algorithm that searches for the optimal mapping matrix to minimize both distortion and file expansion. Experimental results show that the proposed method outperforms existing 1D and 2D histogram modification schemes. On average, it increases the embedding capacity by 5000 bits, improves the image quality by 0.2 dB, and reduces the file size by 4000 bits.

Image manipulation localization (IML) aims to segment manipulated regions in suspicious images. However, most existing methods rely solely on intrinsic features extracted from the input image and passively model local or global inconsistencies, making it difficult to accurately delineate manipulated regions with ambiguous boundaries. To address these challenges, we propose a prototype memory-based neighboring feature fusion network (PNF-Net), which is inspired by a biological memory mechanism. PNF-Net simulates selective preference by learning manipulation-trace prototypes as memory priors, thereby guiding representation learning toward consistent and discriminative manipulation cues. Specifically, we propose a memory-guided localization module (MLM) that models the consistencies and anomalies between manipulated regions and the background as memory priors, enabling precise localization. We then propose a neighboring feature interaction module (NFIM) that preserves fine-grained details from neighboring shallow features, enhances global semantics from neighboring deep features, and effectively fuses them. Finally, a verification fusion module (VFM) is designed to enrich contextual semantics and improve the completeness and accuracy of localization results. Extensive experiments on multiple benchmark datasets show that our PNF-Net outperforms most state-of-the-art IML models. Our code is available on https://github.com/vpsg-research/PNF-Net.

With the rapid development of generative models, the visual quality of generated images has become almost indistinguishable from real images, which poses a huge challenge to content authenticity verification. A key limitation of existing detectors is their reliance on model-specific cues, resulting in poor generalization to unseen models. Based on the observation of local differences in the generated images, we found that the generated images lack device-specific sensor noise and unnatural pixel intensity variations caused by the oversimplified generation process. These discrepancies provide important forensic cues for distinguishing between real and generated images. We propose the Feature Aggregation for Localized Context and Noise Network (FALCON-Net), which leverages these discrepancies to enhance detection capabilities. FALCON-Net integrates two complementary modules to enhance detection capabilities: the Intrinsic Noise Pattern Isolation (INP) module isolates device-specific noise patterns by analyzing high-frequency features in the frequency domain, while the Local Variation Pattern (LVP) module models the complex relationships between local pixels to capture directional intensity variations and reveal unnatural regularities in generated images. By combining these sensor-level and local structural cues, FALCON-Net identifies fundamental generative inconsistencies, ensuring robustness to post-processing and strong generalization to unseen models. Extensive experimental results show that FALCON-Net achieves the state-of-the-art performance in detecting generated images and shows good generalization ability to unseen generative models. The code is available at https://github.com/humiaomiaohaha/FALCON-Net.

Most existing steganographic techniques are primarily designed for grayscale images.When directly applied to color images without considering inter-channel color interactions, their security can be significantly compromised.In this paper, we propose a novel color image steganography method based on generative adversarial networks (GANs).Our framework features a dual-branch generator and a discriminator equipped with multiple steganalytic networks, each focused on a specific color channel.This architecture enables the progressive learning of asymmetric embedding costs across channels from scratch.We also introduce a phased training strategy that facilitates the learning of inter-channel color interactions and optimizes the security of each color channel in distinct phases, improving overall security.Furthermore, we propose a new adaptive update strategy and introduce the Mean Absolute Deviation (MAD) loss function to maintain a dynamic balance between the generator and discriminator, thereby progressively enhancing the generator's steganographic performance during training.Extensive comparative experiments demonstrate that the proposed method achieves state-of-the-art security against color image steganalysis.Comprehensive ablation studies further validate the effectiveness and rationale behind our approach.

Screen-shooting watermarking is an effective means of protecting screen content from unauthorized capture and illegal dissemination. However, existing methods are primarily designed for full-image capture, making them ineffective for partial screen-shooting prevalent in real-world scenarios. To address this limitation, we propose FPSMark, a flexible watermarking method tailored for partial screen-shooting that embeds consistent watermarks in multiple uniformly distributed cover blocks. Specifically, considering that robustness requirements vary according to the layout of each image, we model the mathematical relationship between the watermark block count and robustness, proving the flexibility of FPSMark in ensuring partial screen-shooting robustness. Moreover, partial screen-shooting disrupts watermark synchronization, posing challenges for precise watermark localization. To overcome this, we design an intrinsic signal localization network optimized with a hybrid loss. The localization network exploits the inherent distinctions between the watermark and non-watermark features, while the hybrid loss constrains the network at three dimensions: pixel-level, region-level, and sample-level. Experimental results demonstrate the superiority of FPSMark, showing robust performance across partial capture percentages. Its extraction accuracy exceeds 98% even with only half of the image captured, and it achieves 82% accuracy at a 40% capture ratio, whereas existing methods achieve only around 50% under the same conditions.

The demand for resilient watermarking technology in the context of the screen-shooting scenario is steadily on the rise. The principal objective of this technique is to embed messages into the cover image, with the ability to effectively recover the message from the screen-captured image at the extraction end. However, current watermarking methods result in low visual quality watermarked images and are insufficiently robust in screen-shooting scenarios. This is mainly because they only utilize spatial domain information during embedding, and they do not consider the impact of noise that introduced during screen capturing. This paper introduces an innovative network framework, including the wavelet domain concatenation and recovery mechanism, to overcome the dual challenges encountered in robust watermarking, namely visual fidelity and robustness. For fidelity, we present a cascade network operating in the wavelet domain. This network excel at detecting watermark information in the wavelet domain. This capability makes it more sensitive to high and low-frequency details. Discrete wavelet transform can make CNN focus on different frequency characteristics, and the use of discrete inverse wavelet transform in upsampling can make the information high fidelity. As a result, it can more accurately identify and preserve critical visual details in this frequency domain, leading to an overall enhancement in visual quality. For robustness, a recovery network is specifically designed to mitigate the influence of noise introduced during screen-shooting on watermark information extraction. Experimental validation of our proposed method substantiates its effectiveness in significantly enhancing the visual quality and the accuracy of the watermarked images.

Interpretable deepfake detection is gaining attention for providing explainable, trustworthy results, avoiding the limitations of ‘black-box’ models. Current interpretable methods focus on visible artifacts in low-visual-quality deepfakes, but these artifacts become less apparent in high-visual-quality deepfakes generated by advanced models. With advancements in deep generative models, producing high-visual-quality deepfakes has become a strategy to evade detection. To address this, we propose${\sf ADA-FInfer}$, an adaptive frame selection and interpretable face representation inference method for detecting high-visual-quality deepfakes.${\sf ADA-FInfer}$adaptively selects frames by analyzing optical flow to reveal manipulations. We also introduce an adaptive attack method that manipulates specific frames, and our adaptive selection strategy shows resistance to such attacks.${\sf ADA-FInfer}$uses an encoder to learn face representations from source and target faces, applying a representation-prediction loss to maximize the distinction between real and fake videos. To provide further insights, we employ the joint entropy, mutual information, and conditional entropy analyses to explain the method's effectiveness. Extensive experiments and ablation studies demonstrate that${\sf ADA-FInfer}$achieves promising performance in detecting high-visual-quality deepfakes.

Existing image inpainting methods face limitations in detail restoration. Although transformer-based models have made certain progress recently, the lack of hierarchical feature interaction and insufficient consideration of the importance of features at different network levels lead to semantic ambiguity in image reconstruction. To enhance the visual quality and accuracy of image inpainting, we adopt a multi-level feature fusion approach and propose a novel, efficient hierarchical feature collaboration transformer (HFCT). Our approach comprises two modules: dual stream gated feature fusion (DSGF) and region-separated attention module (RSAM), effectively capturing features at different levels of the network and enhancing inter-level information exchange. The DSGF module uses soft gating to fuse primary and advanced features, strengthening the connection from local to global consistency and reducing artifacts. The RSAM module resolves attention isolation issues in feature fusion through region-separated attention, strengthening the understanding of feature relationships, capturing more image semantics, and improving restoration accuracy. Extensive experiments on the Paris StreetView, CelebA-HQ, and Places2 benchmark datasets demonstrate that our proposed method achieves superior image inpainting quality compared to several state-of-the-art inpainting algorithms.

As an important carrier of information transmission in online social networks (OSNs), the authenticity protection of images is of great significance. However, the abuse of image processing technology makes its security questionable. Meantime, lossy operations adopted by OSNs will change the forgery artifacts, which brings challenges to robust image forgery detection. To address this issue, considering the suppression of lossy noise caused by transmission, a novel multi-domain probability estimation network (PRest-Net) is proposed. Firstly, we design a multi-domain probability estimation method to capture the most differentiated regional information from the spatial, residual, and wavelet domains. Since the wavelet coefficient of semantic information is larger than that of lossy noise, and the edge texture can be highlighted in the residual image, the negative effect of lossy noise would be reduced and semantic forgery traces can be exposed more easily. We further design a forgery detector composed of low-level feature extraction, high-level feature extraction, and regional edge difference learning module, which can adaptively learn rich forgery clues. Extensive experimental results are provided to validate the superiority of PRest-Net compared with existing state-of-the-art detectors in the scenarios of detecting forged images transmitted over various OSNs.

Recently, there has been a growing demand for flow-based video frame interpolation methods, which introduce correlation volumes to supervise the correlation of bidirectional optical flows. However, they often overlook the symmetry of the bidirectional motion field by consuming substantial computational cost, which is reflected in the fact that these methods often require a long runtime. To address these issues, in this article, we propose a bidirectional 3D correlation volume which is suitable for video frame interpolation. By decomposing the 4D correlation volume into two 3D correlation volumes in the horizontal and vertical directions, we significantly enhance the model’s inference speed with a minor sacrifice compared to our baseline. Additionally, when handling 2K video frames, our method achieves several-fold improvement in inference speed compared to other methods which implied correlation volume. The code is available at https://github.com/famt0531 .

With the rise of the metaverse, the rapid advancement of Deepfakes technology has become closely intertwined. Within the metaverse, individuals exist in digital form and engage in interactions, transactions, and communications through virtual avatars. However, the development of Deepfakes technology has led to the proliferation of forged information disseminated under the guise of users’ virtual identities, posing significant security risks to the metaverse. Hence, there is an urgent need to research and develop more robust methods for detecting deep forgeries to address these challenges. This article explores deepfake video detection by leveraging the spatiotemporal inconsistencies generated by deepfake generation techniques, thereby proposing the interactive spatiotemporal inconsistency learning and interactive fusion (ST-ILIF) detection method, which consists of phase-aware and sequence streams. The spatial inconsistencies exhibited in frames of deepfake videos are primarily attributed to variations in the structural information contained within the phase component of the Fourier domain. To mitigate the issue of overfitting the content information, a phase-aware stream is introduced to learn the spatial inconsistencies from the phase-based reconstructed frames. Additionally, considering that deepfake videos are generated frame by frame and lack temporal consistency between frames, a sequence stream is proposed to extract temporal inconsistency features from the spatiotemporal difference information between consecutive frames. Finally, through feature interaction and fusion of the two streams, the representation ability of intermediate and classification features is further enhanced. The proposed method, which was evaluated on four mainstream datasets, outperformed most existing methods, and extensive experimental results demonstrated its effectiveness in identifying deepfake videos. Our source code is available at https://github.com/qff98/Deepfake-Video-Detection .

Fake images spread on online social networks can lead to public misconceptions, causing potential harm to politics, economics and social culture. Researching the authenticity of digital images is of great significance. Almost all online social networks perform multiple lossy operations on uploaded images, which will change the forgery traces and degrade the forgery detection performance of existing forensic methods. Therefore, forgery detection for images transmitted on online social networks has become a severe challenge. To solve this problem, we propose a multi-domain probability estimation network (PRest-Net). Specifically, considering that the wavelet coefficient of useful information is larger than that of lossy noise, and the image residual processing can highlight the edge texture, we design a multi-domain probability estimation method based on feature coupling and selective search to mine forgery traces in the spatial, residual, and wavelet domains. As a result, the lossy noise introduced by transmission on online social networks can be suppressed and the most differentiated regional information can be captured. Furthermore, a forgery detector based on low-level and high-level feature learning is proposed to adaptively capture rich operation clues and local edge differences between tampered and real areas, achieving fine-grained tampered region prediction. Extensive experiments show that the proposed PRest-Net outperforms existing state-of-the-art forgery detectors and is robust to online social network transmission.

The proliferation and application of the Internet of Things (IoT) have significantly increased the production and processing of large volumes of images. Due to privacy concerns, such data are often encrypted before being outsourced to third parties for subsequent retrieval and processing. Existing encrypted image retrieval schemes primarily aim to improve search accuracy but are limited in terms of retrieval efficiency and storage expenses, rendering them challenging to be implemented on IoT devices. To address these issues, we propose a segmented hash-based privacy-preserving image retrieval (SHPIR) scheme that enables accurate and efficient retrieval of encrypted images in IoT networks. First, we propose a feature extraction model based on the convolutional neural network (CNN) to generate low-dimensional segmented hash codes that represent both image categories and features accurately. Next, we design the cryptographic hierarchical index structure based on the segmented hash codes to improve retrieval efficiency and accuracy. We also design a secure Hamming distance computation algorithm and employ the learning with errors (LWEs)-based secure k-nearest neighbor (kNN) algorithm to preserve the privacy of feature vectors and file-access patterns. Finally, we provide a security analysis verifying that the SHPIR scheme can protect image privacy as well as indexing and query privacy. Additionally, we constructed a real IoT environment to test the practical effectiveness and applicability of our scheme. Experimental evaluation indicates that our proposed scheme outperforms existing state-of-the-art schemes in retrieval accuracy, search efficiency, and storage costs, making it more suitable for real-world IoT applications.

Generative text steganography has received considerable attention in the covert communication community for the benefit of sending secret messages without the need to modify carriers. Existing methods typically choose the next word when generating a stego-text based on conditional probability encoding of candidates, which may lead to generating inadequate words for the underlying secret message. How to generate a semantically controllable stego-text with a high capacity on secure embedding of a secret message is a main challenge. We address this challenge by proposing a new paradigm to generative text steganography that takes advantage of certain social media through apparently normal behaviors from the sender. In particular, we make use of the live commenting feature provided by public video sharing platforms (PVSPs), which allow viewers to make comments on video scenes that will fly on screens when the scenes are shown. We show that this feature can be used to construct a generative steganographic system. The sender generates at random a number of distracting words and a certain invertible matrix called W-dmatrix based on the total number of message words and distracting words. The sender then transforms a sequence of indexes of these words to a sequence, selects one or more videos with a sufficiently large number of total frames, and generates a comment on each frame in the sequence. The receiver extracts commented frame indexes, uses the shared stego-key to generate the same W-dmatrix as the sender, and obtains the secret message using the inverse of the matrix. The stego-key consists of a vocabulary generator and a W-dmatrix generator (WMG) based on pseudorandomly generated numbers. To generate comments on frames that conform to comments made by viewers, we devise a neural ResNet-LSTM model to generate a comment for an input image based on its content. Theoretical analysis shows that commented video frames (CVF) is covert, secure, efficient, and feasible to conceal any message of arbitrary length. We implement CVF and present evaluation results from multiple aspects that our work outperforms the existing stego-methods.

With the rapid growth of Internet technology, security concerns have risen, particularly with the prevalence of Deepfakes, a popular visual forgery technique. Therefore, there is necessary to research more powerful methods to detect Deepfakes. However, many Convolutional Neural Networks-based detection methods struggle with cross-database performance, often overfitting to specific color textures. We observe that image noises can weaken the influence of color textures and expose the forgery traces in the noise domain. This is because tampering techniques, when altering face images, disrupt the consistency of feature distribution in the noise space. And the forgery traces in the noise space are complementary to the tampering artifacts present in the image space information. Therefore, we propose a novel face forgery detection network that combines spatial domain and noise domain. Our Dual Feature Fusion Module and Local Enhancement Attention Module contribute to more comprehensive feature representations, enhancing our method’s discriminative ability. Experimental results demonstrate superior performance compared to existing methods on mainstream datasets. https://github.com/jhchen1998/DeepfakeDetection.

Malicious applications of deep face swapping technology pose security threats such as misinformation dissemination and identity fraud. Some research propose the utilization of robust watermarking methods to track the copyright of facial images, facilitating post-forgery identity attribution. However, these methods cannot fundamentally prevent or eliminate the adverse impacts of face swapping. To address this issue, we present Dual Defense, an innovative framework based on robust adversarial watermarking. It simultaneously tracks image copyrights and disrupts the face swapping model by one-time embedding the robust adversarial watermark. Specifically, we propose an Original-domain Feature Emulation Attack (OFEA) method, which makes the traceable watermark adversarial through specially designed original domain adversarial loss. Additionally, we conduct a wavelet domain image structural information compensation loss, combined with a channel attention mechanism, to jointly balance watermark invisibility, adversariality, and traceability. Furthermore, we design a more comprehensive and rational evaluation method to thoroughly assess the effectiveness of adversarial attacks against face swapping models. Extensive experiments demonstrate that Dual Defense exhibits exceptional cross-task generality and dataset generalization. It maintains impressive adversariality and traceability in both original and robust settings, surpassing current forgery defense methods that possess only one of these capabilities.

Full-reference image dehazing quality assessment (FR-IDQA) evaluates the visual quality of a dehazed image by measuring its differences with a clear reference. The existing FR-IDQA methods are not convincing due to the lack of well-aligned datasets of hazy and clear image pairs and the limited hand-crafted features make it difficult to simulate the complicated perception by the human visual system (HVS). In this work, we build a real-world image dataset, namely RW-Haze, which comprises natural hazy images and their well-aligned clear references. Each clear image is paired with several hazy images with diverse haze levels from slight to heavy. Meanwhile, the existing FR-IDQA works evaluate the dehazed image quality in a global manner, without considering local haze distributions in the original hazy image. Actually, the perceived haze in a natural hazy image is not uniformly distributed, and the haze density varies with scene depth. Based on this priori observation, we design a haze density-aware convolutional neural network (CNN), namely DehIQA, for FR-IDQA. It adopts transfer learning to alleviate the issue of lacking sufficient labeled data. Specifically, we divide image dehazing assessment into two tasks. The source task is to classify unpaired clear and hazy images, which enforces the deep network to learn haze-related features. The target task is image quality assessment, which is achieved by transferring the trained model for the source task to the target task. Considering the fact that the perceived distortion in a dehazed image is also not uniform, we present a haze density-aware mechanism into DehIQA, which assigns different weights for different local regions in a dehazed image in terms of the dark channel of the original hazy image. Extensive experimental results show that DehIQA outperforms the state-of-the-art (SOTA) works on the benchmark dataset and achieves better consistency with human perceptions.

With the prevalence of electronic devices in our daily lives, content leakages frequently occur, and to enable leakage tracing, screen-shooting resistant watermarking has attracted tremendous attention. However, current studies often overlook a thoughtful investigation of the cross-media screen-camera process and fail to consider the effect of grayscale deviation on the screen. In this paper, we proposescreen-shootingdistortionsimulation ($\bf {SSDS}$), which involves a grayscale deviation function for constructing a more practical noise layer. We divide SSDS into screen displaying and camera shooting. For screen displaying, different viewing angles result in grayscale deviation with distinct intensities, and we simulate the distortions by modeling the relative position of the viewing point and the screen plane. For camera shooting, a series of distortion functions are used to approximate the perturbations in the camera pipeline, including defocus blur, noise and JPEG compression. Furthermore, the gradient-guided encoder is designed to conduct the embedding in the texture region using a modification cost map. Experimental results show that our proposed watermarking framework outperforms the state-of-the-art methods in terms of robustness and visual quality.

Recent years have witnessed the booming growth of 5G and 6G technology, which has brought unprecedented massive data transmission, causing severe privacy issues. However, traditional information encryption and multimedia covert communication fail to protect the identities of communication parties and the originality of messages. The emergence of blockchain provides a promising solution to solve these problems. Its anonymity manages to hide the identities of communication parties, and immutability ensures the message is undestroyable. However, the existing blockchain-based covert communication schemes suffer the issues of low embedding capacity and high time cost. In this paper, an amount-based covert communication scheme over the blockchain is proposed, in which a unique coding method is devised for hiding messages into transaction amounts to improve the embedding capacity. Compared with existing address-based methods, the proposed scheme can apply any address and reduce the time of obtaining special addresses. Besides, we innovate the way to prove the concealment by calculating the relative entropy of the transaction amount between Bitcoin and the proposed scheme. The security of our method is demonstrated by comparing the probability of attackers acquiring secret messages under different adversary capabilities. The experimental results verify that the proposed approach outperforms the existing schemes regarding embedding capacity, time costs, number of transactions, concealment, and security.

As an important part of the text-to-speech (TTS) system, vocoders convert acoustic features into speech waveforms. The difference in vocoders is key to producing different types of forged speech in the TTS system. With the rapid development of general adversarial networks (GANs), an increasing number of GAN vocoders have been proposed. Detectors often encounter vocoders of unknown types, which leads to a decline in the generalization performance of models. However, existing studies lack research on detection generalization based on GAN vocoders. To solve this problem, this study proposes vocoder detection of spoofed speech based on GAN fingerprints and domain generalization. The framework can widen the distance between real speech and forged speech in feature space, improving the detection model’s performance. Specifically, we utilize a fingerprint extractor based on an autoencoder to extract GAN fingerprints from vocoders. We then weight them to the forged speech for subsequent classification to learn the forged speech features with high differentiation. Subsequently, domain generalization is used to further improve the generalization ability of the model for unseen forgery types. We achieve domain generalization using domain-adversarial learning and asymmetric triplet loss to learn a better generalized feature space in which real speech is compact and forged speech synthesized by different vocoders is dispersed. Finally, to optimize the training process, curriculum learning is used to dynamically adjust the contributions of the samples with different difficulties in the training process. Experimental results show that the proposed method achieves the most advanced detection results among four GAN vocoders. The code is available at https://github.com/multimedia-infomation-security/GAN-Vocoder-detection .

Malicious Deepfakes have led to a sharp conflict over distinguishing between genuine and forged faces. Although many countermeasures have been developed to detect Deepfakes ex-post, undoubtedly, passive forensics has not considered any preventive measures for the pristine face before foreseeable manipulations. To complete this forensics ecosystem, we thus put forward the proactive solution dubbed SepMark, which provides a unified framework for source tracing and Deepfake detection. SepMark originates from encoder-decoder-based deep watermarking but with two separable decoders. For the first time the deep separable watermarking, SepMark brings a new paradigm to the established study of deep watermarking, where a single encoder embeds one watermark elegantly, while two decoders can extract the watermark separately at different levels of robustness. The robust decoder termed Tracer that resists various distortions may have an overly high level of robustness, allowing the watermark to survive both before and after Deepfake. The semi-robust one termed Detector is selectively sensitive to malicious distortions, making the watermark disappear after Deepfake. Only SepMark comprising of Tracer and Detector can reliably trace the trusted source of the marked face and detect whether it has been altered since being marked; neither of the two alone can achieve this. Extensive experiments demonstrate the effectiveness of the proposed SepMark on typical Deepfakes, including face swapping, expression reenactment, and attribute editing. Code will be available at https://github.com/sh1newu/SepMark.

Unlike hiding bit-level messages, hiding image-level messages is more challenging, which requires large capacity, high imperceptibility, and high security. Although recent advances in hiding image-level messages have been remarkable, existing schemes are limited to lossless spatial images as covers and cannot be directly applied to JPEG images, the ubiquitous lossy format images in daily life. The difficulties of migration are caused by the lack of targeted design and the loss of details due to lossy decompression and compression. Considering that taking DCT densely on 8X8 image patches is the core of the JPEG compression standard, we design a novel model called EFDR, which can comprehensively Exploit Fine-grained D CT Representations and embed the secret image into quantized DCT coefficients to avoid the lossy process. Specifically, we transform the JPEG cover image and hidden secret image into fine-grained DCT representations that compact the frequency and are associated with the inter-block and intra-block correlations. Subsequently, the fine-grained DCT representations are further enhanced by a sub-band features enhancement module. Afterward, a transformer-based invertibility module is designed to fuse enhanced sub-band features. Such a design enables a fine-grained self-attention on each sub-band and captures long-range dependencies while maintaining excellent reversibility for hiding and recovery. To our best knowledge, this is the first attempt to embed a color image of equal size in a color JPEG image. Extensive experiments demonstrate the effectiveness of our EFDR with superior performance. Code will be available at https://github.com/yangjunxue/EFDR.

The recent success of deep image watermarking has demonstrated the potential of deep learning for watermarking, which has drawn increasing attention to deep video watermarking with the objective to improve its robustness and perceptual quality. Compared to images, video watermarking is much more challenging due to the rich structures of video data and the diversity of attacks in video transmission pipeline. The existing deep video watermarking schemes are far from satisfactory in dealing with temporal attacks, e.g., frame averaging, frame dropping and transcoding. To this end, a novel deep framework for Robustness Enhanced Video watermarking (REVMark) is proposed in this paper, aiming at improving the overall robustness, especially in dealing with H.264/AVC compression, while maintaining good visual quality. REVMark has an encoder/decoder structure with a pre-processing block (TAsBlock) to effectively extract the temporal-associated features on aligned frames. To ensure the end-to-end robust training, a distortion layer is integrated into the REVMark to resemble various attacks in real-world scenarios, among which, a new differentiable simulator of video compression, namely DiffH264, is developed to approximately simulate the process of H.264/AVC compression. In addition, the mask loss is incorporated to guide the encoder to embed the watermark in the human-imperceptible regions, thus improving the perceptual quality of the watermarked video. Experimental results demonstrate that the proposed scheme can outperform other SOTA methods while achieving 10X faster inference.

Image forgery detection has aroused widespread research interest in both academia and industry because of its potential security threats. Existing forgery detection methods achieve excellent tampered regions localization performance when forged images have not undergone post-processing, which can be detected by observing changes in the statistical features of images. However, forged images may be carefully post-processed to conceal forgery boundaries in a particular scenario. It becomes tough challenging to these methods. In this paper, we perform an analogous analysis between image forgery detection and blind signal separation, and formulate the post-processed image forgery detection problem into a signal noise separation problem. We also propose a signal noise separation-based (SNIS) network to solve the problem of detecting post-processed image forgery. Specifically, we first adopt the signal noise separation module to separate tampered region from the complex background region with post-processing noise, which weakens or even eliminates the negative impact of post-processing on forgery detection. Then, the multi-scale feature learning module uses a parallel atrous convolution architecture to learn high-level global features from multiple perspectives. Besides, a feature fusion module is utilized to enhance the discriminability of tampered regions and real regions by strengthening the boundary information. Finally, the prediction module is designed to predict the tampered region and classify the type of tampering operation. Extensive experiments show that the proposed SNIS is not only effective for forgery detection on forged images without post-processing, but also promising in robustness against multiple post-processing attacks. Furthermore, SNIS is robust in detecting forged images from unknown sources.

As a face manipulation technique, the misuse of Deepfakes poses potential threats to the state, society, and individuals. Several countermeasures have been proposed to reduce the negative effects produced by Deepfakes. Current detection methods achieve satisfactory performance in dealing with uncompressed videos. However, videos are generally compressed when spread over social networks because of limited bandwidth and storage space, which generates compression artifacts and the detection performance inevitably decreases. Hence, how to effectively identify compressed Deepfake videos over social networks becomes a significant problem in video forensics. In this paper, we propose a facial-muscle-motions-based (FAMM) framework to solve the problem of compressed Deepfake video detection. Specifically, we first locate faces from consecutive frames and extract landmarks from the face images. Then, continuous facial landmarks are utilized to construct facial muscle motion features by modeling the five sensory and face regions. Finally, we fuse the diverse forensic knowledge using Dempster-Shafer theory and provide the final detection results. Furthermore, we demonstrate the effectiveness of FAMM through analyzing mutual information, compression procedure, and facial landmarks for compressed Deepfake videos. Theoretical analyses illustrate that compression does not affect facial muscle motion feature construction and the differences in designed features exist between the real and Deepfake videos. Extensive experimental results conclude that the proposed method outperforms the state-of-the-art methods in detecting compressed Deepfake videos. More importantly, FAMM achieves comparable detection performance on compressed videos that are over real-world social networks.

In Intelligent Transportation Systems, images are the main data sources to be analyzed for providing intelligent and precision cognitive services. Therefore, how to protect the privacy of sensitive images in the process of information transmission has become an important research issue, especially in future no non-private data era. In this article, we design the Rearrangement-Arnold Cat Map (R-ACM) to disturb the relationship between adjacent pixels and further propose an efficient Bit-level Image Encryption Algorithm ($\text{B}{l}$-IEA) based on R-ACM. Experiments show that the correlation coefficients of two adjacent pixels are 0.0022 in the horizontal direction, -0.0105 in the vertical direction, and -0.0035 in the diagonal direction respectively, which are obviously weaker than that of the original image with high correlations of adjacent pixels. What’s more, the NPCR is 0.996120172, and the UACI is 0.334613406, which indicate that$\text{B}{l}$-IEA has stronger ability to resist different attacks compared with other solutions. Especially, the lower time complexity and only one round permutation make it particularly suitable to be used in the time-limited intelligent transportation field.

Weakly-supervised temporal action localization (WTAL) is a challenging task in understanding untrimmed videos, in which no frame-wise annotation is provided during training, only the video-level category label is available. Current methods mainly adopt temporal attention branches to conduct foreground-background separation with RGB and optical flow features simply concatenated, regardless of the discriminative spacial features and the complementarity between different modalities. In this work, we propose a Multi-Dimensional Attention (MDA) method to explore attention mechanism across three dimensions in weakly supervised action localization,i.e., 1) temporal attention that focuses on segments containing action instances, 2) channel attention that discovers the most relevant cues for action description, and 3) modal attention that fuses RGB and flow information adaptively based on feature magnitudes during background modeling. In addition, we introduce a similarity constraint loss to refine the action segment representation in feature space, which helps the network to detect less discriminative frames of an action to capture the full action boundaries. The proposed MDA with similarity constraints can be easily applied to existing action detection frameworks with few parameters. Extensive experiments on THUMOS’14 and ActivityNet v1.2 datasets show that the proposed method outperforms the current state-of-the-art WTAL approaches, and achieves comparable results with some advanced fully-supervised methods.

The aim of operation chain detection for a given manipulated image is to reveal the operations involved and the order in which they were applied, which is significant for image processing and multimedia forensics. Currently,allexisting approaches simply treat image operation chain detection as a classification problem and consider only chains of at most two operations. Considering the complex interplay between operations and the exponentially increasing solution space, detecting longer operation chains is extremely challenging. To address this issue, in this work, we devise a new methodology for image operation chain detection. Different from existing approaches based on classification modeling, we strategically conduct operation chain detection within a machine translation framework. Specifically, the chain in our work is modeled as a sentence in a target language, with each possible operation represented by a word in that language. When executing chain detection, we propose first transforming the input image into a sentence in a latent source language from the learned deep features. Then, we propose translating the latent language into the target language within a machine translation framework and finally decoding all operations, arranged in order. Besides, a chain inversion strategy and a bi-directional modeling mechanism are developed to improve the detection performance. We further design a weighted cross-entropy loss to alleviate the problems presented by imbalance among chain lengths and chain categories. Our method can detect operation chains containing up to seven operations and obtains very promising results in various scenarios for the detection of both short and long chains.

Deepfake has ignited hot research interests in both academia and industry due to its potential security threats. Many countermeasures have been proposed to mitigate such risks. Current Deepfake detection methods achieve superior performances in dealing with low-visual-quality Deepfake media which can be distinguished by the obvious visual artifacts. However, with the development of deep generative models, the realism of Deepfake media has been significantly improved and becomes tough challenging to current detection models. In this paper, we propose a frame inference-based detection framework (FInfer) to solve the problem of high-visual-quality Deepfake detection. Specifically, we first learn the referenced representations of the current and future frames’ faces. Then, the current frames’ facial representations are utilized to predict the future frames’ facial representations by using an autoregressive model. Finally, a representation-prediction loss is devised to maximize the discriminability of real videos and fake videos. We demonstrate the effectiveness of our FInfer framework through information theory analyses. The entropy and mutual information analyses indicate the correlation between the predicted representations and referenced representations in real videos is higher than that of high-visual-quality Deepfake videos. Extensive experiments demonstrate the performance of our method is promising in terms of in-dataset detection performance, detection efficiency, and cross-dataset detection performance in high-visual-quality Deepfake videos.

To assess the authenticity and integrity of digital image history, forensic analysts not only need to identify the types of tampering operations that multimedia has undergone but also distinguish their topological combination order. However, due to the interaction among tampering operations, a forensic method proposed for the operation identification yields unacceptably low performance when the content of a given image was forged by operations in a multi-operator chain. In this paper, we propose an efficient detection framework for determining the type and order of the tampering operations. Specifically, we first divide the operation detection problem into two aspects: operation type identification and their combination order detection. Then, by exploring the feature coupling relationships among different features with Pearson's correlation coefficient, optimal coupled features are designed for tampering operations detection. Finally, the detection of multiple operations applied to the previously JPEG-compression images is examined, and the effectiveness of our proposed framework has been demonstrated by simulation results.

The development of technologies that can generate Deepfake videos is expanding rapidly. These videos are easily synthesized without leaving obvious traces of manipulation. Though forensically detection in high-definition video datasets has achieved remarkable results, the forensics of compressed videos is worth further exploring. In fact, compressed videos are common in social networks, such as videos from Instagram, Wechat, and Tiktok. Therefore, how to identify compressed Deepfake videos becomes a fundamental issue. In this paper, we propose a two-stream method by analyzing the frame-level and temporality-level of compressed Deepfake videos. Since the video compression brings lots of redundant information to frames, the proposed frame-level stream gradually prunes the network to prevent the model from fitting the compression noise. Aiming at the problem that the temporal consistency in Deepfake videos might be ignored, we apply a temporality-level stream to extract temporal correlation features. When combined with scores from the two streams, our proposed method performs better than the state-of-the-art methods in compressed Deepfake videos detection.

With the coming era of cloud technology, cloud storage is an emerging technology to store massive digital images, which provides steganography a new fashion to embed secret information into massive images. Specifically, a resourceful steganographer could embed a set of secret information into multiple images adaptively, and share these images in cloud storage with the receiver, instead of traditional single image steganography. Nevertheless, it is still an open issue how to allocate embedding payload among a sequence of images for security performance enhancement. This paper formulates adaptive payload distribution in multiple images steganography based on image texture features and provides the theoretical security analysis from the steganalyst's point of view. Two payload distribution strategies based on image texture complexity and distortion distribution are designed and discussed respectively. The proposed strategies can be employed together with these state-of-the-art single image steganographic algorithms. The comparisons of the security performance against the modern universal pooled steganalysis are given. Furthermore, this paper compares the per image detectability of these multiple images steganographic schemes against the modern single image steganalyzer. Extensive experimental results show that the proposed payload distribution strategies could obtain better security performance.

Recent studies have shown that physiological signals such as heart beat and breathing can be remotely captured from human faces using a regular color camera under ambient light. This technology, referred to as remote photoplethysmography (rPPG), can be used to collect the physiological status of users who are in front of a camera, which may raise privacy concerns. To avoid the privacy abuse of the rPPG technology, this paper develops PulseEdit, a novel and efficient algorithm that can edit the physiological signals in facial videos without affecting visual appearance and thus protect the user’s physiological signal from disclosure. PulseEdit can either remove the trace of the physiological signal in a video or transform the video to contain a target physiological signal chosen by a user. Experimental results show that PulseEdit can effectively edit physiological signals in facial videos and prevent heart rate measurement based on rPPG. It is possible to utilize PulseEdit in adversarial scenarios against rPPG-based visual security algorithms. We present analyses on the performance of PulseEdit against rPPG-based liveness detection and rPPG-based deepfake detection, and demonstrate its ability to circumvent these visual security algorithms and its important role in supporting the design of attack-resilient systems.

Recently, many forensic techniques have been developed to detect the use of a certain processing operation. When utilizing several manipulations to alter an image, artifacts left by manipulations that have been applied later can potentially disguise traces left by manipulations that were applied earlier. Therefore, the detection of manipulations become difficult. In this paper, we focus on identifying the manipulations in an image operation chain composed of multiple manipulations in a certain order. To address this issue, we analyze the relationship between manipulations identification and blind signal separation. Then, we propose a features decoupling method based on blind signal separation, which decouples the coupled features due to the superimposed processing artifacts and exploits the decoupled features to identify multiple operations. The experiments carried out on two image operation chains confirm the effectiveness of the proposed method.

Forgery of paper invoices and certificates negatively causes huge economic loss every year. In the molding process of paper, plant fibres inside will distribute randomly and form the unique microscopic surface. Accordingly, forensic researchers have presented many techniques to build its feature descriptor for paper anti-counterfeiting. Among those methods, normal vector field and reconstructed surface based on multi-view photos captured by mobile camera have indicated incredibly effectiveness and practicality. However, the inefficiency of existing models using mobile camera is the neck in real time detection scenarios. In this paper, we focus on the efficient microstructure orientation estimation of paper surface for authentication. To address this problem, we investigate the reflection characteristics of rough surface and quantify the light attenuation accurately. Then, we model the entire propagation path of light emitted by flash, and exploit single photo captured by mobile camera to reconstruct microscopic inclination based on beam energy loss. Reduction of required pictures significantly enhances forensics efficiency. Through comparison experiments, we demonstrate that our proposed method can be used for paper discrimination effectively with speedy feature extraction.

Currently, many effective techniques have been proposed to estimate the parameters of tampering operations. Most of them consider the situation that an image is tampered by only one operation. However, multiple manipulation operations are always used to tamper an image in our daily life. Moreover, since the tampering traces of previous operations may be weakened or eliminated by later ones, the detecting accuracy of methods used for detecting single operations would be reduced. In this paper, we propose a new method to estimate the parameters of operations in different manipulation chains. Especially, we first investigate the correlation of operations and divide the degree of interactions between them into uncoupled and coupled. Furthermore, resizing and median filtering are adopted to reveal the assessment framework. Meanwhile, we propose well-directed features including energy density of difference-image to estimate operation parameters. The experiment proves the effectiveness of our method.

In traditional steganographic schemes, RGB three channels payloads are assigned equally in a true color image. In fact, the security of color image steganography relates not only to data-embedding algorithms but also to different payload partition. How to exploit inter-channel correlations to allocate payload for performance enhancement is still an open issue in color image steganography. In this paper, a novel channel-dependent payload partition strategy based on amplifying channel modification probabilities is proposed, so as to adaptively assign the embedding capacity among RGB channels. The modification probabilities of three corresponding pixels in RGB channels are simultaneously increased, and thus the embedding impacts could be clustered, in order to improve the empirical steganographic security against the channel co-occurrences detection. The experimental results show that the new color image steganographic schemes, incorporated with the proposed strategy, can effectively make the embedding changes concentrated mainly in textured regions, and achieve better performance on resisting the modern color image steganalysis.

With the coming of the era of big data, digital images are growing explosively on the Internet. Traditional steganography which embeds payload into one cover image is facing new challenges. In fact, a steganographer could utilize multiple images to embed payload simultaneously. How to fully exploit image features to allocate payload for performance enhancement is still an open issue. In this paper, we first establish a framework to formulate data embedding in multiple images. Two new embedding strategies based on image texture complexity and distortion distribution are designed for payload distribution, which can be implemented together with these state-of-art single image steganographic algorithms. Experimental results demonstrate that two proposed embedding strategies in multiple images steganography could obtain better statistical undetectability.

With the development of cloud computing, the advantages of low cost and high computation ability meet the demands of complicated computation of multimedia processing. Outsourcing computation of cloud could enable users with limited computing resources to store and process distributed multimedia application data without installing multimedia application software in local computer terminals, but the main problem is how to protect the security of user data in untrusted public cloud services. In recent years, the privacy-preserving outsourcing computation is one of the most common methods to solve the security problems of cloud computing. However, the existing computation cannot meet the needs for the large number of nodes and the dynamic topologies. In this paper, we introduce a novel privacy-preserving outsourcing computation method which combines GM homomorphic encryption scheme and Bloom filter together to solve this problem and propose a new privacy-preserving outsourcing set intersection computation protocol. Results show that the new protocol resolves the privacy-preserving outsourcing set intersection computation problem without increasing the complexity and the false positive probability. Besides, the number of participants, the size of input secret sets, and the online time of participants are not limited.

Abstract Some steganography methods for gray‐scale image can be extended to true RGB color image by treating each of its three color channels as a gray‐scale image. In modern popular steganography, most embedding changes are highly concentrated on those complex textural regions with smaller embedding distortions. However, the existing steganalysis methods for color images directly extract steganalytic features from the whole image. In this paper, we propose a content‐adaptive steganalysis strategy for color images. The new strategy aims to extract spatial rich model features from each color channel and just extract color rich model features from the pixels that may have been modified. In order to locate those suspected pixels, we first calculate the embedding costs of each channel, and then a subset of pixels with smaller embedding costs is selected. Experimental results show that the proposed strategy performs better than the state‐of‐the‐art color image steganalysis method. Copyright © 2017 John Wiley & Sons, Ltd.

To improve the stego-image quality and provide a larger embedding capacity, a novel steganographic method based on four-pixel differencing and modulus function is presented. We process a block of four neighboring pixels, and form three two-pixel groups to record the information of secret data. In each group, the difference value between two pixels is exploited to estimate how many secret bits will be embedded. Two pixels will be adjusted so that the sum of the remainders of them is equal to secret data. In order to extract the secret data exactly, four pixel values in a block are adjusted synchronously by using modulus function. An optimization problem is formulated to minimize the embedding distortion. A theoretical proof is given to ensure the solvability of the problem. The experimental results show the proposed method not only has a larger embedding capacity but also provides better stego-image quality.