The proliferation of AI-generated imagery poses escalating challenges for multimedia forensics, yet many existing detectors depend on assumptions about the internals of specific generative models, limiting their cross-model applicability. We introduce a self-supervised approach for detecting AI-generated images that leverages camera metadata-specifically exchangeable image file format (EXIF) tags-to learn features intrinsic to digital photography. Our pretext task trains a feature extractor solely on camera-captured photographs by classifying categorical EXIF tags (e.g., camera model and scene type) and pairwise-ranking ordinal and continuous EXIF tags (e.g., focal length and aperture value). Using these EXIF-induced features, we first perform one-class detection by modeling the distribution of photographic images with a Gaussian mixture model and flagging low-likelihood samples as AI-generated. We then extend to binary detection that treats the learned extractor as a strong regularizer for a classifier of the same architecture, operating on high-frequency residuals from spatially scrambled patches. Extensive experiments across various generative models demonstrate that our EXIF-induced detectors substantially advance the state of the art, delivering strong generalization to in-the-wild samples and robustness to common benign image perturbations.

Deep functionality hiding is an emerging technique that embeds confidential or sensitive functions within seemingly benign deep learning models (DLMs), which perform ordinary machine learning tasks. This enables such models to execute covert tasks while remaining undetected. Despite the rapid progress in deep functionality hiding, countermeasures remain unexplored. In this paper, we propose Distribution Offset Analysis (DOA), a novel method for detecting hidden functionalities in DLMs. Our key insight is that the weight distribution of a benign DLM typically follows a Gaussian distribution, whereas a container DLM with hidden functionalities exhibits notable statistical deviations from this Gaussian pattern. In our methodology, we first compute the distributional distance (i.e.,offsets) between the model's weights and an ideal Gaussian distribution. We then fuse these offsets with weight features into a unified representation, which is subsequently used to train a meta-classifier for hidden functionality detection. Through extensive experiments, we demonstrate the effectiveness of the proposed DOA method, which achieves an average detection rate of over 87% against existing state-of-the-art deep functionality hiding techniques.

Steganography aims to hide secret data into a cover media by subtle perturbation. Recently, deep neural network (DNN) based steganography has attracted a lot of research interests. Most of the existing DNN-based steganographic schemes design the secret encoder and decoder in an asymmetrical manner, the performance of which is heavily dependent on the content of cover media. In addition, they are weak in preventing unauthorized data extraction from the stego-media (i.e., the media with hidden data). To address these two issues, we propose a novel DNN steganographic framework termed the Cover Independent DNN Steganography (CIDS). In our CIDS, we take advantage of the powerful generative models to obtain AI-generated cover media for both the sender and receiver according to a key. Then, we propose a pair of symmetrical secret encoder and decoder to conduct a bijective transformation between the secrets and perturbations. While the perturbation is combined with the cover media to produce the stego-media. Such a secret encoding/decoding strategy focuses only on the secrets and is independent of the cover media. We also propose to encrypt the perturbations to effectively prevent unauthorized data extraction. Comprehensive experiments demonstrate the advantage of our CIDS over the state-of-the-art approaches for image steganography.

Steganography can be mainly classified into two categories: modification-based and generation-based approaches. Modification-based steganography embeds secret messages by altering specific statistical properties of the cover image, offering relatively high capacity and robustness. However, such modifications also make the stego images more susceptible to be detected by steganalysis tools. In contrast, generation-based steganography, which directly generates them via generative models, provides higher security but suffers from limited hiding capacity and poor robustness. To address these limitations, we propose a unified deep network for image steganography (U-StegNet) that integrates the high capacity and robustness of modification-based steganography with the high security of generation-based steganography. Specifically, we design a dual stage hiding strategy. In the first stage, we utilize a generative model to produce an intermediate image that hides part of the secret message while preserving statistical properties consistent with natural images, thereby enhancing resistance to steganalysis. In the second stage, we design a U-Net-based encoder to hide the remaining secret message into the intermediate image, while simultaneously enhancing fine-grained visual details. Moreover, we introduce a message division module that adaptively allocates the message length for each stage based on the image content and capacity constraints, thereby improving both imperceptibility and robustness. Extensive experimental results demonstrate the effectiveness of our U-StegNet. It achieves a PSNR of 44.41 dB and an SSIM of 0.9968 on stego images, which demonstrates the high imperceptibility of our method. Furthermore, it maintains strong robustness across nine types of common attacks, with message extraction accuracy nearly above 98%.

Diffusion models have made tremendous progress in generating visually realistic images. However, these images are statistically different from the real images, which could be accurately classified by carefully designed detectors. To evade detection, researchers have proposed various adversarial example generation schemes for AI-generated images. Despite the progress, most of these schemes have the tendency to post-process the images and the distortion is inevitable. In this paper, we propose an Adversarial Diffusion Model (ADM), which is able to directly generate high quality and undetectable images from scratch on top of a pre-trained stable diffusion model. In the ADM, an adversarial denoising U-Net is proposed for searching an adversarial latent. This latent is helpful for generating a prompt consistent adversarial example which is able to deceive the detector. Then, we propose a latent compensation module to make the adversarial examples have a similar reconstruction error to that of the real images. We further propose an adversarial decoder to minimize the difference between the high-frequency components of the real and adversarial examples. Comprehensive experiments are carried out to demonstrate the advantages of our ADM in generating adversarial AI-generated images.

Digital watermarking has shown its effectiveness in protecting multimedia content. However, existing watermarking is predominantly tailored for specific media types, rendering them less effective for the protection of content displayed on computer screens, which is often multi-modal and dynamic. Visual Screen Content (VSC), is particularly susceptible to theft and leakage through screenshots, a vulnerability that current watermarking methods fail to adequately address. To address these challenges, we propose ScreenMark, a robust and practical watermarking method designed specifically for arbitrary VSC protection. ScreenMark utilizes a three-stage progressive watermarking framework. Initially, inspired by diffusion principles, we initialize the mutual transformation between regular watermark information and irregular watermark patterns. Subsequently, these patterns are integrated with screen content using a pre-multiplication alpha blending technique, supported by a pre-trained screen decoder for accurate watermark retrieval. The progressively complex distorter enhances the robustness of the watermark in real-world screenshot scenarios. Finally, the model undergoes fine-tuning guided by a joint-level distorter to ensure optimal performance. To validate the effectiveness of ScreenMark, we compiled a dataset comprising 100,000 screenshots from various devices and resolutions. Extensive experiments on different datasets confirm the superior robustness, imperceptibility, and practical applicability of the method.

Ceramic artworks with elegant patterns present enormous collectible value and profits. To claim the copyright, the builder usually pastes their conspicuous stamp on the bottom or side of the ceramic artworks, which inevitably affects the external image of the artwork. In addition, the stamp is weak in resisting forgery attacks due to its visible nature. To address the above issues, we propose in this paper a novel framework for embedding invisible watermarking into patterns of the ceramic artworks. In the framework, a template-based watermarking embedding scheme is designed to map the watermark to an invisible template, which is added to the ceramic pattern to create its watermarked version. A distortion layer is further proposed to model the distortion of ceramic patterns in the ceramic manufacturing process, where a color-halftoning and an adaptive brightness adjustment strategy are developed to counter the print and firing operations that introduce the most significant distortions. Finally, a deep decoder is learned to extract the watermarking from the distorted pattern. Various experiments have been conducted to demonstrate the advantage of our proposed method for protecting the copyright of the ceramic artworks, which provides reliable watermark extraction accuracy without the need for a conspicuous stamp.

Embedding links in brand logos is a promising technology, which allows consumers to access the online information of products by capturing physical logo images. Previous physical data hiding methods primarily embed data within cover media in a global manner, making them ineffective for processing brand logos in vector graphics format with a transparent background. To address this issue, we propose in this paper a novel physical deep hiding scheme for invisibly embedding links in printed trademarks. Specifically, the encoder embeds links only into the area of the brand logo under the constraints of a mask, which is generated from the transparency information of the logo image. A background variation distortion is introduced into the distortion layer that approximate practical logo print-camera environments, such that the decoder could be learnt to retrieve the link from the camera-captured logo with various backgrounds. A feature prompt subspace modulator is further proposed and employed in the encoder to enhance the invisibility of the encoded logo pattern and in the decoder to boost hyperlink extraction accuracy. Various experiments have been conducted to demonstrate the advantage of our proposed method for embedding links in printed brand logos, which provides reliable extraction accuracy under both simulated and real scenarios.

The proliferation of digital images on the Internet has provided unprecedented convenience, but also poses significant risks of malicious theft and misuse. Digital watermarking has long been researched as an effective tool for copyright protection. However, it often falls short when addressing partial image theft, a common yet little-researched issue in practical applications. Most existing schemes typically require the entire image as input to extract watermarks. However, in practice, malicious users often steal only a portion of the image to create new content. The stolen portion can have arbitrary shape or content, being fused with a new background and may have undergone geometric transformations, making it challenging for current methods to extract correctly. To address the issues above, we propose WOFA (Watermarking One for All), a robust watermarking scheme against partial image theft. First of all, we define the entire process of partial image theft and construct a dataset accordingly. To gain robustness against partial image theft, we then design a comprehensive distortion layer that incorporates the process of partial image theft and several common distortions in channel. For easier network convergence, we employ a multi-level network structure on the basis of the commonly used embedder-distortion layer-extractor architecture and adopt a progressive training strategy. Abundant experiments demonstrate that our superior performance in the scenario of partial image theft, offering a more reliable solution for protecting digital images against unauthorized use in practical use.

The prevalence of AI-generated images has evoked concerns regarding the potential misuse of image generation technologies. In response, numerous detection methods aim to identify AI-generated images by analyzing generative artifacts. Unfortunately, most detectors quickly become obsolete with the development of generative models. In this paper, we first design a low-level feature extractor that transforms spatial images into feature space, where different source images exhibit distinct distributions. The pretext task for the feature extractor is to distinguish between images that differ only at the pixel level. This image set comprises the original image as well as versions that have been subjected to varying levels of noise and subsequently denoised using a pre-trained diffusion model. We employ the diffusion model as a denoising tool rather than an image generation tool. Then, we frame the AI-generated image detection task as a one-class classification. We estimate the low-level intrinsic feature distribution of real photographic images and identify features that deviate from this distribution as indicators of AI-generated images. We evaluate our method against over 20 different generative models, including those in GenImage and DRCT-2M datasets. Extensive experiments demonstrate its effectiveness on AI-generated images produced not only by diffusion models but also by GANs, flow-based models, and their variants.

Audio watermarking has been widely applied in copyright protection and source tracing. However, due to the inherent characteristics of audio signals, watermark localization and resistance to desynchronization attacks remain significant challenges. In this paper, we propose a learning-based scheme named SyncGuard to address these challenges. Specifically, we design a frame-wise broadcast embedding strategy to embed the watermark in arbitrary-length audio, enhancing time-independence and eliminating the need for localization during watermark extraction. To further enhance robustness, we introduce a meticulously designed distortion layer. Additionally, we employ dilated residual blocks in conjunction with dilated gated blocks to effectively capture multi-resolution time-frequency features. Extensive experimental results show that SyncGuard efficiently handles variable-length audio segments, outperforms state-of-the-art methods in robustness against various attacks, and delivers superior auditory quality.

Digital steganography embeds secret messages into images via invisible modifications, posing challenges for steganalysis, which seeks to detect these alterations by analyzing subtle shifts in image distributions. Previous steganalysis efforts primarily focus on enhancing the steganographic signal while suppressing image semantic content, such as through high-pass filtering. However, these empirically designed methods often lack theoretical underpinnings, exhibiting reduced detection accuracy, particularly at low embedding capacities. To address these limitations, we propose an innovative steganalysis approach that transforms images into pure Gaussian noise representations, actively amplifying the subtle distribution shifts introduced by the steganographic processes in spatial images. This paper pioneers the application of diffusion models to magnify steganographic signals, proposing a new paradigm for further research. Specifically, we iteratively perform forward steps of the probability flow in diffusion models to diminish semantic information. By utilizing the natural spreading properties of the diffusion process, we have theoretically validated the efficacy of each forward step in amplifying differences in noise patterns between cover and stego samples. These magnified differences can be easily captured by a simple classifier—a two-layer MLP. Extensive experiments demonstrate the effectiveness of our method, highlighting detection accuracy gains of 10% to 20% under standard conditions and an average 6.7% increase at low embedding rates compared to existing schemes.

The exorbitant cost of training Large Language Models (LLMs) makes it essential to protect the models from illegal copying and unauthorized usage. Recent attempts at LLM protection utilize black-box watermarking schemes, which embed distinctive input-output mapping (i.e., trigger set) directly into the models. However, most of them construct trigger inputs by injecting abnormal characters into normal text, which can easily be filtered out by unauthorized users, leading to a failure in watermark verification. In this paper, we propose a novel filtering-resistant LLM watermarking scheme, which takes advantage of imperceptible text styles to trigger the watermark. To achieve this, we adopt a trigger generation network to transform normal text into stylized sentences, which are assigned a specific watermarking label to build the trigger set. We then fine-tune the LLMs on both the trigger sets and clean samples for watermark embedding and performance stabilization. To boost watermark accuracy, we further propose a feature separation loss term to distinguish between normal and trigger inputs. Experimental results indicate the effectiveness of our proposed scheme for resisting the filtering attack.

Pancreatic NEuroendocrine Tumors (pNETs) are very rare endocrine neoplasms that account for less than 5% of all pancreatic malignancies, with an incidence of only 1–1.5 cases per 100,000. Early detection of pNETs is critical for improving patient survival, but the rarity of pNETs makes segmenting them from CT a very challenging problem. So far, there has not been a dataset specifically for pNETs available to researchers. To address this issue, we propose a pNETs dataset, a well-annotated Contrast-Enhanced Computed Tomography (CECT) dataset focused exclusively on Pancreatic Neuroendocrine Tumors, containing data from 469 patients. This is the first dataset solely dedicated to pNETs, distinguishing it from previous collections. Additionally, we provide the baseline detection networks with a new slice-wise weight loss function designed for the UNet-based model, improving the overall pNET segmentation performance. We hope that our dataset can enhance the understanding and diagnosis of pNET Tumors within the medical community, facilitate the development of more accurate diagnostic tools, and ultimately improve patient outcomes and advance the field of oncology.

The exorbitant cost of training Neural Radiance Fields (NeRF) makes it essential to protect them from illegal copying and unauthorized usage. Recent attempts at NeRF protection utilize watermarking schemes, which subtly alter NeRFs such that their rendered images contain watermarks that can be extracted by a decoder. In this paper, we investigate the robustness of existing NeRF watermarking schemes against surrogate model attacks, which train a surrogate NeRF using the input-output pairs of a victim NeRF. By proposing a Feature-Modulation Surrogate Model Attack (FM-SMA), we successfully crack most of the existing NeRF watermarking schemes. As a remedy, we further propose Texture-Aware Neural Radiance Fields Watermarking (TA-NFW), which resists FM-SMA attacks by embedding watermarks within the textures of NeRF-rendered results. Various experiments have been conducted to demonstrate the vulnerability of existing NeRF watermarking methods and the robustness of TA-NFW against the proposed FM-SMA attack.

Text-to-image (T2I) diffusion models exhibit impressive generation capabilities in recently studies. However, they are vulnerable to backdoor attacks, where model outputs are manipulated by malicious triggers. In this paper, we propose a novel input-level defense method, called Fine-grained Prompt Screening (GrainPS). Our method is motivated by the phenomenon, i.e., Semantics Misalignment, where the backdoor trigger causes the inconsistency between the cross-attention projections of object words (the key words to determine the main content of the generated image) and their true semantics. In particular, we divide each prompt into pieces and conduct fine-grained analysis by examining the impact of the trigger on object words in the cross-attention layers rather than their global influence on the entire generated image. To assess the impact of each word on object words, we formulate "semantics alignment score'' as the metric with a carefully crafted detection strategy to identify the trigger. Therefore, our implementation can detect backdoor input prompts and localize of triggers simultaneously. Evaluations across four advanced backdoor attack scenarios demonstrate the effectiveness of our proposed defense method.

Bitwise Vision AutoRegressive (BVAR) Model, as a distinguished source of young blood, has been taking the lead in the track of text-to-image synthesis, which at the same time raises legal and ethnic concerns such as copyright and authenticity. However, existing methods mainly focus on watermarking within diffusion models, which rely on the distinctive attributes of diffusion steps and cannot be directly transferred to new circumstances. To this end, we propose Safe-BVAR, the first watermark framework to embed bit strings during image generation in BVAR. Our study discovers the local similarity of the inferenced latent feature and the element-wise robustness of image autoencoder. Therefore, combined with the residual-accumulative nature of BVAR, we propose a novel Late Stage Residual Implanter to embed watermark and extract the information based on Local Contextual Extractor. Furthermore, we propose a Distributed Rotational Arranger to enhance watermark against local distortions. Our method is training-free and plug-and-play. Meanwhile, it can be easily applied to flexible-sized images. We evaluate the robustness and invisibility of the watermark, showing that it can resist common image attacks and cast inappreciable influence on the image.

Online face recognition systems usually store face features in the server database for authentication, which are vulnerable to face reconstruction attacks. Various face privacy protection approaches have been proposed to address this issue, where transformation-based schemes are shown to be promising. However, the existing transformation-based schemes are all hand-crafted approaches which are difficult to balance the privacy protection and face recognition. In this paper, we propose to learn a set of discrepant convolutional neural networks (DCNNs) to protect the privacy of face features. We randomly split the original face features into different sub-features. Each of the DCNNs transforms an original sub-feature into a protected one. We adopt appropriate strategies to make the DCNNs as diverse as possible to improve the ability of our protected features to resist different face reconstruction attacks, where a face recognition loss and a privacy protection loss are designed for training. The former ensures that the protected feature can be matched directly using the existing face recognizers, while the latter incorporates a shadow face reconstruction model to interrupt the correlation between the protected features and the face images. Experimental results demonstrate the advantage of our method over existing schemes for face privacy protection. Our protected features can be accurately matched using existing face recognizers, which are capable of resisting both black-box and white-box face reconstruction attacks.

Fake document detection is an important area in image forensics. Most of the existing techniques focus on the detection of digitally forged documents. In this paper, we look into the forgery of generalized physical occlusion, which is a simple and effective strategy to generate fake document images. We propose an Adversary Decomposition Network (ADDNet) to effectively extract generalized physical occlusion features from various types of documents, where two adversarial classifiers are designed and trained for feature decomposition. On top of the ADDNet, we further propose a lightweight Document Adapter (DA) for flexible and scalable fake document detection, which works well when we encounter a new type of document with limited samples for fine-tuning. To facilitate the research, we newly construct a dataset for physical occlusion detection on different types of documents. Various experiments are carried out to demonstrate the advantage of our proposed scheme over the existing schemes for physical occlusion detection, especially when the document is unseen or has limited samples in training.

The widespread emergence of Deepfake content has made it increasingly important to distinguish real and fake faces. Although many methods focus on detecting Deepfake content, only a few address the protection of real faces against forgery. Therefore, this paper proposes a proactive Deepfake detection scheme named FALU, which combines the uniqueness of facial identity features with the robustness of average hashing. The method first divides the input image into facial and non-facial regions, extracts identity-related features from the facial region, encodes them using average hashing, and embeds the result as a watermark into the non-facial region. During detection, the watermark is extracted from the non-facial region and compared with a newly generated hash code from the facial region. High correlation indicates authenticity, while low correlation suggests Deepfake forgery. To facilitate efficient and reliable watermark embedding, FALU integrates the symmetric sampling structure of U-Net with Mamba-like linear attention mechanism, proposing a lightweight encoder network. This scheme ensures the persistent presence of secret information before and after manipulation, thereby enhancing face source detection and tampering identification. Experimental results demonstrate that the proposed scheme effectively counters traditional Deepfake techniques and shows significant potential for preserving personal privacy.

Graphics Interchange Format (GIF) encoding is the art of reproducing an image with limited colors. Existing GIF encoding schemes often introduce unpleasant visual artifacts such as banding artifact, dotted-pattern noise and color shift, especially when the palette size is small. To address the issues above, we propose VivID, a Visually Improved GIF Encoding Network Design, which is compatible with exiting GIF decoders. VivID consists of three modules and two of them provide the functionality within the GIF encoding pipeline. Firstly, in order to reduce the color shift introduced by color quantization, we design the multi-palette extractor to create a GIF image with minimal distortion by extracting a near-optimal palette. This module can significantly improve the image fidelity and gains adaptability to multiple palette sizes after only one-time training. Furthermore, to reduce banding artifact and the dotted-pattern noise caused by dithering process, we propose banding remover which can randomize quantization error to neighbourhood by utilizing a learnable dithering pattern. Moreover, to further eliminate the banding artifacts, we design the banding scorer module, which is a novel metric for evaluating banding artifact and it correlates well with subjective perception. We adopt it as a customized loss for training dithering module. Extensive experiments across various aspects demonstrate that VivID produces visually pleasing results even when the palette size is extremely small, outperforming both traditional and existing learning based GIF encoding methods.

The rapid growth of generative models has led to a new direction in steganography called generative steganography (GS). It allows message-to-image generation without the need for a carrier image. Recently, generative steganography methods have been proposed using generative adversarial networks (GANs) and Flow models. On the one hand, methods that use GANs to generate stego images struggle to fully recover the hidden message because the networks are not reversible. On the other hand, methods based on Flow encounter a problem where the images they create might not look real, mainly because the network has limitations in being reversible. Diffusion models fulfill network reversibility while generating high-quality images. However, the framework of existing diffusion models is reversible, but hidden message recovery is not perfectly reversible, resulting in the recovered message being similar but not exactly the same as the hidden one. Existing diffusion models are typically trained for one-directional image generation tasks, so they face some problems when dealing with bi-directional steganography tasks. If pre-trained diffusion models are directly used to generate stego images, exact secret data extraction through the diffusion process cannot be achieved. In this paper, we present an improved generative steganography based on the diffusion model (GSD), which conceals secret data in the frequency domain of random noise to enhance the security and accuracy of steganography, and re-trains the denoising diffusion implicit model (DDIM) for steganography, called the StegoDiffusion. During training StegoDiffusion, random noise is injected into the clean natural images and then trained through the forward diffusion process to obtain the re-trained StegoDiffusion. Our proposed GSD scheme achieves a 100% extraction accuracy for hidden secret data with a payload of 1 bit-per-pixel (bpp) in a single channel, and generates high-quality stego images in PNG format.

Generative steganography (GS) is a novel data-hiding technique that generates stego images directly from secret data without using cover images, which is different from traditional steganography. However, existing steganography methods have shortcomings in terms of hiding capacity, extraction accuracy, and diversity of stego images. To address these limitations, we propose a high-performance Conditional Flow-based Generative Steganography (CFGS). First, to achieve exact extraction of secret data in high-capacity scenarios, we hide secret data in the frequency domain to resist the impact of stego image distortion. In addition, to enhance the diversity of stego images, we introduce a novel conditional generative flow model (C-Flow) to generate stego images, which consists of two newly designed layers, the Conditional Attention-based Affine Coupling layer and the Conditional Invertible Norm layer. C-Flow can accurately guide the visual content of stego images through different conditions, enhancing the diversity of stego images. Our approach is the first GS method capable of conditional guidance of stego image visual content, and achieves extraction accuracy of hidden secret data equal to or close to 100% for payloads up to 1 bit-per-pixel (bpp). Extensive experiments demonstrate that our proposed approach outperforms state-of-the-art GS methods.

Radiometric calibration (RC) is a critical process in aerospace infrared remote sensing that establishes the relationship between the radiation energy of observed objects and the digital number (DN) output from sensors, which is fundamental for ensuring high-precision applications of infrared remote sensing data. At present, source-based RC (SBRC) is the predominant method, relying on a variety of radiometric sources (RSs) including in-orbit blackbodies, or natural targets such as lakes and oceans. This approach, while effective, imposes constraints on remote sensing systems such as space & weight allocation for RS and additional observation time for RC. Moreover, the reliance on physical calibration sources can introduce uncertainties due to factors such as imperfect emissivity of in-orbit blackbodies, lack of data consistency due to varied RS types, and variations in environmental conditions. In this article, we propose a novel RC method named artificial intelligence RC (AIRC), which directly generates RC coefficients for the in-orbit remote sensing satellites using the physical and environmental parameters of the sensor. We first theoretically prove that RC coefficients can be derived as functions of the sensor states. Next, we propose our neural networks for infrared RC (RCNN), to learn this relationship based on historical high-accuracy calibration data, enabling a shift from reference traceability (RT) to states traceability (ST). Then, to verify the feasibility of the proposed scheme, we train and test a multilayered perceptron (MLP) as a simple implementation of RCNN based on our long-term well-curated RC data from our FengYun-4A Advanced Geosynchronous Radiation Imager (FY-4A AGRI), and the experiments show that the proposed method achieves high-accuracy RC comparable with the official RC method applied on FY-4A AGRI that uses an in-orbit blackbody. Our study showcases how to conduct RC using the “reason (the states of sensor)–results (calibration coefficient)” logic, as supplement to the existing “result (observation to RS)–reason (calibration coefficient)” logic, which promotes constituting a uniform observing system for cross-platform infrared satellites.

Due to the powerful feature extraction capabilities of deep neural networks (DNNs), deep image hashing has extensive applications in the fields such as image authentication, copy detection and content retrieval, making its security a critical concern. Among various security metrics, collision resistance serves as a crucial indicator of deep image hashing methods. Research on collision attacks not only reveals the potential vulnerabilities of deep image hashing but also can promote the development of more robust and secure hashing methods. In this paper, we propose a novel generative collision attack scheme, which achieves several advantages over existing attack schemes based on adversarial examples. Our scheme requires no additional perturbations added to the image, and can simultaneously generate multiple hash collision images of different classes specified by the attacker. To the best of our knowledge, this is the first generative collision attack scheme effective across various deep image hashing methods. Specifically, our attack framework consists of three parts, i.e., a Hash-to-Noise Network (HTNN), a pretrained BigGAN generator and a conditional discriminator. The designed HTNN embeds the hash code of the target image and the attacker-specified generation class information into a “noise” vector. By optimizing various hash distance loss functions between the generated and target images, this “noise” guides the generator to directly generate images that meet the collision requirement. At the same time, the discriminator ensures that the generated images are visually realistic. Extensive experimental results verify that our scheme can effectively generate multiple high-quality images with attacker-specified classes, achieving the high success rate of hash collision attack and the applicability across state-of-the-art deep hashing methods.

Screenshot, which is a common tool in office work, has become a significant threat to organizations like companies and research institutions. Malicious users can easily leak sensitive information like business secrets and research data by taking a screenshot and spreading onto the Internet. While existing watermarking schemes serve as useful tools for leakage tracing, they fall short in the scenario of arbitrary screenshot. Most current methods are file-targeted, focusing on embedding watermark for a single file of one type at a time, making it hard to handle arbitrary content on screen. To address the issues above and better satisfy the need of the scenario, we propose ScreenGuard, a novel watermarking scheme targeted for the screen itself to protect arbitrary screen content shown on it. Unlike previous watermarking schemes, ScreenGuard does not modify the content itself. Instead, we generate a transparent mask template based on the watermark, tile it to the size of the screen to form a complete transparent mask, and overlay this mask onto the screen. This ensures that any screenshots taken will contain our watermark. We then train a locator and a decoder to extract watermarks from suspected leaked screenshots to trace leaks to their source. We summarized five properties that needs to be satisfied in the scenario of arbitrary screenshot (Generalizable, Unseeable, Adaptable, Robust, Dynamic) and evaluate our method on these criteria. Extensive experiments demonstrate that ScreenGuard meets these five properties effectively, showcasing its superiority and broad practical applications.

As an important carrier of information transmission in online social networks (OSNs), the authenticity protection of images is of great significance. However, the abuse of image processing technology makes its security questionable. Meantime, lossy operations adopted by OSNs will change the forgery artifacts, which brings challenges to robust image forgery detection. To address this issue, considering the suppression of lossy noise caused by transmission, a novel multi-domain probability estimation network (PRest-Net) is proposed. Firstly, we design a multi-domain probability estimation method to capture the most differentiated regional information from the spatial, residual, and wavelet domains. Since the wavelet coefficient of semantic information is larger than that of lossy noise, and the edge texture can be highlighted in the residual image, the negative effect of lossy noise would be reduced and semantic forgery traces can be exposed more easily. We further design a forgery detector composed of low-level feature extraction, high-level feature extraction, and regional edge difference learning module, which can adaptively learn rich forgery clues. Extensive experimental results are provided to validate the superiority of PRest-Net compared with existing state-of-the-art detectors in the scenarios of detecting forged images transmitted over various OSNs.

With accelerating development of artificial intelligence, neural network models have been applied in many fields. The structure designing and training for models consume vast manpower and computing resources, which are the core interests of related research institutions and enterprises. However, the high-value attributes of neural network models also attract the attention of pirates, who may steal them for illegal profits and also slightly modify model parameters to escape model piracy detection. In order to solve the problem, in this work, we propose a robust and secure model hashing method based on dynamic branch reorganization and multi-feature fusion. Detailedly, we dynamically adjust the branches in our model hashing network to extract the robust features of all kinds of model parameters for generating the hash sequence. Besides, we integrate the encryption and the feature matrix generation to a unified stage in the hash generation for resisting possible encryption escape attack. Thus, the pirated models, even with some modifications, can be correctly detected through calculating hash distances. Experimental results demonstrate the effectiveness and superiority of our model hashing method with respect to pirated model detection, non-pirated model discrimination and security.

Steganography is the art of hiding secret data into the cover media for covert communication. In recent years, more and more deep neural network (DNN)-based steganographic schemes are proposed to train steganographic networks for secret embedding and recovery, which are shown to be promising. Compared with the handcrafted steganographic tools, steganographic networks tend to be large in size. It raises concerns on how to imperceptibly and effectively transmit these networks to the sender and receiver to facilitate the covert communication. To address this issue, we propose in this paper a Purified and Unified Steganographic Network (PUSNet). It performs an ordinary machine learning task in a purified network, which could be triggered into steganographic networks for secret embedding or recovery using different keys. We formulate the construction of the PUSNet into a sparse weight filling problem to flexibly switch between the purified and steganographic networks. We further instantiate our PUSNet as an image denoising network with two steganographic networks concealed for secret image embedding and recovery. Comprehensive experiments demonstrate that our PUSNet achieves good performance on secret image embedding, secret image recovery, and image denoising in a single architecture. It is also shown to be capable of imperceptibly carrying the steganographic networks in a purified network. Code is available at https://github.com/albblgb/PUSNet

Deep models are being applied in numerous fields and have become a new important digital product. Meanwhile, previous studies have shown that deep models are vulnerable to backdoor attacks, in which compromised models return attacker-desired results when a trigger appears. Backdoor attacks severely break the trustworthiness of deep models. In this paper, we turn this weakness of deep models into a strength, and propose a novel revocable backdoor and deep model trading scenario. Specifically, we aim to compromise deep models without degrading their performance, meanwhile, we can easily detoxify poisoned models without re-training the models. We design specific mask matrices to manage the internal feature maps of the models. These mask matrices can be used to deactivate the backdoors. The revocable backdoor can be adopted in the deep model trading scenario. Sellers train models with revocable backdoors as a trial version. Buyers pay a deposit to sellers and obtain a trial version of the deep model. If buyers are satisfied with the trial version, they pay a final payment to sellers and sellers send mask matrices to buyers to withdraw revocable backdoors. We demonstrate the feasibility and robustness of our revocable backdoor by various datasets and network architectures.

Pioneer researches recognize evidences as crucial elements in fake news detection apart from patterns. Existing evidence-aware methods either require laborious pre-processing procedures to assure relevant and high-quality evidence data, or incorporate the entire spectrum of available evidences in all news cases, regardless of the quality and quantity of the retrieved data. In this paper, we propose an approach named SEE that retrieves useful information from web-searched annotation-free evidences with an early-termination mechanism. The proposed SEE is constructed by three main phases: Searching online materials using the news as a query and directly using their titles as evidences without any annotating or filtering procedure, sequentially Examining the news alongside with each piece of evidence via attention mechanisms to produce new hidden states with retrieved information, and allowing Early-termination within the examining loop by assessing whether there is adequate confidence for producing a correct prediction. We have conducted extensive experiments on datasets with unprocessed evidences, i.e., Weibo21, GossipCop, and pre-processed evidences, namely Snopes and PolitiFact. The experimental results demonstrate that the proposed method outperforms state-of-the-art approaches.

Fake images spread on online social networks can lead to public misconceptions, causing potential harm to politics, economics and social culture. Researching the authenticity of digital images is of great significance. Almost all online social networks perform multiple lossy operations on uploaded images, which will change the forgery traces and degrade the forgery detection performance of existing forensic methods. Therefore, forgery detection for images transmitted on online social networks has become a severe challenge. To solve this problem, we propose a multi-domain probability estimation network (PRest-Net). Specifically, considering that the wavelet coefficient of useful information is larger than that of lossy noise, and the image residual processing can highlight the edge texture, we design a multi-domain probability estimation method based on feature coupling and selective search to mine forgery traces in the spatial, residual, and wavelet domains. As a result, the lossy noise introduced by transmission on online social networks can be suppressed and the most differentiated regional information can be captured. Furthermore, a forgery detector based on low-level and high-level feature learning is proposed to adaptively capture rich operation clues and local edge differences between tampered and real areas, achieving fine-grained tampered region prediction. Extensive experiments show that the proposed PRest-Net outperforms existing state-of-the-art forgery detectors and is robust to online social network transmission.

Multimodal Dialogue agents are often required to respond to conversation history using both textual and visual content. Even though current dialogue studies predominantly strive to generate natural texts or images, they fall short in considering the relevance of multimodal responses within a dialogue context, consequently confining agents from making prudent choices based on multiple alternatives and their associated relevance scores for decision-making. In this paper, we present a bidirectional multimodal dialogue framework that skillfully combines the forward generation of multiple text and image response candidates with reverse selection guided by relevance scores evaluated on dialogue context, facilitating agents in selecting the most suitable multimodal responses. Specifically, the forward generation aspect of our framework leverages a stage-wise approach, first producing textual replies and composite visual descriptions from the dialogue context, followed by the generation of visual responses aligned with the descriptions. In the reverse selection process, visual responses are translated into tangible descriptive texts that, in conjunction with textual responses, are inversely tied back to the dialogue context for relevance assessment, assigning a reference score to each multimodal response candidate to assist the intelligent agent in making informed decisions. Experimental outcomes demonstrate that our proposed bidirectional dialogue response framework markedly elevates performance in both automatic and human evaluations, yielding a range of contextually fitting multimodal responses for selection.

Image steganography is the process of hiding secret data in a cover image by subtle perturbation. Recent studies show that it is feasible to use a fixed neural network for data embedding and extraction. Such Fixed Neural Network Steganography (FNNS) demonstrates favorable performance without the need for training networks, making it more practical for real-world applications. However, the stego-images generated by the existing FNNS methods exhibit high distortion, which is prone to be detected by steganalysis tools. To deal with this issue, we propose a Cover-separable Fixed Neural Network Steganography, namely Cs-FNNS. In Cs-FNNS, we propose a Steganographic Perturbation Search (SPS) algorithm to directly encode the secret data into an imperceptible perturbation, which is combined with an AI-generated cover image for transmission. Through accessing the same deep generative models, the receiver could reproduce the cover image using a pre-agreed key, to separate the perturbation in the stego-image for data decoding. such an encoding/decoding strategy focuses on the secret data and eliminates the disturbance of the cover images, hence achieving a better performance. We apply our Cs-FNNS to the steganographic field that hiding secret images within cover images. Through comprehensive experiments, we demonstrate the superior performance of the proposed method in terms of visual quality and undetectability. Moreover, we show the flexibility of our Cs-FNNS in terms of hiding multiple secret images for different receivers. Code is available at https://github.com/albblgb/Cs-FNNS

Recently, a vast number of image generation models have been proposed, which raises concerns regarding the misuse of these artificial intelligence (AI) techniques for generating fake images. To attribute the AI-generated images, existing schemes usually design and train deep neural networks (DNNs) to learn the model fingerprints, which usually requires a large amount of data for effective learning. In this paper, we aim to answer the following two questions for AI-generated image attribution, 1) is it possible to design useful handcrafted filters to facilitate the fingerprint learning? and 2) how we could reduce the amount of training data after we incorporate the handcrafted filters? We first propose a set of Multi-Directional High-Pass Filters (MHFs) which are capable to extract the subtle fingerprints from various directions. Then, we propose a Directional Enhanced Feature Learning network (DEFL) to take both the MHFs and randomly-initialized filters into consideration. The output of the DEFL is fused with the semantic features to produce a compact fingerprint. To make the compact fingerprint discriminative among different models, we propose a Dual-Margin Contrastive (DMC) loss to tune our DEFL. Finally, we propose a reference based fingerprint classification scheme for image attribution. Experimental results demonstrate that it is indeed helpful to use our MHFs for attributing the AI-generated images. The performance of our proposed method is significantly better than the state-of-the-art for both the closed-set and open-set image attribution, where only a small amount of images are required for training.

Deceptive images can quickly spread via social networking services, posing significant risks. The rapid progress in Image Manipulation Localization (IML) seeks to address this issue. However, the scarcity of public training datasets in the IML task directly hampers the performance of models. To address the challenge, we propose a Prompt-IML framework, which leverages the rich prior knowledge of pre-trained models by employing tunable prompts. Specifically, sets of tunable prompts enable the frozen pre-trained model to extract multi-view features, including spatial and high-frequency features. This approach minimizes redundant architecture for feature extraction across different views, resulting in reduced training costs. In addition, we develop a plug-and-play Feature Alignment and Fusion module that seamlessly integrates into the pre-trained models without additional structural modifications. The proposed module reduces noise and uncertainty in features through interactive processing. The experimental results showcase that our proposed method attains superior performance across 6 test datasets, demonstrating exceptional robustness.

Traditional video steganography methods are based on modifying the covert space for embedding, whereas we propose an innovative approach that embeds secret message within semantic feature for steganography during the video editing process. Although existing traditional video steganography methods excel in balancing security and capacity, they lack adequate robustness against common distortions in online social networks (OSNs). In this paper, we propose an end-to-end robust generative video steganography network (RoGVSN), which achieves visual editing by modifying semantic feature of videos to embed secret message. We exemplify the face-swapping scenario as an illustration to demonstrate the visual editing effects. Specifically, we devise an adaptive scheme to seamlessly embed secret messages into the semantic features of videos through fusion blocks. Extensive experiments demonstrate the superiority of our method in terms of robustness, extraction accuracy, visual quality, and capacity.

Recent advances have emphasized the importance of meme stickers in open-domain dialogue systems.However, previous studies overlook the one-to-many issue that a single sticker could represent various emotions in different dialogue contexts.Additionally, they require retraining the model for new stickers which did not appear in previous training.To address the above issues, we propose in this paper an Emotion-Aware and Efficient Meme Sticker Dialogue generation framework.In the framework, we design an Emotion Adaptive Prompt to capture the emotional cues from the dialogue history, which is sent to an Emotion-Aware Fusion Decoder to guide the generation of text responses and to a meme sticker selector to choose the corresponding sticker.Furthermore, to improve the stickers' selection efficiency, we further incorporate the few-shot learning strategy into the proposed framework to avoid extensive model retraining for unseen meme stickers.Through extensive experiments, we demonstrate the superior performance of the proposed E 2 MSD compared to existing methods regarding the quality of response generation and the efficiency of meme sticker retrieval.

Text-to-image models have shown surprising performance in high-quality image generation, while also raising intensified concerns about the unauthorized usage of personal dataset in training and personalized fine-tuning. Recent approaches, embedding watermarks, introducing perturbations, and inserting backdoors into datasets, rely on adding minor information vulnerable to adversarial training, limiting their ability to detect unauthorized data usage. In this paper, we introduce a novel implicit Zero-Watermarking scheme that first utilizes the disentangled style domain to detect unauthorized dataset usage in text-to-image models. Specifically, our approach generates the watermark from the disentangled style domain, enabling self-generalization and mutual exclusivity within the style domain anchored by protected units. The domain achieves the maximum concealed offset of probability distribution through both the injection of identifier $z$ and dynamic contrastive learning, facilitating the structured delineation of dataset copyright boundaries for multiple sources of styles and contents. Additionally, we introduce the concept of watermark distribution to establish a verification mechanism for copyright ownership of hybrid or partial infringements, addressing deficiencies in the traditional mechanism of dataset copyright ownership for AI mimicry. Notably, our method achieves one-sample verification for copyright ownership in AI mimic generations. The code is available at: [https://github.com/Hlufies/ZWatermarking](https://github.com/Hlufies/ZWatermarking)

Abstract Deep neural networks are vulnerable to backdoor attacks. Previous backdoor attacks have mainly focused on images. Unlike images composed of regular pixels, 3D point clouds are composed of irregular three-dimensional XYZ coordinates, which are widely used in areas such as autonomous driving and 3D measurement. As many deep neural networks have been developed for processing 3D point clouds, these networks also face the risk of backdoor attacks. Nevertheless, backdoor attacks on 3D point clouds have rarely been investigated. This paper proposes a stealthy backdoor attack on point clouds in the physical world, aiming to generate trainable non-rigid deformations as backdoor patterns. Instead of directly adding backdoor patterns onto the point clouds, we deform the 3D space of the point clouds to a new space, ensuring that all point clouds have the same backdoor deformation. We use point cloud alignment to overcome the inconsistency of backdoor deformation caused by shifting and scaling in the physical world. We also propose a physical transformation layer to combat the physical transformations. Additionally, we propose mask contrast learning to eliminate pseudo backdoor patterns to make the network’s backdoor property stealthier. Extensive experiments indicate that the proposed method can achieve better attack success rates and stealthiness.

Abstract Deep neural network model extraction attack is the process of retraining a surrogate model based on the outputs of a target model with a given set of inputs. Such attacks are hard to defend for the sake of model owners’ interest. Recently, some work propose model watermarking scheme for image processing networks, which is able to prove the intellectual property of deep models even after the model extraction attack. This scheme makes sure that, once the target model (an image processing network) is watermarked, we can extract the watermark from the output of the surrogate model. In this paper, we propose a new model extraction attack scheme to fight against the latest method. Instead of directly using the output images of a target model, we propose to use their reconstructed versions for model retraining, where an asymmetrical UNet is proposed for image reconstruction. To thoroughly remove the watermarking traces, we propose and incorporate a referenced subspace attention module in the asymmetrical UNet, which removes the watermark by projecting the outputs of the target model into the subspaces of the reference image. Various experiments demonstrate the effectiveness of our attack.

Abstract Recent studies show that neural networks are vulnerable to backdoor attacks, in which compromised networks behave normally for clean inputs but make mistakes when a pre-defined trigger appears. Although prior studies have designed various invisible triggers to avoid causing visual anomalies, they cannot evade some trigger detectors. In this paper, we consider the stealthiness of backdoor attacks from input space and feature representation space. We propose a novel backdoor attack named sparse backdoor attack, and investigate the minimum required trigger to induce the well-trained networks to make incorrect results. A U-net-based generator is employed to create triggers for each clean image. Considering the stealthiness of the trigger, we restrict the elements of the trigger between −1 and 1. In the aspect of the feature representation domain, we adopt an entanglement cost function to minimize the gap between feature representations of benign and malicious inputs. The inseparability of benign and malicious feature representations contributes to the stealthiness of our attack against various model diagnosis-based defences. We validate the effectiveness and generalization of our method by conducting extensive experiments on multiple datasets and networks.

The ability to detect multiple steganographic methods in speech streams is an important prerequisite for steganalysis methods to move from theory to practical application, but it is also a challenging problem. To address this challenge, we propose a novel steganalysis method based on multi-agent deep learning, which can effectively detect multiple steganography methods in speech streams. Our method utilizes multiple agents to learn the features of multiple sub-training datasets separately and then fuses the information of each agent through the weight parameter aggregation mechanism to obtain the final weight parameter of the steganalysis model. Experimental results show that our proposed method outperforms the state-of-art steganalysis methods. In particular, for low embedding rates, the presented method increases average detection accuracy by about 9%.

Generative steganography, a novel paradigm in information hiding, has garnered considerable attention for its potential to withstand steganalysis. However, existing generative steganography approaches suffer from the limited visual quality of generated images and are challenging to apply to lossy transmissions in real-world scenarios with unknown channel attacks. To address these issues, this paper proposes a novel robust generative image steganography scheme, facilitating zero-shot text-driven stego image generation without the need for additional training or fine-tuning. Specifically, we employ the popular Stable Diffusion model as the backbone generative network to establish a covert transmission channel. Our proposed framework overcomes the challenges of numerical instability and perturbation sensitivity inherent in diffusion models. Adhering to Kerckhoff’s principle, we propose a novel mapping module based on dual keys to enhance robustness and security under lossy transmission conditions. Experimental results showcase the superior performance of our method in terms of extraction accuracy, robustness, security, and image quality.

Steganography is a technique for covert communication between two parties. With the rapid development of deep neural networks (DNN), more and more steganographic networks are proposed recently, which are shown to be promising to achieve good performance. Unlike the traditional handcrafted steganographic tools, a steganographic network is relatively large in size. It raises concerns on how to covertly transmit the steganographic network in public channels, which is a crucial stage in the pipeline of steganography in real world applications. To address such an issue, we propose a novel scheme for steganography of steganographic networks in this paper. Unlike the existing steganographic schemes which focus on the subtle modification of the cover data to accommodate the secrets. We propose to disguise a steganographic network (termed as the secret DNN model) into a stego DNN model which performs an ordinary machine learning task (termed as the stego task). During the model disguising, we select and tune a subset of filters in the secret DNN model to preserve its function on the secret task, where the remaining filters are reactivated according to a partial optimization strategy to disguise the whole secret DNN model into a stego DNN model. The secret DNN model can be recovered from the stego DNN model when needed. Various experiments have been conducted to demonstrate the advantage of our proposed method for covert communication of steganographic networks as well as general DNN models.

Previous researches on multimedia fake news detection include a series of complex feature extraction and fusion networks to gather useful information from the news. However, how cross-modal consistency relates to the fidelity of news and how features from different modalities affect the decision-making are still open questions. This paper presents a novel scheme of Bootstrapping Multi-view Representations (BMR) for fake news detection. Given a multi-modal news, we extract representations respectively from the views of the text, the image pattern and the image semantics. Improved Multi-gate Mixture-of-Expert networks (iMMoE) are proposed for feature refinement and fusion. Representations from each view are separately used to coarsely predict the fidelity of the whole news, and the multimodal representations are able to predict the cross-modal consistency. With the prediction scores, we reweigh each view of the representations and bootstrap them for fake news detection. Extensive experiments conducted on typical fake news detection datasets prove that BMR outperforms state-of-the-art schemes.

Multimodal Dialogue agents are often required to respond to conversation history using both textual and visual content. Even though current dialogue studies predominantly strive to generate natural texts or images, they fall short in considering the relevance of multimodal responses within a dialogue context, consequently confining agents from making prudent choices based on multiple alternatives and their associated relevance scores for decision-making. In this paper, we present a bidirectional multimodal dialogue framework that skillfully combines the forward generation of multiple text and image response candidates with reverse selection guided by relevance scores evaluated on dialogue context, facilitating agents in selecting the most suitable multimodal responses. Specifically, the forward generation aspect of our framework leverages a stage-wise approach, first producing textual replies and composite visual descriptions from the dialogue context, followed by the generation of visual responses aligned with the descriptions. In the reverse selection process, visual responses are translated into tangible descriptive texts that, in conjunction with textual responses, are inversely tied back to the dialogue context for relevance assessment, assigning a reference score to each multimodal response candidate to assist the intelligent agent in making informed decisions. Experimental outcomes demonstrate that our proposed bidirectional dialogue response framework markedly elevates performance in both automatic and human evaluations, yielding a range of contextually fitting multimodal responses for selection.

RAW files are the initial measurement of scene radiance widely used in most cameras, and the ubiquitously-used RGB images are converted from RAW data through Image Signal Processing (ISP) pipelines. Nowadays, digital images are risky of being nefariously manipulated. Inspired by the fact that innate immunity is the first line of body defense, we propose DRAW, a novel scheme of defending images against manipulation by protecting their sources, i.e., camera-shooted RAWs. Specifically, we design a lightweight Multi-frequency Partial Fusion Network (MPF-Net) friendly to devices with limited computing resources by frequency learning and partial feature fusion. It introduces invisible watermarks as protective signal into the RAW data. The protection capability can not only be transferred into the rendered RGB images regardless of the applied ISP pipeline, but also is resilient to post-processing operations such as blurring or compression. Once the image is manipulated, we can accurately identify the forged areas with a localization network. Extensive experiments on several famous RAW datasets, e.g., RAISE, FiveK and SIDD, indicate the effectiveness of our method. We hope that this technique can be used in future cameras as an option for image protection, which could effectively restrict image manipulation at the source.

Existing image cropping detection schemes ignore that recovering the cropped-out contents can unveil the purpose of the behaved cropping attack. This paper presents CLR-Net, a novel image protection scheme addressing the combined challenge of image Cropping Localization and Recovery. We first protect the original image by introducing imperceptible perturbations. Then, typical image post-processing attacks are simulated to erode the protected image. On the recipient’s side, we predict the cropping mask and recover the original image. Besides, we propose a novel Fine-Grained generative JPEG simulator (FG-JPEG) as well as a feature alignment network to improve the real-world robustness. Comprehensive experiments prove that the quality of the recovered image and the accuracy of crop localization are both satisfactory.

Model watermarking has become an important solution to protect the intellectual property right (IPR) of deep neural networks (DNN) models. However, there are few researches on the IPR protection of generative adversarial networks (GAN), which are widely used to generate photorealistic images. In the current backdoor-based watermarking method for GAN models, the trigger pattern of the watermark is easy to be detected and invalidated by the adversary, which may fail to achieve IPR protection. To address this drawback, we propose a new GAN model watermarking method, where an invisible backdoor based on steganography is injected into the target GAN model as a watermark. Experimentally, the proposed method effectively trades off the performance of GAN on original task and the robustness of watermarking for removal attacks. Moreover, the generated triggers can effectively resist being detected by attackers.

Fake news detection (FND) has attracted much research interests in social forensics. Many existing approaches introduce tailored attention mechanisms to fuse unimodal features. However, they ignore the impact of cross-modal similarity between modalities. Meanwhile, the potential of pretrained multimodal feature learning models in FND has not been well exploited. This paper proposes an FND-CLIP framework, i.e., a multimodal Fake News Detection network based on Contrastive Language-Image Pretraining (CLIP). FND-CLIP extracts the deep representations together from news using two unimodal encoders and two pair-wise CLIP encoders. The CLIP-generated multimodal features are weighted by CLIP similarity of the two modalities. We also introduce a modality-wise attention module to aggregate the features. Extensive experiments are conducted and the results indicate that the proposed framework has a better capability in mining crucial features for fake news detection. The proposed FND-CLIP can achieve better performances than previous works on three typical fake news datasets.

The easy sharing of multimedia content on social media has caused a rapid dissemination of fake news, which threatens society’s stability and security. Therefore, fake news detection has garnered extensive research interest in the field of social forensics. Current methods primarily concentrate on the integration of textual and visual features but fail to effectively exploit multi-modal information at both fine-grained and coarse-grained levels. Furthermore, they suffer from an ambiguity problem due to a lack of correlation between modalities or a contradiction between the decisions made by each modality. To overcome these challenges, we present a Multi-grained Multi-modal Fusion Network (MMFN) for fake news detection. Inspired by the multi-grained process of human assessment of news authenticity, we respectively employ two Transformer-based pre-trained models to encode token-level features from text and images. The multi-modal module fuses fine-grained features, taking into account coarse-grained features encoded by the CLIP encoder. To address the ambiguity problem, we design uni-modal branches with similarity-based weighting to adaptively adjust the use of multi-modal features. Experimental results demonstrate that the proposed framework outperforms state-of-the-art methods on three prevalent datasets.

Backdoor attack aims to compromise a model, which returns an adversary-wanted output when a specific trigger pattern appears yet behaves normally for clean inputs. Current backdoor attacks require changing pixels of clean images, which results in poor stealthiness of attacks and increases the difficulty of the physical implementation. This paper proposes a novel physical invisible backdoor based on camera imaging without changing nature image pixels. Specifically, a compromised model returns a target label for images taken by a particular camera, while it returns correct results for other images. To implement and evaluate the proposed backdoor, we take shots of different objects from multi-angles using multiple smartphones to build a new dataset of 21,500 images. Conventional backdoor attacks work ineffectively with some classical models, such as ResNet18, over the above-mentioned dataset. Therefore, we propose a three-step training strategy to mount the backdoor attack. First, we design and train a camera identification model with the phone IDs to extract the camera fingerprint feature. Subsequently, we elaborate a special network architecture, which is easily compromised by our backdoor attack, by leveraging the attributes of the CFA interpolation algorithm and combining it with the feature extraction block in the camera identification model. Finally, we transfer the backdoor from the elaborated special network architecture to the classical architecture model via teacher-student distillation learning. Since the trigger of our method is related to the specific phone, our attack works effectively in the physical world. Experiment results demonstrate the feasibility of our proposed approach and robustness against various backdoor defences.

Recently, print-resist watermarking has attracted much interest. Many watermarking schemes have been proposed to achieve robustness against printing and camera-capturing. Though these studies have shown promising results overall, they overlook the scenario of film-coating photographs, which is a significant and common scenario in real-world. The film-coating process can introduce severe distortions to the original image and easily incapacitate the watermark. To address this issue, we propose WRAP, a novel Watermarking scheme Robust Against film-coating upon Printed photographs. We first construct a large dataset with 120,000 film-coating images to train a style-transfer-based film-coating simulation network. Based on the network, we propose a comprehensive distortion layer which includes film-coating simulation and common disturbances in the printing and camera-capturing process. With the distortion layer, the entire embedding and extraction network can be trained end-to-end to gain robustness against film-coating upon printed photographs. Extensive experiments demonstrate the superior performances of our model in terms of robustness and generalization capability. Our model outperforms state-of-the-art print-resist watermarking schemes when testing in film-coating scenario and achieves outstanding performance across various datasets, types of films, and cameras. To the best of our knowledge, we are the first to conduct research on digital watermarking in film-coating scenario.

Federated learning provides a privacy-preserving manner to collaboratively train models on data distributed over multiple local clients via the coordination of a global server. In this paper, we focus on label distribution skew in federated learning, where due to the different user behavior of the client, label distributions between different clients are significantly different. When faced with such cases, most existing methods will lead to a suboptimal optimization due to the inadequate utilization of label distribution information in clients. Inspired by this, we propose a label-masking distillation approach termed FedLMD to facilitate federated learning via perceiving the various label distributions of each client. We classify the labels into majority and minority labels based on the number of examples per class during training. The client model learns the knowledge of majority labels from local data. The process of distillation masks out the predictions of majority labels from the global model, so that it can focus more on preserving the minority label knowledge of the client. A series of experiments show that the proposed approach can achieve state-of-the-art performance in various cases. Moreover, considering the limited resources of the clients, we propose a variant FedLMD-Tf that does not require an additional teacher, which outperforms previous lightweight approaches without increasing computational costs. Our code is available at https://github.com/wnma3mz/FedLMD.

Image steganography is the art of concealing secret information in images in a way that is imperceptible to unauthorized parties. Recent advances show that is possible to use a fixed neural network (FNN) for secret embedding and extraction. Such fixed neural network steganography (FNNS) achieves high steganographic performance without training the networks, which could be more useful in real-world applications. However, the existing FNNS schemes are vulnerable in the sense that anyone can extract the secret from the stego-image. To deal with this issue, we propose a key-based FNNS scheme to improve the security of the FNNS, where we generate key-controlled perturbations from the FNN for data embedding. As such, only the receiver who possesses the key is able to correctly extract the secret from the stego-image using the FNN. In order to improve the visual quality and undetectability of the stego-image, we further propose an adaptive perturbation optimization strategy by taking the perturbation cost into account. Experimental results show that our proposed scheme is capable of preventing unauthorized secret extraction from the stego-images. Furthermore, our scheme is able to generate stego-images with higher visual quality than the state-of-the-art FNNS scheme, especially when the FNN is a neural network for ordinary learning tasks.

Deep neural network (DNN) watermarking is an emerging technique to protect the intellectual property of deep learning models. At present, many DNN watermarking algorithms have been proposed to achieve provenance verification by embedding identify information into the internals or prediction behaviors of the host model. However, most methods are vulnerable to model extraction attacks, where attackers collect output labels from the model to train a surrogate or a replica. To address this issue, we present a novel DNN watermarking approach, named SSW, which constructs an adaptive trigger set progressively by optimizing over a pair of symmetric shadow models to enhance the robustness to model extraction. Precisely, we train a positive shadow model supervised by the prediction of the host model to mimic the behaviors of potential surrogate models. Additionally, a negative shadow model is normally trained to imitate irrelevant independent models. Using this pair of shadow models as a reference, we design a strategy to update the trigger samples appropriately such that they tend to persist in the host model and its stolen copies. Moreover, our method could well support two specific embedding schemes: embedding the watermark via fine-tuning or from scratch. Our extensive experimental results on popular datasets demonstrate that our SSW approach outperforms state-of-the-art methods against various model extraction attacks in whether trigger set classification accuracy based or hypothesis test based verification. The results also show that our method is robust to common model modification schemes including fine-tuning and model compression.

Many online applications require the users to upload their identity documents for authentication. The fake identity document is one of the main threats which compromises the security and reliability of such online applications. Existing techniques focus on the detection of digitally forged identity documents, which neglect the impact of physical forgeries. In this paper, we look into the problem of detecting physically occluded fake identity documents, which can be easily generated without any image processing knowledge. We observe that the physical occlusions inevitably produce occluded boundaries on the document. To take the advantage, we propose an Occluded Boundary Representation Learning (OBRL) module to progressively learn the occluded boundary features. These are then fed into an Occluded Boundary Message Passing (OBMP) module to effectively diffuse the physical occlusion traces to enhance the backbone features for robust detection. We newly construct a Physically Occluded Fake ID Card image dataset (POID) for evaluation. Various experiments are conducted on the POID, where our scheme is able to achieve 99.6% of accuracy in detecting physically occluded fake ID card images with a mAP of over 85% to localize the occlusion regions.

The widespread use of face retouching filters on short-video platforms has raised concerns about the authenticity of digital appearances and the impact of deceptive advertising. To address these issues, there is a pressing need to develop advanced face retouching techniques. However, the lack of large-scale and fine-grained face retouching datasets has been a major obstacle to progress in this field. In this paper, we introduce RetouchingFFHQ, a large-scale and fine-grained face retouching dataset that contains over half a million conditionally-retouched images. RetouchingFFHQ stands out from previous datasets due to its large scale, high quality, fine-grainedness, and customization. By including four typical types of face retouching operations and different retouching levels, we extend the binary face retouching detection into a fine-grained, multi-retouching type, and multi-retouching level estimation problem. Additionally, we propose a Multi-granularity Attention Module (MAM) as a plugin for CNN backbones for enhanced cross-scale representation learning. Extensive experiments using different baselines as well as our proposed method on RetouchingFFHQ show decent performance on face retouching detection.

Live video commenting, or "bullet screen," is a popular social style on video platforms. Automatic live commenting has been explored as a promising approach to enhance the appeal of videos. However, existing methods neglect the diversity of generated sentences, limiting the potential to obtain human-like comments. In this paper, we introduce a novel framework called "VCMaster" for multimodal live video comments generation, which balances the diversity and quality of generated comments to create human-like sentences. We involve images, subtitles, and contextual comments as inputs to better understand complex video contexts. Then, we propose an effective Hierarchical Cross-Fusion Decoder to integrate high-quality trimodal feature representations by cross-fusing critical information from previous layers. Additionally, we develop a Sentence-Level Contrastive Loss to enlarge the distance between generated and contextual comments by contrastive learning. It helps the model to avoid the pitfall of simply imitating provided contextual comments and losing creativity, encouraging the model to achieve more diverse comments while maintaining high quality. We also construct a large-scale multimodal live video comments dataset with 292,507 comments and three sub-datasets that cover nine general categories. Extensive experiments demonstrate that our model achieves a level of human-like language expression and remarkably fluent, diverse, and engaging generated comments compared to baselines.

Digital images are vulnerable to nefarious tampering attacks such as content addition or removal that severely alter the original meaning. It is somehow like a person without protection that is open to various kinds of viruses. Image immunization (Imuge) is a technology of protecting the images by introducing trivial perturbation, so that the protected images are immune to the viruses in that the tampered contents can be auto-recovered. This paper presents Imuge+, an enhanced scheme for image immunization. By observing the invertible relationship between image immunization and the corresponding self-recovery, we employ an invertible neural network to jointly learn image immunization and recovery respectively in the forward and backward pass. We also introduce an efficient attack layer that involves both malicious tamper and benign image post-processing, where a novel distillation-based JPEG simulator is proposed for improved JPEG robustness. Our method achieves promising results in real-world tests where experiments show accurate tamper localization as well as high-fidelity content recovery. Additionally, we show superior performance on tamper localization compared to state-of-the-art schemes based on passive forensics.

Steganography has received massive attention from the information-hiding community due to its excellent security for covert communication systems. Existing work focuses on improving security on single-modal media while cross-modal media is less explored. However, cross-modal interaction has become a prevalent social manner on current social networks, which arises potential behavioral security issues of single-modal steganography. In this letter, we propose a novel text steganography to explore the practicability of cross-modal steganography. The proposed scheme is composed with image encoder, message encoder, language model, and message extractor networks, where the generated stego texts are semantically consistent with the input reference image. In addition, current generative text steganography schemes are vulnerable to text attack based on synonym substitution since these heuristic algorithms embed information by constructing a mapping between secret messages and candidate tokens. Thus, we design a text attack layer based on synonym substitution to further improve the robustness of generated stego text. Experiments illustrate the superior performance of the proposed cross-modal steganography scheme in terms of security and robustness.

Recent methods with deep neural networks for text steganalysis have succeeded in mining various feature representations. However, a limited number of studies have explicitly analyzed potential security issues of generative text steganography. Furthermore, current text steganalysis approaches lack detailed consideration in the intricate design of deep learning architectures tailored to these challenges. In this article, in order to tackle these problems, we first theoretically and empirically analyze the inevitable embedding distortions of generative text steganography at a semantic and statistical levels. In light of this, we then propose an innovative text steganalysis method based on hierarchical supervised learning and a dual attention mechanism. Concretely, to extract highly effective semantic features, the proposed method involves fine-tuning a BERT extractor through the hierarchical supervised learning that combines signals from multiple softmax classifiers, rather than relying solely on the final one. The mean and standard deviation values in the Gaussian distribution of cover and stego texts are then estimated using an encoder of variational autoencoders and used to capture features representing the statistical distortion of generative text steganography. Subsequently, we introduce a dual attention mechanism that dynamically fuses the semantic and statistical features, thereby creating discriminative feature representations essential for text steganalysis. The experimental results demonstrate that our proposed text steganalysis method surpasses the current state-of-the-art techniques across three distinct text steganalysis scenarios: specific text steganalysis, semi-blind text steganalysis, and blind text steganalysis.

The real-time detection of speech steganography in Voice-over-Internet-Protocol (VoIP) scenarios remains an open problem, as it requires steganalysis methods to perform for low-intensity embeddings and short-sample inputs, as well as provide rapid detection results. To address these challenges, this paper presents a novel steganalysis model based on spatial and temporal feature fusion (STFF-SM). Differing from the existing methods, we take both the integer and fractional pitch delays as input, and design subframe-stitch module to organically integrate subframe-wise integer delays and frame-wise fractional pitch delays. Further, we design a spatial fusion module based on pre-activation residual convolution to extract the pitch spatial features and gradually increase their dimensions to discover finer steganographic distortions to enhance the detection effect, where a Group-Squeeze-Weighting block is introduced to alleviate the information loss in the process of increasing the feature dimension. In addition, we design a temporal fusion module to extract pitch temporal features using the stacked LSTM, where a Gated Feed-Forward Network is introduced to learn the interaction between different feature maps while suppressing the features that are not useful for detection. We evaluated the performance of STFF-SM through comprehensive experiments and comparisons with the state-of-the-art solutions. The experimental results demonstrate that STFF-SM can well meet the needs of real-time detection of speech steganography in VoIP streams, and outperforms the existing methods in detection performance, especially with low embedding strengths and short window sizes.

Image forgery detection has aroused widespread research interest in both academia and industry because of its potential security threats. Existing forgery detection methods achieve excellent tampered regions localization performance when forged images have not undergone post-processing, which can be detected by observing changes in the statistical features of images. However, forged images may be carefully post-processed to conceal forgery boundaries in a particular scenario. It becomes tough challenging to these methods. In this paper, we perform an analogous analysis between image forgery detection and blind signal separation, and formulate the post-processed image forgery detection problem into a signal noise separation problem. We also propose a signal noise separation-based (SNIS) network to solve the problem of detecting post-processed image forgery. Specifically, we first adopt the signal noise separation module to separate tampered region from the complex background region with post-processing noise, which weakens or even eliminates the negative impact of post-processing on forgery detection. Then, the multi-scale feature learning module uses a parallel atrous convolution architecture to learn high-level global features from multiple perspectives. Besides, a feature fusion module is utilized to enhance the discriminability of tampered regions and real regions by strengthening the boundary information. Finally, the prediction module is designed to predict the tampered region and classify the type of tampering operation. Extensive experiments show that the proposed SNIS is not only effective for forgery detection on forged images without post-processing, but also promising in robustness against multiple post-processing attacks. Furthermore, SNIS is robust in detecting forged images from unknown sources.

With the development of robust information hiding (RIH) approaches, secret messages can be extracted successfully from stego-data after transmission through lossy channels of online social networks (OSNs). To interrupt illegal covert communications in OSNs, some methods sanitize the uploaded images by image processing operations to destroy the hidden data that may exist. However, none of the existing methods takes the RIH methods that can resist scaling into consideration, while scaling is a common operation in OSNs. In this paper, we first propose a general framework for image sanitization in OSN platforms, which serves as a countermeasure against the RIH. By using such a framework, the secret messages embedded in the upload images can be removed and the quality of the sanitized image can be well maintained. Our framework contains two deep neural networks: Scaling-Net and SC-Net. The Scaling-Net is dedicated to the sanitization of oversized images while the SC-Net is designed for other images. To achieve a good image quality, we also propose a discriminator for adversarial training of the Scaling-Net and SC-Net. Experimental results on different datasets demonstrate that our proposed method outperforms the state-of-the-art methods. The source code and pretrained models are available at our code repository (https://github.com/zyzhu19/Image_Sanitization).

Content similarity is a representative property of natural images, for example, similar regions, which is utilized by modern steganalysis. Existing JPEG steganographic methods mainly focus on the complexity of content but ignore content similarity. This article investigates content similarity to improve the undetectability of JPEG steganography. Specifically, the content similarity of DCT blocks and the 64 parallel channels is used to design the distortion function. Given a JPEG image, initial embedding costs are assigned for quantized DCT coefficients using an appropriate algorithm among the existing distortion functions. Then, the similarities of blocks and channels are used to update the initial embedding costs, respectively. After combination, the final distortion function can be obtained. Using syndrome trellis coding (STC), which achieves minimal embedding distortion with respect to a given distortion function, secret data are embedded into the cover image with a final distortion function. Experimental results show that our scheme achieves better undetectability than current state-of-the-art JPEG steganographic methods.

JPEG image encryption aims at effectively converting the original JPEG image into a noise-like image that does not contain any useful information of original image. Existing schemes for JPEG image encryption, however, may not attain a good balance in terms of file size increment and encryption security. To address the problem, we design a novel JPEG image encryption scheme. Different from existing schemes, we first predict DC coefficients by an adaptive prediction method. Subsequently, the histogram of DC coefficient prediction errors is encrypted by combining the prediction errors and random integers to reduce the encoded length, which can ensure a very small increment of file size. Furthermore, we construct the RS (run/size) pairs in each DCT block and then implement the permutation for both RS pairs extracted from the upper left corner of each DCT block and all DCT blocks excluding DC coefficients, which can further distort the image contents. Extensive experiments demonstrate that, compared with existing JPEG image encryption schemes, our scheme can ensure not only the JPEG format compatibility for encrypted image, but also keep a very small file size increment and the superior security performance.

Detection of manipulated face images has attracted a lot of interest recently. Various schemes have been proposed to tackle this challenging problem, where the patch-based approaches are shown to be promising. However, the existing patch-based approaches tend to treat different patches equally, which do not fully exploit the patch discrepancy for effective feature learning. In this paper, we propose a Patch Diffusion (PD) module which can be integrated into the existing face manipulation detection networks to boost the performance. The PD consists of Discrepancy Patch Feature Learning (DPFL) and Attention-Aware Message Passing (AMP). The DPFL effectively learns the patch features by a newly designed Pairwise Patch Loss (PPLoss), which takes both the patch importance and correlations into consideration. The AMP diffuses the patches through attention-aware message passing in a graph network, where the attentions are explicitly computed based on the patch features learnt in DPFL. We integrate our PD module into four recent face manipulation detection networks, and carry out the experiments on four popular datasets. The results demonstrate that our PD module is able to boost the performance of the existing networks for face manipulation detection.

Research shows that deep neural networks are vulnerable to back-door attacks. The backdoor network behaves normally on clean examples, but once backdoor patterns are attached to examples, back-door examples will be classified into the target class. In the previous backdoor attack schemes, backdoor patterns are not stealthy and may be detected. Thus, to achieve the stealthiness of backdoor patterns, we explore an invisible and example-dependent backdoor attack scheme. Specifically, we employ the backdoor generation network to generate the invisible backdoor pattern for each example, and backdoor patterns are not generic to each other. However, without other measures, the backdoor attack scheme cannot bypass the neural cleanse detection. Thus, we propose adversarial training to bypass neural cleanse detection. Experiments show that the proposed backdoor attack achieves a considerable attack success rate, invisibility, and can bypass the existing defense strategies.

Deep neural network (DNN) watermarking is one of the main techniques to protect the DNN. Although various DNN watermarking schemes have been proposed, none of them is able to resist the DNN encryption. In this paper, we propose an encryption resistent DNN watermarking scheme, which is able to resist the parameter shuffling based DNN encryption. Unlike the existing schemes which use the kernels separately for watermarking embedding, we propose to embed the watermark into the fused kernels to resist the parameter shuffling. We further propose a MappingNet to map the the fused kernels into a higher dimension to increase the watermarking capacity. The MappingNet and the DNN are jointly trained to conduct final watermark embedding. Experimental results indicate the effectiveness of our proposed scheme for resisting the DNN encryption.

Backdoor attack against image classification task has been widely studied and proven to be successful, while there exist few researches on backdoor attack against vision-language models. In this paper, we explore backdoor attack towards image captioning models by poisoning training data. Assuming the attacker has total access to the training dataset, and cannot intervene in model construction or training process. Specifically, a portion of benign training samples is randomly selected to be poisoned. Afterwards, considering that the captions are usually unfolded around objects in an image, we design an object-oriented method to craft poisons, which aims to modify pixel values by a slight range with the modification number proportional to the scale of the current detected object region. After training with the poisoned data, the attacked model behaves normally on benign images, but for poisoned images, the model will generate some sentences irrelevant to the given image. The attack controls the model behavior on specific test images without scarifying the generation performance on benign test images. Our method proves the weakness of image captioning models to backdoor attack and we hope this work can raise the awareness of defending against backdoor attack in the image captioning field.

Recent research has shown that deep learning based methods offer more accurate detection for image steganalysis than the traditional detection paradigm based on rich media models. Existing network architectures based on deep learning, however, stack more and more convolutional layers to increase local receptive fields for image stegananlysis. Limited by hardware, the detector with several convolutional layers may not extract features of steganography images from a global perspective effectively. In this paper, we propose a Convolutional Vision Transformer for image stegananlysis, which can capture both local and global dependencies among noise features. In image processing phase, our network preserves CNN frame for its capacity of producing image noise residuals. Different from previous methods, we utilize the attention mechanism of vision transformer for feature extraction and classification. The proposed network is validated on two public image datasets (BOSSbase 1.01 and ALASKA #2). Experimental results demonstrate that our network performs well over fixed-size dataset and arbitrary-size dataset.

A large number of multi-party conversation scenarios exist in social networks, which have been seldom studied in the field of human-machine conversation. In this paper, we study a novel task of joint learning for addressee selection and response generation in multi-party conversations. Systems are expected to select whom they address and generate the corresponding response. To solve it, we propose an end-to-end addressee selection and response generation (ASRG) model, containing an addressee selection module and a response generation module. In the selection module, we develop an addressee prediction attention scheme to obtain a unique context vector for each candidate, thereby calculating the probability of the candidate more accurately. In the generation module, we propose a Focus Transformer to generate responses. These two modules are jointly learnt to fully explore the correlations between addressee and response. Experimental results show ASRG remarkably outperforms baselines and generates relevant content for different addressees.

The existing image embedding networks are basically vulnerable to malicious attacks such as JPEG compression and noise adding, not applicable for real-world copyright protection tasks. To solve this problem, we introduce a generative deep network based method for hiding images into images while assuring high-quality extraction from the destructive synthesized images. An embedding network is sequentially concatenated with an attack layer, a decoupling network and an image extraction network. The addition of decoupling network learns to extract the embedded secret image from the attacked image. We also pinpoint the weaknesses of the adversarial training for robustness in previous works and build our improved real-world attack simulator. Experimental results demonstrate the superiority of the proposed method against typical digital attacks by a large margin, as well as the performance boost of the recovered images with the aid of progressive recovery strategy. Besides, we are the first to robustly hide three secret images.

Image cropping can be maliciously used to manipulate the layout of an image and alter the underlying meaning. Previous image cropping detection schemes only predict whether an image has been cropped, ignoring which part of the image is cropped. This paper presents a novel robust watermarking network for image cropping localization. We train an anti-cropping processor (ACP) that embeds a watermark into a target image. The visually indistinguishable protected image is then posted on the social network instead of the original image. At the recipient’s side, ACP extracts the watermark from the attacked image, and we conduct feature matching on the original and extracted watermark to locate the position of the cropping. We further extend our scheme to detect tampering attacks on the attacked image, and a simple yet efficient method (JPEG-Mixup) is proposed that noticeably improves the generalization of JPEG robustness. We demonstrate that our scheme is the first to provide high-accuracy and robust image cropping localization.

The security of data storage is a big issue for companies. They must take effective steps to prevent valuable image datasets from being stolen for illegal commercial purposes. While data encryption is a common solution, it drastically down-grades the visual quality and therefore forbids common yet trivial use such as eye-checking without a decryption. We present a novel solution for dataset protection in this scenario by robustly and reversibly transform the images into adver-sarial images. An invertible Image Dataset Protection NET-work (IDP-Net) is developed to introduce slight and acceptable changes to the images within the dataset. The protected images can be published and circulated on the social networks instead of their original version. Malicious attackers can only observe the images but cannot train pirated models based on them. Meanwhile, IDP-Net ensures the performance of au-thorized models, namely, trusted users can revert the protection and retrieve the protected images to their original version. Therefore, the dataset can be stored within the pro-tected version alone to ensure safety. Extensive experiments demonstrate that IDP-Net can better protect the security of image dataset against defensive methods compared to previ-ous methods. Besides, the introduced distortion is acceptable and the original images can be reconstructed nearly error-free.

Different from supervised learning which requires all training examples to be labeled, Semi-Supervised Learning (SSL) learns from a few labeled training examples and a large number of unlabeled training examples. Recently, studies have shown that SSL is also vulnerable to backdoor attacks. However, their performance is poor. In this paper, we propose a novel unlabeled backdoor poisoning attack against SSL, where only poisoning unlabeled examples in the training set to inject the backdoor into the network. Specifically, our attack exploits the vulnerability of SSL algorithms in guessing pseudo labels of unlabeled examples. We propose a backdoor generation network to generate poisoned examples with both the backdoor property and misleading function, thus inducing the victim model itself to mislabel the poisoned examples as the target class and causing the backdoor to be injected. Our attack achieves favorable attack success rates on the SSL algorithm while bypassing backdoor defenses.

Generative steganography (GS) is a new data hiding manner, featuring direct generation of stego media from secret data. Existing GS methods are generally criticized for their poor performances. In this paper, we propose a novel flow based GS approach - Generative Steganographic Flow (GSF), which provides direct generation of stego images without cover image. We take the stego image generation and secret data recovery process as an invertible transformation, and build a reversible bijective mapping between input secret data and generated stego images. In the forward mapping, secret data is hidden in the input latent of Glow model to generate stego images. By reversing the mapping, hidden data can be extracted exactly from generated stego images. Furthermore, we propose a novel latent optimization strategy to improve the fidelity of stego images. Experimental results show our proposed GSF has far better performances than SOTA works.

Backdoor attacks are rapidly emerging threats to deep neural networks (DNNs). In the backdoor attack scenario, attackers usually implant the backdoor into the target model by manipulating the training dataset or training process. Then, the compromised model behaves normally for benign input yet makes mistakes when the pre-defined trigger appears. In this paper, we analyze the drawbacks of existing attack approaches and propose a novel imperceptible backdoor attack. We treat the trigger pattern as a special kind of noise following a multinomial distribution. A U-net-based network is employed to generate concrete parameters of multinomial distribution for each benign input. This elaborated trigger ensures that our approach is invisible to both humans and statistical detection. Besides the design of the trigger, we also consider the robustness of our approach against model diagnose-based defences. We force the feature representation of malicious input stamped with the trigger to be entangled with the benign one. We demonstrate the effectiveness and robustness against multiple state-of-the-art defences through extensive datasets and networks. Our trigger only modifies less than 1\% pixels of a benign image while the modification magnitude is 1. Our source code is available at https://github.com/Ekko-zn/IJCAI2022-Backdoor.

Steganography usually modifies cover media to embed secret data. A new steganographic approach called generative steganography (GS) has emerged recently, in which stego images (images containing secret data) are generated from secret data directly without cover media. However, existing GS schemes are often criticized for their poor performances. In this paper, we propose an advanced generative steganography network (GSN) that can generate realistic stego images without using cover images. We firstly introduce the mutual information mechanism in GS, which helps to achieve high secret extraction accuracy. Our model contains four sub-networks, i.e., an image generator (G), a discriminator (D), a steganalyzer (S), and a data extractor (E). D and S act as two adversarial discriminators to ensure the visual quality and security of generated stego images. E is to extract the hidden secret from generated stego images. The generator G is flexibly constructed to synthesize either cover or stego images with different inputs. It facilitates covert communication by concealing the function of generating stego images in a normal generator. A module named secret block is designed to hide secret data in the feature maps during image generation, with which high hiding capacity and image fidelity are achieved. In addition, a novel hierarchical gradient decay (HGD) skill is developed to resist steganalysis detection. Experiments demonstrate the superiority of our work over existing methods.

Online social networks have stimulated communications over the Internet more than ever, making it possible for secret message transmission over such noisy channels. In this paper, we propose a Coverless Image Steganography Network, called CIS-Net, that synthesizes a high-quality image directly conditioned on the secret message to transfer. CIS-Net is composed of four modules, namely, the Generation, Adversarial, Extraction, and Noise Module. The receiver can extract the hidden message without any loss even the images have been distorted by JPEG compression attacks. To disguise the behaviour of steganography, we collected images in the context of profile photos and stickers and train our network accordingly. As such, the generated images are more inclined to escape from malicious detection and attack. The distinctions from previous image steganography methods are majorly the robustness and losslessness against diverse attacks. Experiments over diverse public datasets have manifested the superior ability of anti-steganalysis.

Face anonymization with generative models have become increasingly prevalent since they sanitize private information by generating virtual face images, ensuring both privacy and image utility. Such virtual face images are usually not identifiable after the removal or protection of the original identity. In this paper, we formalize and tackle the problem of generating identifiable virtual face images. Our virtual face images are visually different from the original ones for privacy protection. In addition, they are bound with new virtual identities, which can be directly used for face recognition. We propose an Identifiable Virtual Face Generator (IVFG) to generate the virtual face images. The IVFG projects the latent vectors of the original face images into virtual ones according to a user specific key, based on which the virtual face images are generated. To make the virtual face images identifiable, we propose a multi-task learning objective as well as a triplet styled training strategy to learn the IVFG. We evaluate the performance of our virtual face images using different face recognizers on diffident face image datasets, all of which demonstrate the effectiveness of the IVFG for generate identifiable virtual face images.

In this paper, we show that neural networks can hide a specific task while finishing a common one. We leverage the excellent fitting ability of neural networks to train two tasks simultaneously. In a classification example, the first task is normally trained by the traditional crossentropy loss. The output of the second task can be obtained by the product of the penultimate layer and a predefined random matrix. A well-trained network looks like a benign classifier and will not arouse suspicion. For certain people, they can use a predefined random matrix to turn on the second covert function of the neural network. Experimental results demonstrate that the hidden tasks can achieve satisfactory performance without affecting the original tasks.

How to deal with the adversarial examples attracts a lot of interest recently. In this paper, we propose HF-Defend: a novel method to defend against the adversarial examples based on halftoning. Unlike the existing schemes, HF-Defend thoroughly removes the adversarial perturbations by transforming a 8-bit grayscale image (or one of the RGB channels in a color image) into a 1-bit halftoned image. To maintain the image quality and content, we propose a reconstruction module for the recovery of both the main contents and fine details of the image. In particular, we newly design a nonlinear low-pass filter to extract the main contents, and a FilterNet to establish a high-pass filter for the reconstruction of fine details. Experimental results demonstrate the advantage of our HF-Defend over the existing schemes.

Deep learning models can achieve high inference accuracy by extracting rich knowledge from massive well-annotated data, but may pose the risk of data privacy leakage in practical deployment. In this paper, we present an effective teacher-student learning approach to train privacy-preserving deep learning models via differentially private data-free distillation. The main idea is generating synthetic data to learn a student that can mimic the ability of a teacher well-trained on private data. In the approach, a generator is first pretrained in a data-free manner by incorporating the teacher as a fixed discriminator. With the generator, massive synthetic data can be generated for model training without exposing data privacy. Then, the synthetic data is fed into the teacher to generate private labels. Towards this end, we propose a label differential privacy algorithm termed selective randomized response to protect the label information. Finally, a student is trained on the synthetic data with the supervision of private labels. In this way, both data privacy and label privacy are well protected in a unified framework, leading to privacy-preserving models. Extensive experiments and analysis clearly demonstrate the effectiveness of our approach.

Videos are prone to tampering attacks that alter the meaning and deceive the audience. Previous video forgery detection schemes find tiny clues to locate the tampered areas. However, attackers can successfully evade supervision by destroying such clues using video compression or blurring. This paper proposes a video watermarking network for tampering localization. We jointly train a 3D-UNet-based watermark embedding network and a decoder that predicts the tampering mask under simulated attacks. The perturbation made by watermark embedding is close to imperceptible. Considering that there is no off-the-shelf differentiable video codec simulator, we propose to mimic video compression by ensembling simulation results of other typical attacks, e.g., JPEG compression and blurring, as an approximation. Experimental results demonstrate that our method generates watermarked videos with good imperceptibility and robustly and accurately locates tampered areas within the attacked version.

Data hiding with deep neural networks (DNNs) has experienced impressive successes in recent years. A prevailing scheme is to train an autoencoder, consisting of an encoding network to embed (or transform) secret messages in (or into) a carrier, and a decoding network to extract the hidden messages. This scheme may suffer from several limitations regarding practicability, security, and embedding capacity. In this work, we describe a different computational framework to hide images in deep probabilistic models. Specifically, we use a DNN to model the probability density of cover images, and hide a secret image in one particular location of the learned distribution. As an instantiation, we adopt a SinGAN, a pyramid of generative adversarial networks (GANs), to learn the patch distribution of one cover image. We hide the secret image by fitting a deterministic mapping from a fixed set of noise maps (generated by an embedding key) to the secret image during patch distribution learning. The stego SinGAN, behaving as the original SinGAN, is publicly communicated; only the receiver with the embedding key is able to extract the secret image. We demonstrate the feasibility of our SinGAN approach in terms of extraction accuracy and model security. Moreover, we show the flexibility of the proposed method in terms of hiding multiple images for different receivers and obfuscating the secret image.

Some robust data hiding approaches have been proposed to transmit secret data through online social networks (OSNs). Traditional steganalysis tools are inefficient in detecting these tailored steganographic methods. Although some algorithms have been developed to remove the hidden data of images, they are criticized for their low secret removal rate and poor image quality. Moreover, most of them are nongeneric methods, and multiple models must be trained to fit different algorithms. In this letter, we propose a general end-to-end data hiding break approach for OSNs, called the secret data remover (SDR). It is universal for algorithms of both robust steganography and robust watermarking, with which stego images are directly input and clean ones with the same appearances will then be generated. Moreover, we develop two novel techniques, namely, image fusion and latent renewal, to enhance the image quality and improve the overall performance. Experiments show that our proposed method achieves superior performance compared to state-of-the-art works. Hidden secret data are cleared while image quality is maintained or even slightly improved. At the same time, our work can be easily deployed in OSNs.

Video watermarking on the dual tree-complex wavelet (DT CWT) domain is shown to be effective to offer high robustness. Existing DT CWT video watermarking schemes tend to use all the coefficients on the high-pass sub-bands of the DT CWT domain for watermark embedding and detection, which lack of investigating the correlations among different sub-bands and fail to explore the stable coefficients for robust watermarking. In this paper, we propose a novel DT CWT video watermarking scheme by exploring the stable coefficients on joint sub-bands. We first extract a set of candidate coefficients by applying block singular value decomposition (SVD) on the DT CWT domain. Then, we simulate the watermark embedding by modifying the candidate coefficients on each sub-band, from which we identity two pairs of strongly correlated sub-bands termed as the joint sub-bands. The watermark is eventually embedded by modifying the candidate coefficients of the joint sub-bands on a level which is adaptively chosen according to the video resolution. During the watermark detection, we identify and extract a set of stable coefficients from the candidate coefficients of the joint sub-bands to verify the ownership of the video. Extensive experiments demonstrate the advantage of our propose scheme over the latest DT CWT based schemes, which also performs better than the existing non-DT CWT transformed domain video watermarking schemes.

While the existing reserving room before encryption (RRBE) based reversible data hiding in encrypted image (RDHEI) schemes can achieve decent embedding capacity, the capacity of the existing vacating room by encryption (VRBE) based schemes is relatively low. To address this issue, this paper proposes a generalized framework for high-capacity RDHEI for both the RRBE and VRBE cases. First, an efficient embedding room generation algorithm (ERGA) is designed to produce large embedding room using pixel prediction and entropy encoding. Then, we propose two RDHEI schemes, one for RRBE, another for VRBE. In the RRBE scenario, the image owner generates the embedding room with ERGA and encrypts the preprocessed image using stream cipher with two encryption keys. Then, the data hider locates the embedding room and embeds the additional encrypted data. In the VRBE scenario, the cover image is encrypted by an improved block modulation and permutation encryption algorithm, where the spatial redundancy in the plain-text image is greatly preserved. Then, the data hider applies ERGA on the encrypted image to generate the embedding room and conducts data embedding. For both schemes, receivers with different authentication keys can conduct either error-free data extraction or error-free image recovery. The experimental results show that the two proposed schemes outperform many state-of-the-art RDHEI schemes. Besides, they can ensure high security level, where the original image can be hardly discovered from the encrypted version before or after data hiding by unauthorized users.

In recent decades, a large number of perceptual image hashing schemes have been designed to secure the authenticity and integrity of digital images. However, the feasible criterion to evaluate the performances of hashing schemes has not been developed yet. To this end, a unified performance evaluation method for perceptual image hashing schemes is proposed in this paper. The proposed evaluation method contains six modules: robustness, discrimination, tampering detection, security, computational efficiency and hash length. The order relationship analysis (ORA) is employed to assign the score proportion of each module in accordance with the relative importance of performance, which allows the customizability of user. The real scores of modules and the outputted final score can reflect the performances of perceptual image hashing schemes intuitively and convincingly. Experimental results demonstrate that the proposed evaluation method is practical and effective for the complete and comprehensive evaluation of perceptual image hashing schemes.

Adversarial attacks slightly perturb the original image to fool deep neural networks (DNN). Various schemes have been proposed to generate uncompressed adversarial images, which are usually ineffective after being compressed during the transmission. In this paper, we propose to generate JPEG adversarial images directly from the DNN. Two adversarial rounding schemes, including fast rounding and iterative rounding, are proposed to produce quantized DCT coefficients of JPEG adversarial images. Both schemes use the gradients of adversarial images in the DCT domain to guide the rounding. In fast rounding, we propose a novel indicator to evaluate the importance of the DCT coefficients for adversarial attacks, where only those with high importance are adversarially rounded to reduce the distortion. In iterative rounding, we additionally incorporate a loss function to mea-sure the distortion caused by adversarial rounding. The experiments show that our schemes can obtain effective JPEG adversarial images with low distortion.

In this paper, we propose a novel reversible face privacy-preserving scheme. Before uploading facial images onto the cloud, we first cover the facial region with mosaic and train an encoder to generate protected images with original facial information embedded. We train another classifier with protected images for facial expression recognition and a decoder for recovering original facial images. On the cloud service, protected images provide little identity information to malicious attackers. For low-privileged users, they can use the provided classifier to do computer vision tasks with protected images. For authorized users, after content recovery, the nor-mal usage of the facial images will not be affected. Experimental results show that the proposed method is effective in facial images recovery. In addition, the protected images can maintain similar accuracy on typical computer vision tasks compared to the original images.

Recent works have demonstrated that neural networks are vulnerable to adversarial examples. Although existing methods have achieved satisfactory attack success rates, most adversarial examples can be detected by statistical analysis and further removed. In previous methods, adversarial perturbations are added using adversarial loss and distance metrics, in which the positions of modified pixels are not considered. In this paper, we elaborate a microscopical regularization that introduces adversarial perturbations onto rich texture regions. The microscopical regularization is used to evaluate pixel-level differences between a normal image and its adversarial version. We further propose a novel optimization strategy of modification probability matrices to minimize the loss function that satisfies the restriction of L∞. Through extensive experiments, we show that our method can resist statistical analysis by a large margin and achieve better visual quality than others. The proposed microscopical regularization can also be combined with existing approaches to enhance the undetectability and robustness.

Face anti-spoofing is an important step for secure face recognition. One of the main challenges is how to learn and build a general classifier that is able to resist various presentation attacks. Recently, the patch-based face anti-spoofing schemes are shown to be able to improve the robustness of the classifier. These schemes extract subtle liveness cues from small local patches independently, which do not fully exploit the correlations among the patches. In this paper, we propose a Patch-based Compact Graph Network (PCGN) to diffuse the subtle liveness cues from all the patches. Firstly, the image is encoded into a compact graph by connecting each node with its backward neighbors. We then propose an asymmetrical updating strategy to update the compact graph. Such a strategy aggregates the node based on whether it is a sender or receiver, which leads to better message-passing. The updated graph is eventually decoded for making the final decision. We conduct the experiments on four public databases with four intra-database protocols and eight cross-database protocols, the results of which demonstrate the effectiveness of our PCGN for face anti-spoofing.

We introduce Imuge, an image tamper resilient generative scheme for image self-recovery. The traditional manner of concealing image content within the image are inflexible and fragile to diverse digital attack, i.e. image cropping and JPEG compression. To address this issue, we jointly train a U-Net backboned encoder, a tamper localization network and a decoder for image recovery. Given an original image, the encoder produces a visually indistinguishable immunized image. At the recipient's side, the verifying network localizes the malicious modifications, and the original content can be approximately recovered by the decoder, despite the presence of the attacks. Several strategies are proposed to boost the training efficiency. We demonstrate that our method can recover the details of the tampered regions with a high quality despite the presence of various kinds of attacks. Comprehensive ablation studies are conducted to validate our network designs.

With the rapid development of deep learning-based techniques, the general public can use a lot of "machine learning as a service" (MLaaS), which provides end-to-end machine learning solutions. Taking the image classification task as an example, users only need to update their dataset and labels to MLaaS without requiring the specific knowledge of machine learning or a concrete structure of the classifier. Afterward, MLaaS returns a well-trained classifier to them. In this paper, we explore a potential novel task named "deep neural network retrieval" and its application which helps MLaaS to save computation resources. MLaaS usually owns a huge amount of well-trained models for various tasks and datasets. If a user requires a task that is similar to the one having been finished previously, MLaaS can quickly retrieve a model rather than training from scratch. We propose a pragmatic solution and two different approaches to extract the semantic feature of DNN representing the function of DNN, which is analogous to the usage of word2vec in natural language processing. The semantic feature of DNN can be expressed as a vector by feeding some well-designed litmus images into the DNN or as a matrix by reversely constructing the most desired input of DNN. Both methods can consider the topological information and parameters of the DNN simultaneously. Extensive experiments, including multiple datasets and networks, also demonstrate the efficiency of our method and show the high accuracy of deep neural network retrieval.

Abstract Animated emoji is a kind of GIF image, which is widely used in online social networks (OSN) for its efficiency in transmitting vivid and personalized information. Aiming at realizing covert communication in animated emoji, this paper proposes an improved steganography framework in animated emoji. We propose a self-reference algorithm to improve the steganography security. Meanwhile, the relations between adjacent frames of the cover GIF image are considered to further improve the distortion function. After that we embed the secret message into the GIF image using the popular framework of Syndrome Trellis Coding (STC). Experimental results show that the proposed method can provide better security performances than state-of-the-art works.

In this paper, we propose a privacy protection scheme using image dual-inpainting and data hiding. In the proposed scheme, the privacy contents in the original image are concealed, which are reversible that the privacy content can be perfectly recovered. We use an interactive approach to select the areas to be protected, that is, the protection data. To address the disadvantage that single image inpainting is susceptible to forensic localization, we propose a dual-inpainting algorithm to implement the object removal task. The protection data is embedded into the image with object removed using a popular data hiding method. We further use the pattern noise forensic detection and the objective metrics to assess the proposed method. The results on different scenarios show that the proposed scheme can achieve better visual quality and antiforensic capability than the state-of-the-art works.

Outsourcing computation allows an outsourcer with limited resource to delegate the computation load to a powerful server without the exposure of true inputs and outputs. It is well known that modular exponentiation is one of the most expensive operations in public key cryptosystems. Currently, most of outsourcing algorithms for modular exponentiation are based on two untrusted servers or have small checkability with single server. In this paper, we first propose an efficient outsourcing algorithm of modular exponentiation based on two untrusted servers, where the outsourcer can detect the error based on Euler theorem with a probability of 1 if one of the servers misbehaves. We then present an outsourcing algorithm of modular exponentiation with single server, and the outsourcer can also check the failure with a probability of 1. Therefore, the proposed algorithm with single server improves efficiency and checkability simultaneously compare with the previous ones. Finally, we provide the experimental evaluations to demonstrate that the proposed two algorithms are the most efficient ones in all of the outsourcing algorithms for an outsourcer.

This paper proposes a new framework of lossless data hiding (LDH) in JPEG images. The proposed framework contains two algorithms, i.e., the optimized basic LDH and the relay transfer based extension. In the basic algorithm, we aim to preserve the filesize after data embedding. The data hiding process is optimized by variable-length-code (VLC) mapping, combination and permutation. In the extended algorithm, we focus on embedding more bits into the bitstream with a condition that the filesize increment is allowed. To decrease the filesize increment, we propose a relay transfer based algorithm to preprocess the JPEG bitstream. Subsequently, we embed data into the processed bitstream using the basic LDH. Both algorithms provide better performances than previous arts. After lossless data hiding, the marked JPEG bitstream is compliant to common JPEG decoders. Since all operations are implemented on VLCs and the Huffman codes, no distortion is generated on the image pixels. Experimental results demonstrate that the proposed approach outperforms previous methods.

Batch steganography is a technique that hides information into multiple covers. To achieve a better performance on the security of data hiding, we propose a novel strategy of batch steganography using a generative network. In this method, the approaches of cover selection, payload allocation, and distortion evaluation are considered in the round. We define a quality metric to evaluate the distortion between the cover image and the stego. When training the generation function, we define an objective function containing two parts: the entropy loss and the steganalytic loss. While the entropy loss is used to represent the gap between the payload inside stego images and the entire embedding capacity, the steganalytic loss is used to assess the data embedding impact using the proposed quality metric. With back-propagation, we minimize the objective function to obtain an optimal solution. Accordingly, different payloads can be allocated to different images, and the ± 1 modification probability for pixels in each cover can be calculated. Finally, we embed information into the selected images by STC. Experimental results show that the proposed method achieves a better undetectability against modern steganalytic tools.

In current steganalysis, relying on a large scale of samples, widely-adopted supervised schemes require the training stage while few studies focus on the design of a training-free unsupervised adaptive detector with high efficiency. To fill the gap, we investigate an adaptive statistical model-based detector designed for detecting JPEG steganography. First, in virtue of hypothesis testing theory, together with the distribution of quantized DCT coefficients, we establish the general framework of the statistical model-based detector. Second, based on the framework, we mainly analyze the performance of the detector relying on the selection of the statistical model, parameters estimation, and less significant payload prediction. Third, to improve the reliability of detection, based on the strategy of assigning weights for DCT channels, the novel adaptive statistical model-based detectors are proposed to aim at detecting JPEG steganography, involving the channel-selected or non-channel-selected algorithm. Extensive experiments highlight the effectiveness of the proposed methodology. Moreover, when detecting JPEG images adopted by two steganographic schemes with the small payload, the experimental results show the Area Under Curve (AUC) of our proposed optimal adaptive detector can achieve as high as 0.9567 and 0.9895 respectively, which are both better than that of non-adaptive detector.

This letter focus on a practical scenario of cover selection for steganography, in which a part of the available images of the sender have been processed to improve visual quality, e.g., contrast enhancement, image denoising. In this case, not only the embedding distortion (caused by steganography), but also the processing distortion (caused by processing) should be considered when selecting cover image. We propose a cover selection method to combine the two kinds of distortion together to measure the suitability for steganography of the available images. To calculate the processing distortion, a classifier is trained to distinguish the processed images with the original ones. The classifier is then used to measure the possibility of the existence of processing. High possibility means high processing distortion. In addition, the current distortion minimization framework designed for steganography is employed to calculate the embedding distortion. Finally, both kinds of distortion are combined to form the total distortion, and the image with the minimal total distortion is selected as cover. Using the selected image for embedding, high undetectability can be achieved.

High-dimensional feature extraction based on co-occurrence matrix improves the detection performance of steganalysis, but it is difficult to be realized for massive image data by an analyzer with limited computational ability. We solve this problem by verifiable outsourcing computation, which allows a computationally weak client to outsource the evaluation of a function to a powerful but untrusted server. In this paper, we propose a verifiable outsourcing scheme of feature extraction based on co-occurrence matrix with single untrusted cloud server. The original images are protected from the server by using a projection of one to many with trapdoor, which can be realized by a symmetric probabilistic encryption scheme we present. The analyzer can obtain true results of feature extraction and detect any failure with a probability of 1 if the server misbehaves. Finally, we provide the simulations on the outsourcing of extracting ccJRM features in cloud computing. The theory analysis and experiment result also show that the proposed outsourcing scheme could greatly decrease the computation cost of the analyzer without exposure of the original images and extraction results.

Steganalysis is a technique for detecting the existence of secret information hidden in digital media. In this paper, we propose a novel scheme for JPEG steganalysis. In this scheme, we first design the diverse base filters which are able to obtain the image residuals from various directions. Then, we propose a cascade filter generation strategy to construct a set of high order cascade filters from the base filters. We further select the cascade filters with the maximum diversity. The selected filters are convolved with the decompressed JPEG image to obtain residuals which capture the subtle embedding traces. The residuals, termed as the maximum diversity cascade filter residual, are eventually used to extract features to train an ensemble classifier for classification. The experiments are carried out on the detection of stego-images generated using common JPEG steganographic schemes, the results of which demonstrate the effectiveness of the proposed scheme for JPEG steganalysis.

With the widespread use of biometrics recognition systems, it is of paramount importance to protect the privacy of biometrics. In this paper, we propose to protect the fingerprint privacy by the artificial fingerprint, which is generated based on three pieces of information, i) the original minutiae positions; ii) the artificial fingerprint orientation; and iii) the artificial minutiae polarities. To make it real-look alike and diverse, we propose to generate the artificial fingerprint orientation by a model taking both the global and local fingerprint orientation into account. Its parameters can be easily guided by an user specific key with simple constraints. The artificial minutiae polarities are generated from the same key, where a block based and a function based approach are proposed for the minutiae polarities generation. These information are properly integrated to form a real-look alike artificial fingerprint. It is difficult for the attacker to distinguish such a fingerprint from the real fingerprints. If it is stolen, the complete fingerprint minutiae feature will not be compromised, and we can generate a different artificial fingerprint using another key. Experimental results show that the artificial fingerprint can be recognized accurately.

This paper proposes a new steganography method for hiding data into dynamic GIF (Graphics Interchange Format) images. When using the STC framework, we propose a new algorithm of cost assignment according to the characteristics of dynamic GIF images, including the image palette and the correlation of interframes. We also propose a payload allocation algorithm for different frames. First, we reorder the palette of GIF images to reduce the modifications on pixel values when modifying the index values. As the different modifications on index values would result in different impacts on pixel values, we assign the elements with less impact on pixel values with small embedding costs. Meanwhile, small embedding costs are also assigned for the elements in the regions that the interframe changes are large enough. Finally, we calculate an appropriate payload for each frame using the embedding probability obtained from the proposed distortion function. Experimental results show that the proposed method has a better security performance than state-of-the-art works.

This paper proposes a novel method of reversible data hiding in encrypted images (RDH-EI). We first provide an RDH-EI approach using single-level embedding, in which three parties are involved, including an image owner, a data hider, and a recipient. The image owner encrypts an original image into a ciphertext image. After dividing the original image into blocks, the owner pseudo-randomly permutes all blocks by a permutation key. With an encryption key, the image owner further encrypts the contents of all blocks using a stream cipher algorithm, during which pixels inside each block share the same stream bytes. Once the encrypted image is uploaded onto the server, a data hider embeds additional messages into the ciphertext. The data hider divides the encrypted image into blocks and selects peak pixels from each block using an embedding key. With the peak pixels, the data hider embeds an additional message using histogram shifting inside each block. On the receiver side, a recipient extracts the hidden message using the embedding key, and losslessly recover the original image with the permutation key and the encryption key. Based on the single-level algorithm, we further construct a multi-level approach. The embedding process is iteratively used to generate the marked encrypted images. Compared with state-of-the-art works, the proposed method achieves a better embedding efficiency and an error-free recovery.

This paper proposes a novel framework of reversible data hiding in encrypted JPEG bitstream. We first provide a JPEG encryption algorithm to encipher a JPEG image to a smaller size and keep the format compliant to JPEG decoders. After an image owner uploads the encrypted JPEG bitstreams to cloud storage, the server embeds additional messages into the ciphertext to construct a marked encrypted JPEG bitstream. During data hiding, we propose a combined embedding algorithm including two stages, the Huffman code mapping and the ordered histogram shifting. The embedding procedure is reversible. When an authorized user requires a downloading operation, the server extracts additional messages from the marked encrypted JPEG bitstream and recovers the original encrypted bitstream losslessly. After downloading, the user obtains the original JPEG bitstream by a direct decryption. The proposed framework outperforms previous works on reversible data hiding in encrypted images. First, since the tasks of data embedding/extraction and bitstream recovery are all accomplished by the server, the image owner and the authorized user are required to implement no extra operations except JPEG encryption or decryption. Second, the embedding payload is larger than state-of-the-art works.

International audience

While most techniques of reversible data hiding in encrypted images (RDH-EI) are developed for uncompressed images, this paper provides a separable reversible data hiding protocol for encrypted JPEG bitstreams. We first propose a JPEG encryption algorithm, which enciphers an image to a smaller size and keeps the format compliant to JPEG decoder. After a content owner uploads the encrypted JPEG bitstream to a remote server, a data hider embeds an additional message into the encrypted copy without changing the bitstream size. On the recipient side, the original bitstream can be reconstructed losslessly using an iterative recovery algorithm based on the blocking artifact. Since message extraction and image recovery are separable, anyone who has the embedding key can extract the message from the marked encrypted copy. Experimental results show that the proposed method outperforms a previous work in terms of separation capability, embedding capacity and security.

This paper proposes a method of reversible data hiding in encrypted images (RDH-EI) based on progressive recovery. Three parties are involved in the framework, including the content owner, the data-hider, and the recipient. The content owner encrypts the original image using a stream cipher algorithm and uploads a ciphertext to the server. The data-hider on the server divides the encrypted image into three channels and, respectively, embeds different amount of additional bits into each one to generate a marked encrypted image. On the recipient side, additional message can be extracted from the marked encrypted image, and the original image can be recovered without any errors. While most of the traditional methods use one criterion to recover the whole image, we propose to do the recovery by a progressive mechanism. Rate-distortion of the proposed method outperforms state-of-the-art RDH-EI methods.

This letter proposes a novel reversible data hiding (RDH) method with an adaptive embedding capability by extending the generalized integer transformation (GIT). We modify the GIT algorithm to a further generalized form, and accordingly we propose an adaptive embedding algorithm. When hiding data into the original image, parameters of the extended GIT can be identified according to the content of each block. Along with a multi-level location map, a better embedding efficiency of the net payload can be achieved. On the decoding side, the original image can be losslessly recovered after extracting the hidden message. Experimental results show that the proposed method outperforms state-of-the-art integer-transformation based RDH methods.

This paper proposes a novel scheme of reversible data hiding in encrypted images using distributed source coding. After the original image is encrypted by the content owner using a stream cipher, the data-hider compresses a series of selected bits taken from the encrypted image to make room for the secret data. The selected bit series is Slepian-Wolf encoded using low-density parity check codes. On the receiver side, the secret bits can be extracted if the image receiver has the embedding key only. In case the receiver has the encryption key only, he/she can recover the original image approximately with high quality using an image estimation algorithm. If the receiver has both the embedding and encryption keys, he/she can extract the secret data and perfectly recover the original image using the distributed source decoding. The proposed method outperforms the previously published ones.

Extreme learning machine (ELM), proposed by Huang et al., was developed for generalized single hidden layer feedforward networks with a wide variety of hidden nodes. ELMs have been proved very fast and effective especially for solving function approximation problems with a predetermined network structure. However, it may contain insignificant hidden nodes. In this paper, we propose dynamic adjustment ELM (DA-ELM) that can further tune the input parameters of insignificant hidden nodes in order to reduce the residual error. It is proved in this paper that the energy error can be effectively reduced by applying recursive expectation-minimization theorem. In DA-ELM, the input parameters of insignificant hidden node are updated in the decreasing direction of the energy error in each step. The detailed theoretical foundation of DA-ELM is presented in this paper. Experimental results show that the proposed DA-ELM is more efficient than the state-of-art algorithms such as Bayesian ELM, optimally-pruned ELM, two-stage ELM, Levenberg-Marquardt, sensitivity-based linear learning method as well as the preliminary ELM.

This correspondence proposes a framework of reversible data hiding (RDH) in an encrypted JPEG bitstream. Unlike existing RDH methods for encrypted spatial-domain images, the proposed method aims at encrypting a JPEG bitstream into a properly organized structure, and embedding a secret message into the encrypted bitstream by slightly modifying the JPEG stream. We identify usable bits suitable for data hiding so that the encrypted bitstream carrying secret data can be correctly decoded. The secret message bits are encoded with error correction codes to achieve a perfect data extraction and image recovery. The encryption and embedding are controlled by encryption and embedding keys respectively. If a receiver has both keys, the secret bits can be extracted by analyzing the blocking artifacts of the neighboring blocks, and the original bitstream perfectly recovered. In case the receiver only has the encryption key, he/she can still decode the bitstream to obtain the image with good quality without extracting the hidden data.

This paper proposes a novel scheme of compressing encrypted images with auxiliary information. The content owner encrypts the original uncompressed images and also generates some auxiliary information, which will be used for data compression and image reconstruction. Then, the channel provider who cannot access the original content may compress the encrypted data by a quantization method with optimal parameters that are derived from a part of auxiliary information and a compression ratio-distortion criteria, and transmit the compressed data, which include an encrypted sub-image, the quantized data, the quantization parameters and another part of auxiliary information. At receiver side, the principal image content can be reconstructed using the compressed encrypted data and the secret key. Experimental result shows the ratio-distortion performance of the proposed scheme is better than that of previous techniques.

This paper proposes a novel scheme of scalable coding for encrypted images. In the encryption phase, the original pixel values are masked by a modulo-256 addition with pseudorandom numbers that are derived from a secret key. After decomposing the encrypted data into a downsampled subimage and several data sets with a multiple-resolution construction, an encoder quantizes the subimage and the Hadamard coefficients of each data set to reduce the data amount. Then, the data of quantized subimage and coefficients are regarded as a set of bitstreams. At the receiver side, while a subimage is decrypted to provide the rough information of the original content, the quantized coefficients can be used to reconstruct the detailed content with an iteratively updating procedure. Because of the hierarchical coding mechanism, the principal original content with higher resolution can be reconstructed when more bitstreams are received.

This paper proposes a novel watermarking scheme with flexible self-recovery quality. The embedded watermark data for content recovery are calculated from the original discrete cosine transform (DCT) coefficients of host image and do not contain any additional redundancy. When a part of a watermarked image is tampered, the watermark data in the area without any modification still can be extracted. If the amount of extracted data is large, we can reconstruct the original coefficients in the tampered area according to the constraints given by the extracted data. Otherwise, we may employ a compressive sensing technique to retrieve the coefficients by exploiting the sparseness in the DCT domain. This way, all the extracted watermark data contribute to the content recovery. The smaller the tampered area, the more available watermark data will result in a better quality of recovered content. It is also shown that the proposed scheme outperforms previous techniques in general.

This paper proposes two novel self-embedding watermarking schemes based upon a reference sharing mechanism, in which the watermark to be embedded is a reference derived from the original principal content in different regions and shared by these regions for content restoration. After identifying tampered blocks, both the reference data and the original content in the reserved area are used to recover the principal content in the tampered area. By using the first scheme, the original data in five most significant bit layers of a cover image can be recovered and the original watermarked image can also be retrieved when the content replacement is not too extensive. In the second scheme, the host content is decomposed into three levels, and the reference sharing methods with different restoration capabilities are employed to protect the data at different levels. Therefore, the lower the tampering rate, the more levels of content data are recovered, and the better the quality of restored results.

When hiding information in an image for self recovery, the amount of embedding data affects the embedding influence and the recovery quality. The purpose of this paper is to reduce the amount of embedding data while maintaining good recovery quality. We propose an approach to generate reference data from the original image by encoding different types of blocks into different number of bits. In reconstructing the reference image, a fast inpainting method is used to recover the contents of corrupted regions accompanied with the extracted bits.