We present a novel approach for indoor scene synthesis, which learns to arrange decomposed cuboid primitives to represent 3D objects within a scene. Unlike conventional methods that use bounding boxes to determine the placement and scale of 3D objects, our approach leverages cuboids as a straightforward yet highly effective alternative for modeling objects. This allows for compact scene generation while minimizing object intersections. Our approach, coined CasaGPT for Cuboid Arrangement and Scene Assembly, employs an autoregressive model to sequentially arrange cuboids, producing physically plausible scenes. By applying rejection sampling during the fine-tuning stage to filter out scenes with object collisions, our model further reduces intersections and enhances scene quality. Additionally, we introduce a refined dataset, 3DFRONT-NC, which eliminates significant noise presented in the original dataset, 3D-FRONT. Extensive experiments on the 3D-FRONT dataset as well as our dataset demonstrate that our approach consistently outperforms the state-of-the-art methods, enhancing the realism of generated scenes, and providing a promising direction for 3D scene synthesis. Code is available at https://github.com/CASAGPT/CASA-GPT

In this paper, we tackle the copy-paste image-to-image composition problem with a focus on object placement learning. Prior methods have leveraged generative models to reduce the reliance for dense supervision. However, this often limits their capacity to model complex data distributions. Alternatively, transformer networks with a sparse contrastive loss have been explored, but their over-relaxed regularization often leads to imprecise object placement. We introduce BootPlace, a novel paradigm that formulates object placement as a placement-by-detection problem. Our approach begins by identifying suitable regions of interest for object placement. This is achieved by training a specialized detection transformer on object-subtracted backgrounds, enhanced with multi-object supervisions. It then semantically associates each target compositing object with detected regions based on their complementary characteristics. Through a boostrapped training approach applied to randomly object-subtracted images, our model enforces meaningful placements through extensive paired data augmentation. Experimental results on established benchmarks demonstrate BootPlace’s superior performance in object repositioning, markedly surpassing state-of-the-art baselines on Cityscapes and OPA datasets with notable improvements in IOU scores. Additional ablation studies further showcase the compositionality and generalizability of our approach, supported by user study evaluations. Code is available at https://github.com/RyanHangZhou/BootPlace

With the rise of social media and the proliferation of facial recognition surveillance, concerns surrounding privacy have escalated significantly. While numerous studies have concentrated on safeguarding users against unauthorized face recognition, a new and often overlooked issue has emerged due to advances in facial restoration techniques: traditional methods of facial obfuscation may no longer provide a secure shield, as they can potentially expose anonymous information to human perception. Our empirical study shows that blind face restoration (BFR) models can restore obfuscated faces with high probability by simply retraining them on obfuscated (e.g., pixelated) faces. To address it, we propose a transferable adversarial obfuscation method for privacy protection against BFR models. Specifically, we observed a common characteristic among BFR models, namely, their capability to approximate an inverse mapping of a transformation from a high-quality image domain to a low-quality image domain. Leveraging this shared model attribute, we have developed a domain-consistent adversarial method for generating obfuscated images. In essence, our method is designed to minimize overfitting to surrogate models during the perturbation generation process, thereby enhancing the generalization of adversarial obfuscated facial images. Extensive experiments on various BFR models demonstrate the effectiveness and transferability of the proposed method.

Abstract In this paper, we address the problem of plausible object placement for the challenging task of realistic image composition. We propose DiffPop, the first framework that utilizes plausibility‐guided denoising diffusion probabilistic model to learn the scale and spatial relations among multiple objects and the corresponding scene image. First, we train an unguided diffusion model to directly learn the object placement parameters in a self‐supervised manner. Then, we develop a human‐in‐the‐loop pipeline which exploits human labeling on the diffusion‐generated composite images to provide the weak supervision for training a structural plausibility classifier. The classifier is further used to guide the diffusion sampling process towards generating the plausible object placement. Experimental results verify the superiority of our method for producing plausible and diverse composite images on the new Cityscapes‐OP dataset and the public OPA dataset, as well as demonstrate its potential in applications such as data augmentation and multi‐object placement tasks. Our dataset and code will be released.

Notwithstanding the prominent performance shown in various applications, point cloud recognition models have often suffered from natural corruptions and adversarial perturbations. In this paper, we delve into boosting the general robustness of point cloud recognition, proposing Point-Cloud Contrastive Adversarial Training (PointCAT). The main intuition of PointCAT is encouraging the target recognition model to narrow the decision gap between clean point clouds and corrupted point clouds by devising feature-level constraints rather than logit-level constraints. Specifically, we leverage a supervised contrastive loss to facilitate the alignment and the uniformity of hypersphere representations, and design a pair of centralizing losses with dynamic prototype guidance to prevent features from deviating outside their belonging category clusters. To generate more challenging corrupted point clouds, we adversarially train a noise generator concurrently with the recognition model from the scratch. This differs from previous adversarial training methods that utilized gradient-based attacks as the inner loop. Comprehensive experiments show that the proposed PointCAT outperforms the baseline methods, significantly enhancing the robustness of diverse point cloud recognition models under various corruptions, including isotropic point noises, the LiDAR simulated noises, random point dropping, and adversarial perturbations. Our code is available at: https://github.com/shikiw/PointCAT.

We introduce an end-to-end learning framework for image-to-image composition, aiming to plausibly compose an object represented as a cropped patch from an object image into a background scene image. As our approach emphasizes more on semantic and structural coherence of the composed images, rather than their pixel-level RGB accuracies, we tailor the input and output of our network with structure-aware features and design our network losses accordingly, with ground truth established in a self-supervised setting through the object cropping. Specifically, our network takes the semantic layout features from the input scene image, features encoded from the edges and silhouette in the input object patch, as well as a latent code as inputs, and generates a 2D spatial affine transform defining the translation and scaling of the object patch. The learned parameters are further fed into a differentiable spatial transformer network to transform the object patch into the target image, where our model is trained adversarially using an affine transform discriminator and a layout discriminator. We evaluate our network, coined SAC-GAN, for various image composition scenarios in terms of quality, composability, and generalizability of the composite images. Comparisons are made to state-of-the-art alternatives, including Instance Insertion, ST-GAN, CompGAN and PlaceNet, confirming superiority of our method.

In the past few years, the widespread use of 3D printing technology enables the growth of the market of 3D printed products. On Esty, a website focused on handmade items, hundreds of individual entrepreneurs are selling their 3D printed products. Inspired by the positive effects of machine-readable tags, like barcodes, on daily product marketing, we propose AnisoTag, a novel tagging method to encode data on the 2D surface of 3D printed objects based on reflection anisotropy. AnisoTag has an unobtrusive appearance and much lower extraction computational complexity, contributing to a lightweight low-cost tagging system for individual entrepreneurs. On AnisoTag, data are encoded by the proposed tool as reflective anisotropic microstructures, which would reflect distinct illumination patterns when irradiating by collimated laser. Based on it, we implement a real-time detection prototype with inexpensive hardware to determine the reflected illumination pattern and decode data according to their mapping. We evaluate AnisoTag with various 3D printer brands, filaments, and printing parameters, demonstrating its superior usability, accessibility, and reliability for practical usage.

Existing image cropping detection schemes ignore that recovering the cropped-out contents can unveil the purpose of the behaved cropping attack. This paper presents CLR-Net, a novel image protection scheme addressing the combined challenge of image Cropping Localization and Recovery. We first protect the original image by introducing imperceptible perturbations. Then, typical image post-processing attacks are simulated to erode the protected image. On the recipient’s side, we predict the cropping mask and recover the original image. Besides, we propose a novel Fine-Grained generative JPEG simulator (FG-JPEG) as well as a feature alignment network to improve the real-world robustness. Comprehensive experiments prove that the quality of the recovered image and the accuracy of crop localization are both satisfactory.

Deep 3D point cloud models are sensitive to adversarial attacks, which poses threats to safety-critical applications such as autonomous driving. Robust training and defend-by-denoising are typical strategies for defending adversarial perturbations. However, they either induce massive computational overhead or rely heavily upon specified priors, limiting generalized robustness against attacks of all kinds. To remedy it, this paper introduces a novel distortion-aware defense framework that can rebuild the pristine data distribution with a tailored intensity estimator and a diffusion model. To perform distortion-aware forward diffusion, we design a distortion estimation algorithm that is obtained by summing the distance of each point to the best-fitting plane of its local neighboring points, which is based on the observation of the local spatial properties of the adversarial point cloud. By iterative diffusion and reverse denoising, the perturbed point cloud under various distortions can be restored back to a clean distribution. This approach enables effective defense against adaptive attacks with varying noise budgets, enhancing the robustness of existing 3D deep recognition models.

We present ShaDDR, an example-based deep generative neural network which produces a high-resolution textured 3D shape through geometry detailization and conditional texture generation applied to an input coarse voxel shape. Trained on a small set of detailed and textured exemplar shapes, our method learns to detailize the geometry via multi-resolution voxel upsampling and generate textures on voxel surfaces via differentiable rendering against exemplar texture images from a few views. The generation is interactive, taking less than 1 second to produce a 3D model with voxel resolutions up to 5123. The generated shape preserves the overall structure of the input coarse voxel model, while the style of the generated geometric details and textures can be manipulated through learned latent codes. In the experiments, we show that our method can generate higher-resolution shapes with plausible and improved geometric details and clean textures compared to prior works. Furthermore, we showcase the ability of our method to learn geometric details and textures from shapes reconstructed from real-world photos. In addition, we have developed an interactive modeling application to demonstrate the generalizability of our method to various user inputs and the controllability it offers, allowing users to interactively sculpt a coarse voxel shape to define the overall structure of the detailized 3D shape. Code and data are available at https://github.com/qiminchen/ShaDDR.

Digital images are vulnerable to nefarious tampering attacks such as content addition or removal that severely alter the original meaning. It is somehow like a person without protection that is open to various kinds of viruses. Image immunization (Imuge) is a technology of protecting the images by introducing trivial perturbation, so that the protected images are immune to the viruses in that the tampered contents can be auto-recovered. This paper presents Imuge+, an enhanced scheme for image immunization. By observing the invertible relationship between image immunization and the corresponding self-recovery, we employ an invertible neural network to jointly learn image immunization and recovery respectively in the forward and backward pass. We also introduce an efficient attack layer that involves both malicious tamper and benign image post-processing, where a novel distillation-based JPEG simulator is proposed for improved JPEG robustness. Our method achieves promising results in real-world tests where experiments show accurate tamper localization as well as high-fidelity content recovery. Additionally, we show superior performance on tamper localization compared to state-of-the-art schemes based on passive forensics.

Whereas cryptography easily arouses attacks by means of encrypting a secret message into a suspicious form, steganography is advantageous for its resilience to attacks by concealing the message in an innocent-looking cover signal. Minimal distortion steganography, one of the mainstream steganography frameworks, embeds messages while minimizing the distortion caused by the modification on the cover elements. Due to the unavailability of the original cover signal for the receiver, message embedding is realized by finding the coset leader of the syndrome function of steganographic codes migrated from channel coding, which is complex and has limited performance. Fortunately, deep generative models and the robust semantic of generated data make it possible for the receiver to perfectly reproduce the cover signal from the stego signal. With this advantage, we propose cover-reproducible steganography where the source coding, e.g., arithmetic coding, serves as the steganographic code. Specifically, the decoding process of arithmetic coding is used for message embedding and its encoding process is regarded as message extraction. Taking text-to-speech and text-to-image synthesis tasks as two examples, we illustrate the feasibility of cover-reproducible steganography. Steganalysis experiments and theoretical analysis are conducted to demonstrate that the proposed methods outperform the existing methods in most cases.

Deep-learning based watermarking framework has been extensively studied recently. The main structure of such framework is an encoder, a noise layer and a decoder. By training with different distortion sets in the noise layer, the whole network can realize different robustness. However, such framework has a huge drawback that the noise layer must be differentiable, otherwise it cannot be trained end-to-end. But for practical use, much distortions are non-differentiable, so such framework cannot be applied. To address such limitations, this paper propose a triple-phase watermarking framework for practical distortions. The proposed framework consists of three phases including a noise-free initial phase, a mask-guided frequency enhancement phase and an adversarial-training phase. Phase 1 aims to initialize an encoder to embed watermark with high visual quality and a decoder to extract the watermark. In order to generate high quality watermarked image, we design the just noticeable difference (JND)-mask image loss in phase 1 to guide the encoder. At phase 2, based on the investigation of the encoded features and distortions, we propose a mask-guided frequency enhancement algorithm to enhance the encoded feature which ensures the survival of such features after distortion, so that there will be enough features to be learned in phase 3. And phase 3 aims to train a stronger decoder to extract the watermark from the image after practical distortions. The combination of these 3 phases can well handle the non-differentiable problems and make the whole network trainable. Various experiments indicate the superior performance of the proposed scheme in the view of traditional differentiable image processing distortion robustness and practical non-differentiable distortion robustness.

In recent years, many model intellectual property (IP) proof methods for IP protection have been proposed, such as model watermarking and model fingerprinting. However, with the increasing number of models transmitted and deployed on the Internet, quickly finding the suspect model among thousands of models on model-sharing platforms such as GitHub is in great demand, which concurrently triggers the new security problem of model copy detection for IP protection. As an important part of the model IP protection system, the model copy detection task has not received enough attention. Due to the high computational complexity, both model watermarking and model fingerprinting lack the capability to efficiently find suspected infringing models among tens of millions of models. In this article, inspired by the hash-based image retrieval methods, we introduce a novel model copy detection mechanism: perceptual hashing for convolutional neural networks (CNNs). The proposed perceptual hashing algorithm can convert the weights of CNNs to fixed-length binary hash codes so that the lightly modified version has the similar hash code as the original model. By comparing the similarity of a pair of hash codes between a query model and a test model in the model library, similar versions of a query model can be retrieved efficiently. To the best of our knowledge, this is the first perceptual hashing algorithm for deep neural network models. Specifically, we first select the important model weights based on the model compression theory, then calculate the normal test statistics (NTS) on the segments of important weights, and finally encode the NTS features into hash codes. The experiment performed on a model library containing 3,565 models indicates that our perceptual hashing scheme has a superior copy detection performance.

Adversary and invisibility are two fundamental but conflict characters of adversarial perturbations. Previous adversarial attacks on 3D point cloud recognition have often been criticized for their noticeable point outliers, since they just involve an “implicit constrain” like global distance loss in the time-consuming optimization to limit the generated noise. While point cloud is a highly structured data format, it is hard to constrain its perturbation with a simple loss or metric properly. In this paper, we propose a novel Point-Cloud Sensitivity Map to boost both the efficiency and imperceptibility of point perturbations. This map reveals the vulnerability of point cloud recognition models when encountering shape-invariant adversarial noises. These noises are designed along the shape surface with an “explicit constrain” instead of extra distance loss. Specifically, we first apply a reversible coordinate transformation on each point of the point cloud input, to reduce one degree of point freedom and limit its movement on the tangent plane. Then we calculate the best attacking direction with the gradients of the transformed point cloud obtained on the white-box model. Finally we assign each point with a non-negative score to construct the sensitivity map, which benefits both white-box adversarial invisibility and black-box query-efficiency extended in our work. Extensive evaluations prove that our method can achieve the superior performance on various point cloud recognition models, with its satisfying adversarial imperceptibility and strong resistance to different point cloud defense settings. Our code is available at: https://github.com/shikiw/SI-Adv.

Recently almost all the mainstream deepfake detection methods use Convolutional Neural Networks (CNN) as their backbone. However, due to the overreliance on local texture information which is usually determined by forgery methods of training data, these CNN-based methods cannot generalize well to unseen data. To get out of the predicament of prior methods, in this paper, we propose a novel transformer-based framework to model both global and local information and analyze anomalies of face images. In particular, we design attention leading module, multi-forensics module and variant residual connections for deepfake detection, and leverage token-level contrast loss for more detailed supervision. Experiments on almost all popular public deepfake datasets demonstrate that our method achieves state-of-the-art performance in cross-dataset evaluation and comparable performance in intra-dataset evaluation.

The existing image embedding networks are basically vulnerable to malicious attacks such as JPEG compression and noise adding, not applicable for real-world copyright protection tasks. To solve this problem, we introduce a generative deep network based method for hiding images into images while assuring high-quality extraction from the destructive synthesized images. An embedding network is sequentially concatenated with an attack layer, a decoupling network and an image extraction network. The addition of decoupling network learns to extract the embedded secret image from the attacked image. We also pinpoint the weaknesses of the adversarial training for robustness in previous works and build our improved real-world attack simulator. Experimental results demonstrate the superiority of the proposed method against typical digital attacks by a large margin, as well as the performance boost of the recovered images with the aid of progressive recovery strategy. Besides, we are the first to robustly hide three secret images.

Abstract Robust 3D mesh watermarking is a traditional research topic in computer graphics, which provides an efficient solution to the copyright protection for 3D meshes. Traditionally, researchers need manually design watermarking algorithms to achieve sufficient robustness for the actual application scenarios. In this paper, we propose the first deep learning-based 3D mesh watermarking network, which can provide a more general framework for this problem. In detail, we propose an end-to-end network, consisting of a watermark embedding sub-network, a watermark extracting sub-network and attack layers. We employ the topology-agnostic graph convolutional network (GCN) as the basic convolution operation, therefore our network is not limited by registered meshes (which share a fixed topology). For the specific application scenario, we can integrate the corresponding attack layers to guarantee adaptive robustness against possible attacks. To ensure the visual quality of watermarked 3D meshes, we design the curvature consistency loss function to constrain the local geometry smoothness of watermarked meshes. Experimental results show that the proposed method can achieve more universal robustness while guaranteeing comparable visual quality.

Steganography is the art and science of hiding secret messages in public communication so that the presence of secret messages cannot be detected. There are two distribution-preserving steganographic frameworks, one is sampler-based and the other is compression-based. The former requires a perfect sampler which yields data following the same distribution, and the latter needs the explicit distribution of generative objects. However, these two conditions are too strict even unrealistic in the traditional data environment, e.g., the distribution of natural images is hard to seize. Fortunately, generative models bring new vitality to distribution-preserving steganography, which can serve as the perfect sampler or provide the explicit distribution of generative media. Taking text-to-speech generation task as an example, we propose distribution-preserving steganography based on WaveGlow and WaveRNN, which corresponds to the former two categories. Steganalysis experiments and theoretical analysis are conducted to demonstrate that the proposed methods can preserve the distribution.

Invertible grayscale is a special kind of grayscale from which the original color can be recovered. Given an input color image, this seminal work tries to hide the color information into its grayscale counterpart while making it hard to recognize any anomalies. This powerful functionality is enabled by training a hiding sub-network and restoring sub-network in an end-to-end way. Despite its expressive results, two key limitations exist: 1) The restored color image often suffers from some noticeable visual artifacts in the smooth regions. 2) It is very sensitive to JPEG compression, i.e., the original color information cannot be well recovered once the intermediate grayscale image is compressed by JPEG. To overcome these two limitations, this article introduces adversarial training and JPEG simulator respectively. Specifically, two auxiliary adversarial networks are incorporated to make the intermediate grayscale images and final restored color images indistinguishable from normal grayscale and color images. And the JPEG simulator is utilized to simulate real JPEG compression during the online training so that the hiding and restoring sub-networks can automatically learn to be JPEG robust. Extensive experiments demonstrate that the proposed method is superior to the original invertible grayscale work both qualitatively and quantitatively while ensuring the JPEG robustness. We further show that the proposed framework can be applied under different types of grayscale constraints and achieve excellent results.

Three-dimensional (3-D) meshes are commonly used to represent virtual surfaces and volumes. Over the past decade, 3-D meshes have emerged in industrial, medical, and entertainment applications, being of large practical significance for 3-D mesh steganography and steganalysis. In this article, we provide a systematic survey of the literature on 3-D mesh steganography and steganalysis. Compared with an earlier survey (Girdhar et al., 2017), we propose a new taxonomy of steganographic algorithms with four categories: 1) two-state domain, 2) LSB domain, 3) permutation domain, and 4) transform domain. Regarding steganalysis algorithms, we divide them into two categories: 1) universal steganalysis and 2) specific steganalysis. For each category, the history of technical developments and the current technological level are introduced and discussed. Finally, we highlight some promising future research directions and challenges in improving the performance of 3-D mesh steganography and steganalysis.

Deep neural networks have been proved that they are vulnerable to adversarial examples, which are generated by adding human-imperceptible perturbations to images. To defend these adversarial examples, various detection based methods have been proposed. However, most of them perform poorly on detecting adversarial examples with extremely slight perturbations. By exploring these adversarial examples, we find that there exists compliance between perturbations and prediction confidence, which guides us to detect few-perturbation attacks from the aspect of prediction confidence. To detect both few-perturbation attacks and large-perturbation attacks, we propose a method beyond image space by a two-stream architecture, in which the image stream focuses on the pixel artifacts and the gradient stream copes with the confidence artifacts. The experimental results show that the proposed method outperforms the existing methods under oblivious attacks and is verified effective to defend omniscient attacks as well.

We introduce Imuge, an image tamper resilient generative scheme for image self-recovery. The traditional manner of concealing image content within the image are inflexible and fragile to diverse digital attack, i.e. image cropping and JPEG compression. To address this issue, we jointly train a U-Net backboned encoder, a tamper localization network and a decoder for image recovery. Given an original image, the encoder produces a visually indistinguishable immunized image. At the recipient's side, the verifying network localizes the malicious modifications, and the original content can be approximately recovered by the decoder, despite the presence of the attacks. Several strategies are proposed to boost the training efficiency. We demonstrate that our method can recover the details of the tampered regions with a high quality despite the presence of various kinds of attacks. Comprehensive ablation studies are conducted to validate our network designs.

Reversible Data Hiding (RDH) in JPEG images is valuable for many applications, such as archive management and image authentication. Recently, there emerged a lot of related works for JPEG RDH, however, the current methods utilize the histogram-shifting-based framework in a constant distortion metric, which does not consider the property of DCT coefficients whose modification distortion is miscellaneous with respect to DCT frequency. In this paper, we propose a novel JPEG RDH scheme under multi-distortion metric. At first, the modification distortion is defined as the impact in the spatial domain caused by modifying DCT coefficients. Different from previous works, here we select coefficients with values ±1, ±2 as cover, which will not cause extra bitrate-expansion and decrease the shift distortion. By minimizing both the modification distortion and shift distortion, a heuristic block selecting strategy is proposed. With the selected coefficients and the corresponding distortion, we use recursive histogram modification under the inconsistent-distortion metric for message embedding. The experimental results show that the proposed method can effectively improve visual quality as well as bring small bitstream expansion. Additionally, the undetectability of the proposed method outperforms those of the existing methods.

The standard tensor voting technique shows its versatility in tasks such as object recognition and semantic segmentation by recognizing feature points and sharp edges that can segment a model into several patches. We propose a neighborhood-level representation-guided tensor voting model for 3D mesh steganalysis. Because existing steganalytic methods do not analyze correlations among neighborhood faces, they are not very effective at discriminating stego meshes from cover meshes. In this paper, we propose to utilize a tensor voting model to reveal the artifacts caused by embedding data. In the proposed steganalytic scheme, the normal voting tensor (NVT) operation is performed on original mesh faces and smoothed mesh faces separately. Then, the absolute values of the differences between the eigenvalues of the two tensors (from the original face and the smoothed face) are regarded as features that capture intricate relationships among the vertices. Subsequently, the extracted features are processed with a nonlinear mapping to boost the feature effectiveness. The experimental results show that the proposed feature sets prevail over state-of-the-art feature sets including LFS64 and ELFS124 under various steganographic schemes.

In this paper, we look into the problem of 3D adversary attack, and propose to leverage the internal properties of the point clouds and the adversarial examples to design a new self-robust deep neural network (DNN) based 3D recognition systems. As a matter of fact, on one hand, point clouds are highly structured. Hence for each local part of clean point clouds, it is possible to learn what is it (``part of a bottle") and its relative position (``upper part of a bottle") to the global object center. On the other hand, with the visual quality constraint, 3D adversarial samples often only produce small local perturbations, thus they will roughly keep the original global center but may cause incorrect local relative position estimation. Motivated by these two properties, we use relative position (dubbed as ``gather-vector") as the adversarial indicator and propose a new robust gather module. Equipped with this module, we further propose a new self-robust 3D point recognition network. Through extensive experiments, we demonstrate that the proposed method can improve the robustness of the target attack under the white-box setting significantly. For I-FGSM based attack, our method reduces the attack success rate from 94.37 \% to 75.69 \%. For C\&W based attack, our method reduces the attack success rate more than 40.00 \%. Moreover, our method is complementary to other types of defense methods to achieve better defense results.

Deep neural networks have made tremendous progress in 3D point-cloud recognition. Recent works have shown that these 3D recognition networks are also vulnerable to adversarial samples produced from various attack methods, including optimization-based 3D Carlini-Wagner attack, gradient-based iterative fast gradient method, and skeleton-detach based point-dropping. However, after a careful analysis, these methods are either extremely slow because of the optimization/iterative scheme, or not flexible to support targeted attack of a specific category. To overcome these shortcomings, this paper proposes a novel label guided adversarial network (LG-GAN) for real-time flexible targeted point cloud attack. To the best of our knowledge, this is the first generation based 3D point cloud attack method. By feeding the original point clouds and target attack label into LG-GAN, it can learn how to deform the point clouds to mislead the recognition network into the specific label only with a single forward pass. In detail, LG-GAN first leverages one multi-branch adversarial network to extract hierarchical features of the input point clouds, then incorporates the specified label information into multiple intermediate features using the label encoder. Finally, the encoded features will be fed into the coordinate reconstruction decoder to generate the target adversarial sample. By evaluating different point-cloud recognition models (e.g., PointNet, PointNet++ and DGCNN), we demonstrate that the proposed LG-GAN can support flexible targeted attack on the fly while guaranteeing good attack performance and higher efficiency simultaneously.

Recent work has demonstrated that neural networks are vulnerable to adversarial examples. To escape from the predicament, many works try to harden the model in various ways, in which adversarial training is an effective way which learns robust feature representation so as to resist adversarial attacks. Meanwhile, the self-supervised learning aims to learn robust and semantic embedding from data itself. With these views, we introduce self-supervised learning to against adversarial examples in this paper. Specifically, the self-supervised representation coupled with k-Nearest Neighbour is proposed for classification. To further strengthen the defense ability, self-supervised adversarial training is proposed, which maximizes the mutual information between the representations of original examples and the corresponding adversarial examples. Experimental results show that the self-supervised representation outperforms its supervised version in respect of robustness and self-supervised adversarial training can further improve the defense ability efficiently.

Deep convolutional neural networks have made outstanding contributions in many fields such as computer vision in the past few years and many researchers published well-trained network for downloading. But recent studies have shown serious concerns about integrity due to model-reuse attacks and backdoor attacks. In order to protect these open-source networks, many algorithms have been proposed such as watermarking. However, these existing algorithms modify the contents of the network permanently and are not suitable for integrity authentication. In this paper, we propose a reversible watermarking algorithm for integrity authentication. Specifically, we present the reversible watermarking problem of deep convolutional neural networks and utilize the pruning theory of model compression technology to construct a host sequence used for embedding watermarking information by histogram shift. As shown in the experiments, the influence of embedding reversible watermarking on the classification performance is less than ±0.5% and the parameters of the model can be fully recovered after extracting the watermarking. At the same time, the integrity of the model can be verified by applying the reversible watermarking: if the model is modified illegally, the authentication information generated by original model will be absolutely different from the extracted watermarking information.

Steganography is an information hiding technique for covert communication. So far Syndrome-Trellis Codes (STC), a convolutional codes-based method, is the only near-optimal coding method, i.e., it can approach the rate-distortion bound of content-adaptive steganography in practice. However, as a secure communication application, steganography needs the diversity of coding methods. This paper proposes another and a better near-optimal steganographic coding method based on polar codes, using Successive Cancellation List (SCL) decoding algorithm to minimize additive distortion in steganography. Considering a steganographic channel as a binary symmetric channel, the proposed Steganographic Polar Codes (SPC) chooses parity-check matrix by setting embedding payload as the initial value of Arikan's heuristic and computes decoding channel metric from the optimal modification probability of minimal distortion model. To overcome the inherent defect of polar codes only suiting for code length of a power of 2, we introduce three strategies to generalize SPC for arbitrary length. Experimental results validate the versatility of SPC to minimize arbitrary distortion. When compared with STC, the overall coding performance of SPC is more superior with low embedding complexity. This work verifies the availability of polar codes for the practical construction of steganographic codes and provides a methodology for designing better steganographic codes based on any advance of polar coding/decoding.

Steganography is the art of covert communication, which aims to hide the secret messages into cover medium while achieving high undetectability. To this end, the framework of minimal distortion embedding is widely adopted for adaptive steganography, where a well-designed distortion function is significant. In this paper, inspired by the phenomenon that the modification of audio samples with the low amplitude will be easily detected, a novel distortion is presented for audio steganography. Taking the fragility of the low amplitude audio samples into account, the proposed distortion is inversely proportional to the amplitude. Furthermore, in order to resist the strong steganalysis, the derivative filter is utilized for acquiring the residual of audio, which plays an important role in distortion definition. The experimental results show that the proposed distortion outperforms the state-of-the-art methods defending strong steganalytic methods. To take a step forward, considering the mutual impact caused by embedding modification, the non-additive extensions of the proposed methods are put forward. The extending experiments show that in most cases, the proposed non-additive extensions can achieve higher level of security than the original methods.

This paper designs a novel underpainting based camera shooting resilient (CSR) document watermarking algorithm for dealing with the leak source tracking problem. By applying such algorithm, we can extract the authentication watermark information from the candid photographs. The watermarked underpainting contains three significant properties. 1) Inconspicuousness. The watermarked underpainting is inconspicuous and it will not easily be maliciously attacked. 2) Robustness. We propose DCT-based watermark embedding algorithm and distortion compensation based extracting algorithm, which make the watermark robust to camera shooting process. 3) Autocorrelation. We design the flip-based method to arrange the watermarked underpainting. So that a complete watermark region can be accurately located even if part of the document is recorded. Compared with previous watermarking algorithms, the proposed scheme guaranteed content independent embedding as well as the robustness to the camera shooting process. Besides, the proposed scheme satisfies the accuracy of extraction even when the captured document is incomplete.

Previous studies have exhibited that incorporating side-information, e.g., a high-quality precover image, can significantly improve steganographic security for JPEG images. This motivates us to estimate the side-information for traditional steganographic scenario in which only a JPEG image is available. It is expected to achieve high-level security by utilizing the estimated side-information similar to side-informed steganography, even though the estimated side-information is not perfectly precise. In this paper, a general framework of side-information estimated (SIE) JPEG steganography is proposed, under which the core problems are how to better estimate the precover and modulate the distortion function correspondingly. To address the two problems, we test several denoising filters and a deblocking filter to obtain the estimated precover, and we introduce two implementation models for modulating the costs. We finally recommend the combination of the deblocking filter and the modulation model using the polarity of the estimated rounding error. The experimental results show that the proposed method dramatically improves the existing additive distortions for images of an arbitrary quality factor and outperforms the state-of-the-art methods based on estimating side-information when resisting modern steganalysis.

Neural networks are vulnerable to adversarial examples, which poses a threat to their application in security sensitive systems. We propose a Denoiser and UPsampler Network (DUP-Net) structure as defenses for 3D adversarial point cloud classification, where the two modules reconstruct surface smoothness by dropping or adding points. In this paper, statistical outlier removal (SOR) and a data-driven upsampling network are considered as denoiser and upsampler respectively. Compared with baseline defenses, DUP-Net has three advantages. First, with DUP-Net as a defense, the target model is more robust to white-box adversarial attacks. Second, the statistical outlier removal provides added robustness since it is a non-differentiable denoising operation. Third, the upsampler network can be trained on a small dataset and defends well against adversarial attacks generated from other point cloud datasets. We conduct various experiments to validate that DUP-Net is very effective as defense in practice. Our best defense eliminates 83.8% of C&W and l2 loss based attack (point shifting), 50.0% of C&W and Hausdorff distance loss based attack (point adding) and 9.0% of saliency map based attack (point dropping) under 200 dropped points on PointNet.

Currently, the most successful model for image adaptive steganography is the framework of minimal distortion, in which a reasonable definition of costs can improve the security level. In the authors' previous work, they developed a rule for cost reassignment in spatial domain called the ‘controversial pixel prior (CPP)’ rule, which defines controversial pixels by utilizing the controversies among several comparable schemes. The CPP rule gives controversial pixels higher modification priorities. In this study, they investigate migrating the CPP rule from the spatial domain to the joint photographic experts group (JPEG) domain and name it the J‐CPP rule. In JPEG images, the cover elements are discrete cosine transform (DCT) coefficients and variant factors mayinfluence the distortion definition includingquantisation step, inter‐blocks correlation and block energy. However, there is no evidence to reveal which factor is of highest priority for promoting security. In this work, they investigate which factor is more helpful in promoting J‐CPP rule, and they finally determine to set the spatial block residual as a penalty to perfect J‐CPP rule. Through extensive experiments on different JPEG steganographic algorithms and steganalysis features, they demonstrate that the J‐CPP rule can improve the security of JPEG adaptive steganography.

Minimal distortion steganography is the most successful model for adaptive steganography, in which the cost function determines the security. Texture complexity is the major factor in defining cost function in images. In this paper, we proposed a method to improve the cost function of JPEG steganography by exploiting the texture in microscale. The proposed scheme is designed by using a “microscope” to highlight details in an image, so that distortion definition can be more refined. Linear unsharp masking acts as the microscope, because it can accentuate the texture region as well as maintain the original characteristics of images. Inter-block spreading rule is proposed to further strengthen the security. We improve the state-of-the-art schemes, J-UNIWARD and UERD, as J-UNIWARD has outstanding performance on resisting detection while UERD has significant lower computational complexity. In order to keep high efficiency of UERD, filtering in the DCT domain is introduced. Extending experiments show that in most cases the proposed methods (J-MSUNIWARD and MSUERD) can achieve a higher level of security than the original methods.

This paper proposes a novel screen-shooting resilient watermarking scheme, which means that if the watermarked image is displayed on the screen and the screen information is captured by the camera, we can still extract the watermark message from the captured photo. To realize such demands, we analyzed the special distortions caused by the screen-shooting process, including lens distortion, light source distortion, and moiré distortion. To resist the geometric deformation caused by lens distortion, we proposed an intensity-based scale-invariant feature transform (I-SIFT) algorithm which can accurately locate the embedding regions. As for the loss of image details caused by light source distortion and moiré distortion, we put forward a small-size template algorithm to repeatedly embed the watermark into different regions, so that at least one complete information region can survive from distortions. At the extraction side, we designed a cross-validation-based extraction algorithm to cope with repeated embedding. The validity and correctness of the extraction method are verified by hypothesis testing. Furthermore, to boost the extraction speed, we proposed a SIFT feature editing algorithm to enhance the intensity of the keypoints, based on which, the extraction accuracy and extraction speed can be greatly improved. The experimental results show that the proposed watermarking scheme achieves high robustness for screen-shooting process. Compared with the previous schemes, our algorithm provides significant improvement in robustness for screen-shooting process and extraction efficiency.

We propose a novel technique for steganography on 3-D meshes so as to resist steganalysis. The majority of existing methods modulate vertex coordinates to embed messages in a nonadaptive way. We take account of complexity of local regions as joint distortion of a triple unit (vertice) and coding method such as syndrome trellis codes to adaptively embed messages, which owns stronger security with respect to existing steganalysis. Key to the distortion is a novel formulation of adaptive steganography, which relies on some effective steganalytic features such as variation of vertex normal. We provide quantitative and qualitative comparisons of our method with several baselines against steganalytic features LFS64, LFS76, and ensemble classifiers, and show that it outperforms the current state of the art. Meanwhile, we proposed an attacking method on steganography proposed by Chao et al. (2009) with a high detection rate.

While most techniques of reversible data hiding in encrypted images (RDH-EI) are developed for uncompressed images, this paper provides a separable reversible data hiding protocol for encrypted JPEG bitstreams. We first propose a JPEG encryption algorithm, which enciphers an image to a smaller size and keeps the format compliant to JPEG decoder. After a content owner uploads the encrypted JPEG bitstream to a remote server, a data hider embeds an additional message into the encrypted copy without changing the bitstream size. On the recipient side, the original bitstream can be reconstructed losslessly using an iterative recovery algorithm based on the blocking artifact. Since message extraction and image recovery are separable, anyone who has the embedding key can extract the message from the marked encrypted copy. Experimental results show that the proposed method outperforms a previous work in terms of separation capability, embedding capacity and security.

Reversible data hiding in encrypted domain (RDH-ED) has greatly attracted researchers as the original content can be losslessly reconstructed after the embedded data are extracted, while the content owner's privacy remains protected. Most of the existing RDH-ED algorithms are designed for grayscale/color images, which cannot be directly applied to other carriers, such as three-dimensional (3D) meshes. With the rapid development of 3D related applications, 3D models have been widely used on the Internet, which motivated us to design a reliable RDH-ED scheme for 3D meshes. The proposed method maps decimals of the vertex coordinates into integers first, so that a bit-stream encryption technique can be executed. With a data-hiding key, several least-significant bits are operated to embed data. By using the encryption key, a receiver can roughly reconstruct the content of the mesh. According to the data-hiding key, with the aid of spatial correlation in natural mesh models, the embedded data can be successfully extracted and the original mesh can be perfectly recovered. Experiments show that the proposed method has a high data-embedding payload, maintains high values of the decrypted meshes, and has low computational complexity.

Message hiding in texture image synthesis is a novel steganography approach by which we resample a smaller texture image and synthesize a new texture image with a similar local appearance and an arbitrary size. However, the mirror operation over the image boundary is flawed and is easy to attack. We propose an attacking method on this steganography, which can not only detect the stego-images but can also extract the hidden messages.