Recent research has shown that architectures utilizing reinforcement learning (RL) are effective in cost-based image steganography. However, these architectures only learn embedding probabilities rather than costs, and are trained for a specific embedding payload, making it difficult to extend the trained model to serve other payloads. In this paper, we propose a payload-independent cost learning framework using RL called PICO-RL. This framework directly learns universal costs that can be applied to any payload. PICO-RL incorporates an optimal probability approximation (OPA) module that can calculate the required probability map for embedding simulation directly from a learned cost map for any payload, eliminating the need for time-consuming searches for a valid probability scaling parameter. Additionally, PICO-RL uses an advanced steganalysis environment network to provide more effective reward feedback for learning. During RL training, the learned cost maps of different payloads converge and eventually become similar under the OPA constraint, resulting in payload independence. Experimental results demonstrate that a well-trained PICO-RL model, which acts as a universal cost function, defines costs with superior security performance against steganalysis and has better coding compatibility when encoding with practical steganographic codes.

The quality of videos is the primary concern of video service providers. Built upon deep neural networks, video quality assessment (VQA) has rapidly progressed. Although existing works have introduced the knowledge of the human visual system (HVS) into VQA, there are still some limitations that hinder the full exploitation of HVS, including incomplete modeling with few HVS characteristics and insufficient connection among these characteristics. In this article, we present a novel spatial-temporal VQA method termed HVS-5M, wherein we design five modules to simulate five characteristics of HVS and create a bioinspired connection among these modules in a cooperative manner. Specifically, on the side of the spatial domain, the visual saliency module first extracts a saliency map. Then, the content-dependency and the edge masking modules extract the content and edge features, respectively, which are both weighted by the saliency map to highlight those regions that human beings may be interested in. On the other side of the temporal domain, the motion perception module extracts the dynamic temporal features. Besides, the temporal hysteresis module simulates the memory mechanism of human beings and comprehensively evaluates the video quality according to the fusion features from the spatial and temporal domains. Extensive experiments show that our HVS-5M outperforms the state-of-the-art VQA methods. Ablation studies are further conducted to verify the effectiveness of each module toward the proposed method. The source code is available at https://github.com/GZHU-DVL/HVS-5M.

Deep Image Hiding (DIH) aims to imperceptibly hide images within image. To improve its security performance, some DIH methods design Security Metrics (SMs) to guide the learning of their hiding networks. However, these methods focus on optimizing their anti-steganalysis ability on specific SMs, resulting in inferior generalization ability. To overcome these limitations, in this paper, we introduce meta-learning into DIH and propose Meta Security Metric-based DIH (MSM-DIH). In the MSM-DIH, the Invertible Neural Network (INN)-based hiding network is learned under the guidance of a learnable meta SM generalized from multiple fixed source SMs, and each SM is composed of a metric network and a contrastive loss function. Specifically, MSM-DIH is trained with bi-level optimization. In the outer optimization, a meta SM is learned to assign higher security scores for more advanced stego images. Besides, the domain knowledge of steganalysis is transferred from the multiple pre-trained source metric networks to the meta metric network, so as to enhance the generalization ability of the meta SM. In the inner optimization, the hiding network is learned to generate more secure stego images according to the learned meta SM. Experimental results show that our MSM-DIH has achieved the best security performance in most cases.

Since the generative adversarial network (GAN) was proposed by Ian Goodfellow et al. in 2014, it has been widely used in various fields. However, there are only a few works related to image steganography so far. Existing GAN-based steganographic methods mainly focus on the design of generator, and just assign a relatively poorer steganalyzer in discriminator, which inevitably limits the performances of their models. In this paper, we propose a novel Steganographic method based on Generative Multi-Adversarial Network (Steg-GMAN) to enhance steganography security. Specifically, we first employ multiple steganalyzers rather than a single steganalyzer like existing methods to enhance the performance of discriminator. Furthermore, in order to balance the capabilities of the generator and the discriminator during training stage, we propose an adaptive way to update the parameters of the proposed GAN model according to the discriminant ability of different steganalyzers. In each iteration, we just update the poorest one among all steganalyzers in discriminator, while update the generator with the gradients derived from the strongest one. In this way, the performance of generator and discriminator can be gradually improved, so as to avoid training failure caused by gradient vanishing. Extensive comparative results show that the proposed method can achieve state-of-the-art results compared with the traditional steganography and the modern GAN-based steganographic methods. In addition, a large number of ablation experiments verify the rationality of the proposed model.

In recent years, although cost learning methods have made great progress in single-image steganography, its development in batch steganography is relatively slower, which is a more practical communication scenario in the real world. The difficulties are capturing the full view of the image batch and building connections between cost learning and payload allocation by neural networks. To address the issues, this paper proposes a cost learning framework for batch steganography called JoCoP (Joint Cost Learning and Payload Allocation), wherein the policy network is designed to learn the optimal embedding policies for a batch of images via the collaboration between a cost learning module and a payload allocation module. In specific layers of the policy network, in the cost learning module, the intermediate feature maps of embedding costs are extracted for different images independently, which are sent to the payload allocation module. In the payload allocation module, to implement implicit payload allocation, the feature maps corresponding to different images within the same batch are adjusted by an image-wise attention mechanism. Afterwards, these adjusted feature maps are returned to the cost learning module for subsequent feature extraction in the next layer. Owing to the collaboration between the two modules and the batch-level receptive field in the image-wise attention mechanism, the embedding costs and the payload allocation can be jointly optimized in an end-to-end manner. Experimental results show that the proposed JoCoP outperforms existing methods against both single-image steganalyzers and pooled steganalyzers based on feature extraction and convolutional neural networks.

No-Reference Video Quality Assessment (NR-VQA) plays an essential role in improving the viewing experience of end-users. Driven by deep learning, recent NR-VQA models based on Convolutional Neural Networks (CNNs) and Transformers have achieved outstanding performance. To build a reliable and practical assessment system, it is of great necessity to evaluate their robustness. However, such issue has received little attention in the academic community. In this paper, we make the first attempt to evaluate the robustness of NR-VQA models against adversarial attacks, and propose a patch-based random search method for black-box attack. Specifically, considering both the attack effect on quality score and the visual quality of adversarial video, the attack problem is formulated as misleading the estimated quality score under the constraint of just-noticeable difference (JND). Built upon such formulation, a novel loss function called Score-Reversed Boundary Loss is designed to push the adversarial video’s estimated quality score far away from its ground-truth score towards a specific boundary, and the JND constraint is modeled as a strict $L_2$ and $L_\infty$ norm restriction. By this means, both white-box and black-box attacks can be launched in an effective and imperceptible manner. The source code is available at https://github.com/GZHU-DVL/AttackVQA.

While Deep Reinforcement Learning (DRL) has achieved outstanding performance in extensive applications, exploiting its vulnerability with adversarial attacks is essential towards building robust DRL systems. In this work, we aim to propose a novel Decoupled Adversarial Policy (DAP) for attacking the DRL mechanism, whereas the adversarial agent can decompose the adversarial policy into two separate sub-policies: 1) the switch policy which determines if an attacker should launch the attack, and 2) the lure policy which determines the action an attacker induces the victim to take. If the adversarial agent samples an injection action from the switch policy, the attacker can query the pre-constructed database for universal perturbation in the real-time manner, misleading the victim to take the induced action sampled from the lure policy. To train the adversarial agent to learn DAP, we utilize those samples wherein both of the sub-actions from DAP are not restricted by each other or by the external constraint, but can actually affect the attacker’s behaviors. Therefore, we propose trajectory clipping and padding in data pruning, and Decoupled Proximal Policy Optimization (DPPO) in optimizing. Extensive experiments on different Atari games demonstrate the effectiveness of our proposed method. In addition, it can simultaneously implement the real-time and few-steps attack, which outperforms the existing counterparts.

Recently, the success of non-additive steganography has demonstrated that asymmetric distortion can remarkably improve security performance compared with symmetric cost functions. However, most of current existing additive steganographic methods are still based on symmetric distortion. In this paper, for the first time we optimize asymmetric distortion for additive steganography and propose an A3C (Asynchronous Advantage Actor-Critic) based steganographic framework, called ReLOAD. ReLOAD is composed of an actor and a critic, where the former guides action selection for pixel-wise distortion modulation, and the latter evaluates the performance of modulated distortion. Meanwhile, a reward function that considers embedding effects is proposed to unify the goal of steganography and reinforcement learning, so that the minimization of embedding effects can be achieved by learning secure policy to maximize total rewards. Statistical analysis shows that compared with non-additive steganography, ReLOAD achieves lower change rates and makes embedding traces more consistent with cover image textures. Comprehensive experiments conducted on both hand-crafted feature-based and deep learning-based steganalyzers show that ReLOAD significantly promotes the state-of-the-art security performance of current additive methods and even outperforms non-additive steganography when the modification distribution gets sparser.

Jointly learning from multiple datasets can help building versatile intelligent systems yet may give rise to serious concerns of data privacy and model selection. Specifically, on the one hand, these datasets can be distributed at various local clients, who may not be willing or do not ought to share data with each other. On the other hand, it is unrealistic to choose a model architecture that can well suit the disparate patterns and distributions carried by the various datasets in a priori. Whereas many works in federated learning [1] and neural architecture search [2] have been proposed to address one of the two concerns, very few have attempted the both. To close the gap, in this paper we deliver a framework, termedMulti-Granular Federated Neural Architecture Search(MGFNAS), to enable the automation of model architecture search in a federated and thus privacy-preserved setting. We argue that our MGFNAS framework is general in the sense that it does not impose any restriction on the search space or strategy, such that most existing neural architecture search techniques can be readily implemented in. The main idea of our framework is to search the optimal neural network architecture in two levels of granularity, enabling the neural-operator-basedmicro-levelsearch and the cell-basedmacro-levelsearch. The main challenge of implementing our framework lies in the fact that, due to the decentralized nature, the local architectures searched by multiple clients can differ drastically in order to fit their own datasets, while a general method to form the global model by aggregating the local architectures in both micro and macro levels is missing. To solve the issue, we propose a novel aggregation function, named Network Architecture Probabilistic Aggregation (NAPA). The key idea of our NAPA function is to treat the network architectures as graphs, of which the sub-graph structures being frequently appeared across multiple clients are modeled by probabilistic distributions. At each round, a global model is formed by sampling from those distributions in an exploration-exploitation fashion. Extensive experiments are carried out, and the results substantiate the viability and effectiveness of our proposed framework.

In black-box attack, excessive queries to target model may cause suspicion and expose attacker's identity. Equipped with advanced meta learning technique, Meta Attack simulates the target model with a surrogate model, significantly reducing the queries. However, it queries for ZOO-gradients to correct the estimated meta-gradients with a fixed frequency, thereby still leading to massive unnecessary queries. To overcome this limitation, this letter takes the dynamic changes of the accuracy of the estimated gradients as a starting point, and develops a Dynamic Meta Attack (DMA). At the beginning of each fine-tuning round, DMA computes the distance between the above two types of gradients. Such distance metric can reflect the accuracy of the meta-gradients, and guide the dynamic adjustment of query frequency for the ZOO-gradients. Moreover, the working flow of the dynamic fine-tuning process can be controlled by a set of parameters, which are of physical significance and easy to be tuned. By this means, DMA merely launches queries at critical moments, greatly saving query resource. Experiments conducted on MNIST and CIFAR10 show that the proposed DMA requires far fewer queries than existing methods while maintaining a satisfying attack success rate and distortion.

Compared to element-wise embedding, vector-wise embedding based on CPV (color pixel vector) shows its superiority in color image steganography. However, when working with an adversarial embedding scheme for introducing adversarial perturbations, its success rate of deceiving a target CNN (convolutional neural network) steganalyzer dramatically drops. In this paper, inspired by the I-FGSM (iterative fast gradient sign method), we present an effective steganography for color images. Specifically, after decomposing an image into several non-overlapped sub-images, we iteratively and gradually increase the possibilities of generating adversarial perturbations for the CPVs in each sub-image by changing their adversarial costs. The costs are incrementally adjusted with a small step so that their maximum relative variation is minimized. Leveraging a new designed cost adjustment criterion, more modification patterns of CPV can participate in producing effective adversarial perturbations. Extensive experiments demonstrate that the proposed method achieves a high success rate in deceiving the target CNN steganalyzer and stably defending against the detection of other non-target steganalytic schemes for color images.

Although significant progress has been achieved recently in automatic learning of steganographic cost, the existing methods designed for spatial images cannot be directly applied to JPEG images which are more common media in daily life. The difficulties of migration are mainly caused by the characteristics of the$8\times 8$DCT mode structure. To address the issue, in this paper we extend an existing automatic cost learning scheme to JPEG, where the proposed scheme called JEC-RL (JPEG Embedding Cost with Reinforcement Learning) is explicitly designed to tailor the JPEG DCT structure. It works with the embedding action sampling mechanism under reinforcement learning, where a policy network learns the optimal embedding policies via maximizing the rewards provided by an environment network. Following a domain-transition design paradigm, the policy network is composed of three modules, i.e., pixel-level texture complexity evaluation module, DCT feature extraction module, and mode-wise rearrangement module. These modules operate in serial, gradually extracting useful features from a decompressed JPEG image and converting them into embedding policies for DCT elements, while considering JPEG characteristics including inter-block and intra-block correlations simultaneously. The environment network is designed in a gradient-oriented way to provide stable reward values by using a wide architecture equipped with a fixed preprocessing layer with$8\times 8$DCT basis filters. Extensive experiments and ablation studies demonstrate that the proposed method can achieve good security performance for JPEG images against both advanced feature-based and modern CNN-based steganalyzers.

Nowadays a steganography has to face challenges to both feature-based staganalysis and convolutional neural network (CNN) based steganalysis. In this paper, we present a novel steganographic scheme to incorporate synchronizing modification directions and iterative adversarial perturbations to enhance steganographic performance. Firstly an existing steganographic function is employed to compute initial costs. Then the secret message bits are embedded following clustering modification directions profile. If the target CNN classifier discriminates the resulting stego image as the correct class, we change costs in adversarial manners, and then choose a sub-image to re-embed message with changed costs. Adversarial intensity will be iteratively increased until the adversarial stego image can deceive the target CNN classifier, which guarantees that applied adversarial perturbations are minimal and it is unnecessary to search the optimal adversarial intensity. Experiments demonstrate that the proposed method effectively enhances security to counter both feature-based classifiers and CNN classifiers, no matter they are targeted or non-targeted.

Automatic cost learning for steganography based on deep neural networks is receiving increasing attention. Steganographic methods under such a framework have been shown to achieve better security performance than methods adopting hand-crafted costs. However, they still exhibit some limitations that prevent a full exploitation of their potentiality, including using a function-approximated neural-network-based embedding simulator and a coarse-grained optimization objective without explicitly using pixel-wise information. In this article, we propose a new embedding cost learning framework called SPAR-RL (Steganographic Pixel-wise Actions and Rewards with Reinforcement Learning) that overcomes the above limitations. In SPAR-RL, an agent utilizes a policy network which decomposes the embedding process into pixel-wise actions and aims at maximizing the total rewards from a simulated steganalytic environment, while the environment employs an environment network for pixel-wise reward assignment. A sampling process is utilized to emulate the message embedding of an optimal embedding simulator. Through the iterative interactions between the agent and the environment, the policy network learns a secure embedding policy which can be converted into pixel-wise embedding costs for practical message embedding. Experimental results demonstrate that the proposed framework achieves state-of-the-art security performance against various modern steganalyzers, and outperforms existing cost learning frameworks with regard to learning stability and efficiency.

Steganographic schemes are commonly designed in a way to preserve image statistics or steganalytic features. Since most of the state-of-the-art steganalytic methods employ a machine learning (ML)-based classifier, it is reasonable to consider countering steganalysis by trying to fool the ML classifiers. However, simply applying perturbations on stego images as adversarial examples may lead to the failure of data extraction and introduce unexpected artifacts detectable by other classifiers. In this paper, we present a steganographic scheme with a novel operation called adversarial embedding (ADV-EMB), which achieves the goal of hiding a stego message while at the same time fooling a convolutional neural network (CNN)-based steganalyzer. The proposed method works under the conventional framework of distortion minimization. In particular, ADV-EMB adjusts the costs of image elements modifications according to the gradients back propagated from the target CNN steganalyzer. Therefore, modification direction has a higher probability to be the same as the inverse sign of the gradient. In this way, the so-called adversarial stego images are generated. Experiments demonstrate that the proposed steganographic scheme achieves better security performance against the target adversary-unaware steganalyzer by increasing its missed detection rate. In addition, it deteriorates the performance of other adversary-aware steganalyzers, opening the way to a new class of modern steganographic schemes capable of overcoming powerful CNN-based steganalysis.

Generative adversarial network has shown to effectively generate artificial samples indiscernible from their real counterparts with a united framework of two subnetworks competing against each other. In this letter, we first propose an automatic steganographic distortion learning framework using a generative adversarial network, which is composed of a steganographic generative subnetwork and a steganalytic discriminative subnetwork. Via alternately training these two oppositional subnetworks, our proposed framework can automatically learn embedding change probabilities for every pixel in a given spatial cover image. The learnt embedding change probabilities can then be converted to embedding distortions, which can be adopted in the existing framework of minimal-distortion embedding. Under this framework, the distortion function is directly related to the undetectability against the oppositional evolving steganalyzer. Experimental results show that with adversarial learning, our proposed framework can effectively evolve from nearly naive random ±1 embedding at the beginning to much more advanced content-adaptive embedding which tries to embed secret bits in textural regions. The security performance is also steadily improved with increasing training iterations.

It is conventionally assumed that steganographic schemes for gray-scale images can be directly applied to color images by embedding messages independently in different color channels. However, the correlation among color channels may be disturbed and it is unclear how to preserve the channel correlation so as to increase empirical security. In this paper, we propose a strategy called CMD-C (clustering modification directions for color components). The basic idea of the strategy is to change different color components from the same pixel location towards a positive or negative direction consistently. To implement the strategy, we decompose an image into several sub-images in which segmented hidden message bits are successively embedded. The embedding costs of a sub-image are computed by considering the correlation both within and among color channels. Experimental results show that the proposed CMD-C strategy has made great improvement over conventional methods in resisting state-of-the-art steganalytic methods.

In modern steganography, embedding modifications are highly concentrated on the textural regions within an image, as such regions are difficult to model for steganalysis. Previous studies have shown that compared with non-adaptive strategies, this content adaptive strategy achieves stronger security against existing steganalysis. Based on the experiments and analyses, however, we found that this embedding property would inevitably lead to a large limitation in existing adaptive steganography. That is, it is possible for steganalyzers to estimate the regions that have probably been modified after data hiding. In this paper, we propose an adaptive steganalytic scheme based on embedding probabilities of pixels. The main idea of our scheme is that we assign different weights to different pixels in feature extraction. For those pixels with high embedding probabilities, their corresponding weights are larger, since they should contribute more to steganalysis and vice versa. By doing so, we can concentrate our attention on the regions that have probably been modified and significantly reduce the impact of other unchanged smooth regions. It is expected that our proposed method is an improvement on the existing steganalytic methods, which usually assume every pixel has the same contribution to steganalysis. The extensive experiments evaluated on four typical adaptive steganographic methods have shown the effectiveness of the proposed scheme, especially for low embedding rates, for example, lower than 0.20 bpp.

WOW (Wavelet Obtained Weights) [5] is one of the advanced steganographic methods in spatial domain, which can adaptively embed secret message into cover image according to textural complexity. Usually, the more complex of an image region, the more pixel values within it would be modified. In such a way, it can achieve good visual quality of the resulting stegos and high security against typical steganalytic detectors. Based on our analysis, however, we point out one of the limitations in the WOW embedding algorithm, namely, it is easy to narrow down those possible modified regions for a given stego image based on the embedding costs used in WOW. If we just extract features from such regions and perform analysis on them, it is expected that the detection performance would be improved compared with that of extracting steganalytic features from the whole image. In this paper, we first proposed an adaptive steganalytic scheme for the WOW method, and use the spatial rich model (SRM) based features [4] to model those possible modified regions in our experiments. The experimental results evaluated on 10,000 images have shown the effectiveness of our scheme. It is also noted that our steganalytic strategy can be combined with other steganalytic features to detect the WOW and/or other adaptive steganographic methods both in the spatial and JPEG domains.