Penetration testing (pentesting) assesses cybersecurity through simulated attacks, while the conventional manual-based method is costly, time-consuming, and personnel-constrained. Reinforcement learning (RL) provides an agent-environment interaction learning paradigm, making it a promising way for autonomous pentesting. However, agents’ scalability in large action spaces and policy transferability across scenarios limit the applicability of RL-based autonomous pentesting. To address these challenges, we present a novel autonomous pentesting framework based on reinforcement learning (namely APRIL) to train agents that are scalable and transferable in large action spaces. In APRIL, we construct realistic, bounded, host-level state space via embedding techniques to avoid the complexities of dealing with unbounded network-level information. We employ semantic correlations between pentesting actions as prior knowledge to represent discrete action space into a continuous and semantically meaningful embedding space. Agents are then trained to reason over actions within the action embedding space, where two key methods are applied: an upper-confidence bound-based action refinement method to encourage efficient exploration, and a distance-aware loss to improve learning efficiency and generalization performance. We conduct experiments in simulated scenarios constructed based on virtualized vulnerable environments. The results demonstrate APRIL's scalability in large action spaces and its ability to facilitate policy transfer across diverse scenarios.

The visual perception systems aim to autonomously collect consecutive visual data and perceive the relevant information online like human beings. In comparison with the classical static visual systems focusing on fixed tasks (e.g., face recognition for visual surveillance), the real-world visual systems (e.g., the robot visual system) often need to handle unpredicted tasks and dynamically changed environments, which need to imitate human-like intelligence with open-ended online learning ability. Therefore, we provide a comprehensive analysis of open-ended online learning problems for autonomous visual perception in this survey. Based on "what to online learn" among visual perception scenarios, we classify the open-ended online learning methods into five categories: instance incremental learning to handle data attributes changing, feature evolution learning for incremental and decremental features with the feature dimension changed dynamically, class incremental learning and task incremental learning aiming at online adding new coming classes/tasks, and parallel and distributed learning for large-scale data to reveal the computational and storage advantages. We discuss the characteristic of each method and introduce several representative works as well. Finally, we introduce some representative visual perception applications to show the enhanced performance when using various open-ended online learning models, followed by a discussion of several future directions.

Robot manipulation, for example, pick-and-place manipulation, is broadly used for intelligent manufacturing with industrial robots, ocean engineering with underwater robots, service robots, or even healthcare with medical robots. Most traditional robot manipulations adopt 2-D vision systems with plane hypotheses and can only generate 3-DOF (degrees of freedom) pose accordingly. To mimic human intelligence and endow the robot with more flexible working capabilities, 3-D vision-based robot manipulation has been studied. However, this task is still challenging in the open world especially for general object recognition and pose estimation with occlusion in cluttered backgrounds and human-like flexible manipulation. In this article, we propose a comprehensive analysis of recent progress about the 3-D vision for robot manipulation, including 3-D data acquisition and representation, robot-vision calibration, 3-D object detection/recognition, 6-DOF pose estimation, grasping estimation, and motion planning. We then present some public datasets, evaluation criteria, comparisons, and challenges. Finally, the related application domains of robot manipulation are given, and some future directions and open problems are studied as well.

Multi-view outlier detection has a wide range of applications and has been well investigated in recent years. However, 1) most existing state-of-the-art methods cannot efficiently handle outlier detection problem for large-scale multi-view data, since exploring pairwise constraints among different views causes highly-computational cost; 2) the data collected from original heterogeneous feature spaces further increases the consistent difficulty of multi-view outlier detection. To address these issues, we present a fast multi-view outlier detection model via learning a low-rank latent subspace representation with deep encoder architecture, which can not only efficiently identify the outliers for large-scale data even with numerous data views, but also exploit a discriminative common latent subspace shared by all the views. First, we learn a set of orthogonal bases as view-specific dictionaries from a small dataset, which is randomly sampled from the original dataset. Benefitting from view-specific dictionaries, the sampled data is projected and decomposed as a shared and discriminative latent subspace representations, which correspond to the view-consistent and view-specific components across multiple views, respectively. Then, the obtained discriminative latent representations are applied to train the view-specific deep encoders, which can efficiently compute the abnormal score for the remaining instances. Our proposed model can cost-effectively identify the outliers in large-scale datasets from numerous data views with less computational complexity. Experiments conducted on eight real datasets and a synthesis dataset show that our proposed model outperforms the existing ones on effectiveness and efficiency.

Currently, visual-tactile fusion learning for robotic object recognition has achieved appealing performance, due to the fact that visual and tactile data can offer complementary information. However: 1) the distinct gap between vision and touch makes it difficult to fully explore the complementary information, which would further lead to performance degradation and 2) most of the existing visual-tactile fused learning methods assume that visual and tactile data are complete, which is often difficult to be satisfied in many real-world applications. In this article, we propose a partial visual-tactile fused (PVTF) framework for robotic object recognition to address these challenges. Specifically, we first employ two modality-specific (MS) encoders to encode partial visual-tactile data into two incomplete subspaces (i.e., visual subspace and tactile subspace). Then, a modality gap mitigated (MGM) network is adopted to discover modality-invariant high-level label information, which is utilized to generate gap loss and further help updating the MS encoders for relatively consistent visual and tactile subspaces generation. In this way, the huge gap between vision and touch is mitigated, which would further contribute to mine the complementary visual-tactile information. Finally, to achieve data completeness and complementary visual-tactile information exploration simultaneously, a cycle subspace leaning technique is proposed to project the incomplete subspaces into a complete subspace by fully exploiting all the obtainable samples, where complete latent representations with maximum complementary information can be learned. A lot of comparative experiments conducted on three visual-tactile datasets validate the advantage of the proposed PVTF framework, by comparing with state-of-the-art baselines.

Reversible Data Hiding (RDH) in JPEG images is valuable for many applications, such as archive management and image authentication. Recently, there emerged a lot of related works for JPEG RDH, however, the current methods utilize the histogram-shifting-based framework in a constant distortion metric, which does not consider the property of DCT coefficients whose modification distortion is miscellaneous with respect to DCT frequency. In this paper, we propose a novel JPEG RDH scheme under multi-distortion metric. At first, the modification distortion is defined as the impact in the spatial domain caused by modifying DCT coefficients. Different from previous works, here we select coefficients with values ±1, ±2 as cover, which will not cause extra bitrate-expansion and decrease the shift distortion. By minimizing both the modification distortion and shift distortion, a heuristic block selecting strategy is proposed. With the selected coefficients and the corresponding distortion, we use recursive histogram modification under the inconsistent-distortion metric for message embedding. The experimental results show that the proposed method can effectively improve visual quality as well as bring small bitstream expansion. Additionally, the undetectability of the proposed method outperforms those of the existing methods.

Deep Neural Networks (DNNs) have recently led to significant improvements in many fields. However, DNNs are vulnerable to adversarial examples which are samples with imperceptible perturbations while dramatically misleading the DNNs. Moreover, adversarial examples can be used to perform an attack on various kinds of DNN based systems, even if the adversary has no access to the underlying model. Many defense methods have been proposed, such as obfuscating gradients of the networks or detecting adversarial examples. However it is proved out that these defense methods are not effective or cannot resist secondary adversarial attacks. In this paper, we point out that steganalysis can be applied to adversarial examples detection, and propose a method to enhance steganalysis features by estimating the probability of modifications caused by adversarial attacks. Experimental results show that the proposed method can accurately detect adversarial examples. Moreover, secondary adversarial attacks are hard to be directly performed to our method because our method is not based on a neural network but based on high-dimensional artificial features and Fisher Linear Discriminant ensemble.

Weakly-supervised learning under image-level labels supervision has been widely applied to semantic segmentation of medical lesions regions. However, 1) most existing models rely on effective constraints to explore the internal representation of lesions, which only produces inaccurate and coarse lesions regions; 2) they ignore the strong probabilistic dependencies between target lesions dataset (e.g., enteroscopy images) and well-to-annotated source diseases dataset (e.g., gastroscope images). To better utilize these dependencies, we present a new semantic lesions representation transfer model for weakly-supervised endoscopic lesions segmentation, which can exploit useful knowledge from relevant fully-labeled diseases segmentation task to enhance the performance of target weakly-labeled lesions segmentation task. More specifically, a pseudo label generator is proposed to leverage seed information to generate highly-confident pseudo pixel labels by incorporating class balance and super-pixel spatial prior. It can iteratively include more hard-to-transfer samples from weakly-labeled target dataset into training set. Afterwards, dynamically-searched feature centroids for same class among different datasets are aligned by accumulating previously-learned features. Meanwhile, adversarial learning is also employed in this paper, to narrow the gap between the lesions among different datasets in output space. Finally, we build a new medical endoscopic dataset with 3659 images collected from more than 1100 volunteers. Extensive experiments on our collected dataset and several benchmark datasets validate the effectiveness of our model.

Image dehazing aims to recover the uncorrupted content from a hazy image. Instead of leveraging traditional low-level or handcrafted image priors as the restoration constraints, e.g., dark channels and increased contrast, we propose an end-to-end gated context aggregation network to directly restore the final haze-free image. In this network, we adopt the latest smoothed dilation technique to help remove the gridding artifacts caused by the widely-used dilated convolution with negligible extra parameters, and leverage a gated sub-network to fuse the features from different levels. Extensive experiments demonstrate that our method can surpass previous state-of-the-art methods by a large margin both quantitatively and qualitatively. In addition, to demonstrate the generality of the proposed method, we further apply it to the image deraining task, which also achieves the state-of-the-art performance.

Different from all the previous reversible data hiding schemes, a completely novel one for the color image is proposed, which reversibly embeds messages into the color host image without modifying its corresponding gray version. The property of grayscale invariance is valuable, because many applications and image processing algorithms for color images are based on the corresponding gray versions, such as black and white printing, producing reading materials for color blind people, single-channel image processing, and so on. Thus, in terms of these applications and image processing algorithms, the presented scheme will make the generated color marked image be free for its further uses. In this paper, the unchanged gray version is utilized efficiently in both the embedding processes and the extracting processes. Messages are embedded into the red and blue channels of color image, and then the green channel is adjusted adaptively to remove the offsets from the gray version caused by modifying its red and blue channels. To return the adjusted green channel, error correcting bits guaranteeing the reversibility are regarded as one part of payloads to be recursively embedded. Therefore, the reversibility and the property of grayscale invariance are both achieved.

Recursive code construction (RCC), based on the optimal transition probability matrix (OTPM), approaching the rate-distortion bound of reversible data hiding (RDH) has been proposed. Using the existing methods, OTPM can be effectively estimated only for a consistent distortion metric, i.e., if the host elements at different positions share the same distortion metric. However, in many applications, the distortion metrics are position dependent and should thus be inconsistent. Inconsistent distortion metrics can usually be quantified as a multi-distortion metric. In this paper, we first formulate the rate-distortion problem of RDH under a multi-distortion metric and subsequently propose a general framework to estimate the corresponding OTPM, with which RCC is extended to approach the rate-distortion bound of RDH under the multi-distortion metric. We apply the proposed framework to two examples of inconsistent distortion metrics: RDH in color image and reversible steganography. The experimental results show that the proposed method can efficiently improve upon the existing techniques.

With the popularity of outsourcing data to the cloud, it is vital to protect the privacy of data and enable the cloud server to easily manage the data at the same time. Under such demands, reversible data hiding in encrypted images (RDH-EI) attracts more and more researchers' attention. In this paper, we propose a novel framework for RDH-EI based on reversible image transformation (RIT). Different from all previous encryption-based frameworks, in which the ciphertexts may attract the notation of the curious cloud, RIT-based framework allows the user to transform the content of original image into the content of another target image with the same size. The transformed image, that looks like the target image, is used as the “encrypted image,” and is outsourced to the cloud. Therefore, the cloud server can easily embed data into the “encrypted image” by any RDH methods for plaintext images. And thus a client-free scheme for RDH-EI can be realized, that is, the data-embedding process executed by the cloud server is irrelevant with the processes of both encryption and decryption. Two RDH methods, including traditional RDH scheme and unified embedding and scrambling scheme, are adopted to embed watermark in the encrypted image, which can satisfy different needs on image quality and large embedding capacity, respectively.