Vertical federated learning (VFL) shows promise for enabling collaborative learning among Internet of Vehicle systems (IoVs) without requiring the sharing of private training data. However, existing work has exposed VFL’s vulnerability to privacy-stealing attacks, where an honest but curious server might reconstruct a client’s raw data from client-uploaded embeddings. In this work, we first elucidate the intrinsic mechanisms of privacy attacks from an information theory perspective, which provides a solid foundation for potential defensive strategies. Based on our findings, we introduce PriVFL, a defense mechanism based on information bottleneck theory. PriVFL is designed to safeguard the privacy of VFL-based IoVs by enabling shared embeddings to extract minimal information from input data, while preserving the information essential to target labels. Specifically, PriVFL restricts the information contained in embeddings by reducing the upper bound of mutual information between the raw samples and embeddings uploaded from local clients. Meanwhile, PriVFL ensures the effectiveness of the model by increasing the mutual information lower bound between embeddings and samples’ labels. Our evaluation includes 5 benchmark data sets and 4 different models. Experimental results demonstrate that PriVFL effectively mitigates privacy attacks while preserving the model’s effectiveness. These findings underscore that PriVFL can significantly enhance the privacy of VFL-based IoVs, thereby bolstering the development of practical IoV applications.

Graph neural networks, effectively harnessing the extensive interactive data from Internet of Things (IoT) devices, significantly enhance service quality in IoT systems. However, traditional centralized training leads to the leakage of private data during the data collection and model training phases in IoT scenarios. Federated learning (FL) has emerged as a promising approach, facilitating collaborative model training across diverse IoT devices without sharing sensitive data. The intricate types and relationships among IoT devices from various institutions highlight the issues of class imbalance and graph heterogeneity across different clients. These issues decrease the performance of FL models. In this work, we focus on a more realistic scenario where the IoT institutions have only limited amount and types of data. We propose a cross-institutional federated heterogeneous graph learning method called FedHGL. It aims to mitigate the negative effects of class imbalance while maintaining the private data locally on clients during collaborative training. We employ a heterogeneous graph neural network as the training model for clients. FedHGL generates cross-client minority class samples to enhance the model performance. Additionally, it incorporates a compensation mechanism to prevent forgetting global information. FedHGL designs an adaptive aggregation coefficient that assigns weights to IoT institutions according to the class imbalance of their data, thereby optimizing the aggregation process. Extensive experiments demonstrate the effectiveness of FedHGL for class imbalance and heterogeneous graph data.

Autonomous systems possess the features of inferring their own state, understanding their surroundings, and performing autonomous navigation. With the applications of learning systems, like deep learning and reinforcement learning, the visual-based self-state estimation, environment perception, and navigation capabilities of autonomous systems have been efficiently addressed, and many new learning-based algorithms have surfaced with respect to autonomous visual perception and navigation. In this review, we focus on the applications of learning-based monocular approaches in ego-motion perception, environment perception, and navigation in autonomous systems, which is different from previous reviews that discussed traditional methods. First, we delineate the shortcomings of existing classical visual simultaneous localization and mapping (vSLAM) solutions, which demonstrate the necessity to integrate deep learning techniques. Second, we review the visual-based environmental perception and understanding methods based on deep learning, including deep learning-based monocular depth estimation, monocular ego-motion prediction, image enhancement, object detection, semantic segmentation, and their combinations with traditional vSLAM frameworks. Then, we focus on the visual navigation based on learning systems, mainly including reinforcement learning and deep reinforcement learning. Finally, we examine several challenges and promising directions discussed and concluded in related research of learning systems in the era of computer science and robotics.

Deep learning-based methods mymargin have achieved remarkable performance in 3-D sensing since they perceive environments in a biologically inspired manner. Nevertheless, the existing approaches trained by monocular sequences are still prone to fail in dynamic environments. In this work, we mitigate the negative influence of dynamic environments on the joint estimation of depth and visual odometry (VO) through hybrid masks. Since both the VO estimation and view reconstruction process in the joint estimation framework is vulnerable to dynamic environments, we propose the cover mask and the filter mask to alleviate the adverse effects, respectively. As the depth and VO estimation are tightly coupled during training, the improved VO estimation promotes depth estimation as well. Besides, a depth-pose consistency loss is proposed to overcome the scale inconsistency between different training samples of monocular sequences. Experimental results show that both our depth prediction and globally consistent VO estimation are state of the art when evaluated on the KITTI benchmark. We evaluate our depth prediction model on the Make3D dataset to prove the transferability of our method as well.

Traditional machine learning-based intrusion detection often only considers a single algorithm to identify intrusion data, lack of the flexibility method, low detection rate, no handing high-dimensional data, and cannot solve these problems well. In order to improve the performance of intrusion detection system, a novel general intrusion detection framework was proposed in this paper, which consists of five parts: preprocessing module, autoencoder module, database module, classification module, and feedback module. The data processed by the preprocessing module are compressed by the autoencoder module to obtain a lower-dimensional reconstruction feature, and the classification result is obtained through the classification module. Compressed features of each traffic are stored in the database module which can both provide retraining and testing for the classification module and restore these features to the original traffic for postevent analysis and forensics. For evaluation of the framework performance proposed, simulation was conducted with the CICIDS2017 dataset to the real traffic of the network. As the experimental results, the accuracy of binary classification and multiclass classification is better than previous work, and high-level accuracy was reached for the restored traffic. At the last, the possibility was discussed on applying the proposed framework to edge/fog networks.

Semantic segmentation and depth completion are two challenging tasks in scene understanding, and they are widely used in robotics and autonomous driving. Although several studies have been proposed to jointly train these two tasks using some small modifications, such as changing the last layer, the result of one task is not utilized to improve the performance of the other one despite that there are some similarities between these two tasks. In this article, we propose multitask generative adversarial networks (Multitask GANs), which are not only competent in semantic segmentation and depth completion but also improve the accuracy of depth completion through generated semantic images. In addition, we improve the details of generated semantic images based on CycleGAN by introducing multiscale spatial pooling blocks and the structural similarity reconstruction loss. Furthermore, considering the inner consistency between semantic and geometric structures, we develop a semantic-guided smoothness loss to improve depth completion results. Extensive experiments on the Cityscapes data set and the KITTI depth completion benchmark show that the Multitask GANs are capable of achieving competitive performance for both semantic segmentation and depth completion tasks.

Previous work has shown that adversarial learning can be used for unsupervised monocular depth and visual odometry (VO) estimation, in which the adversarial loss and the geometric image reconstruction loss are utilized as the mainly supervisory signals to train the whole unsupervised framework. However, the performance of the adversarial framework and image reconstruction is usually limited by occlusions and the visual field changes between the frames. This article proposes a masked generative adversarial network (GAN) for unsupervised monocular depth and ego-motion estimations. The MaskNet and Boolean mask scheme are designed in this framework to eliminate the effects of occlusions and impacts of visual field changes on the reconstruction loss and adversarial loss, respectively. Furthermore, we also consider the scale consistency of our pose network by utilizing a new scale-consistency loss, and therefore, our pose network is capable of providing the full camera trajectory over a long monocular sequence. Extensive experiments on the KITTI data set show that each component proposed in this article contributes to the performance, and both our depth and trajectory predictions achieve competitive performance on the KITTI and Make3D data sets.

We describe a novel approach for creating a three-dimensional (3-D) face structure from multiple image views of a human face taken at a priori unknown poses by appropriately morphing a generic 3-D face. A cubic explicit polynomial in 3-D is used to morph a generic face into the specific face structure. The 3-D face structure allows for accurate pose estimation as well as the synthesis of virtual images to be matched with a test image for face identification. The estimation of a 3-D person's face and pose estimation is achieved through the use of a distance map metric. This distance map residual error (geometric-based face classifier) and the image intensity residual error are fused in identifying a person in the database from one or more arbitrary image view(s). Experimental results are shown on simulated data in the presence of noise, as well as for images of real faces, and promising results are obtained.

We describe a novel approach for creating a 3D face structure from multiple image views of a human face taken at a priori unknown poses by appropriately morphing a generic 3D face. A 3D cubic explicit polynomial is used to morph a generic face into the specific face structure. This allows the creation of a database of 3D faces that is used in identifying a person (in the database) from one or more arbitrary image view(s). The estimation of a 3D person's face and its recognition from the database of faces is achieved through the use of a distance map metric. The use of this metric avoids either resorting to the formidable task of establishing feature point correspondences in the image views, or even more severely, relying on the extremely view-sensitive image intensity (texture). Experimental results are shown for images of real faces, and excellent results are obtained.