Recently, machine/deep learning techniques are achieving remarkable success in a variety of intelligent control and management systems, promising to change the future of artificial intelligence (AI) scenarios. However, they still suffer from some intractable difficulty or limitations for model training, such as the out-of-distribution (OOD) issue, in modern smart manufacturing or intelligent transportation systems (ITSs). In this study, we newly design and introduce a deep generative model framework, which seamlessly incorporates the information theoretic learning (ITL) and causal representation learning (CRL) in a dual-generative adversarial network (Dual-GAN) architecture, aiming to enhance the robust OOD generalization in modern machine learning (ML) paradigms. In particular, an ITL- and CRL-enhanced Dual-GAN (ITCRL-DGAN) model is presented, which includes an autoencoder with CRL (AE-CRL) structure to aid the dual-adversarial training with causality-inspired feature representations and a Dual-GAN structure to improve the data augmentation in both feature and data levels. Following a newly designed feature separation strategy, a causal graph is built and improved based on the information theory, which can enhance the causally related factors among the separated core features and further enrich the feature representation with the counterfactual features via interventions based on the refined causal relationships. The ITL is incorporated to improve the extraction of low-dimensional feature representations and learn the optimized causal representations based on the idea of "information flow." A dual-adversarial training mechanism is then developed, which not only enables the generator to expand the boundary of feature distribution in accordance with the optimized feature representation from AE-CRL, but also allows the discriminator to further verify and improve the quality of the augmented data for OOD generalization. Experiment and evaluation results based on an open-source dataset demonstrate the outstanding learning efficiency and classification performance of our proposed model for robust OOD generalization in modern smart applications compared with three baseline methods.

Due to the high dependence of social development on electricity, the failure of energy system equipment often leads to inestimable losses. The use of Internet of Things (IoT) technology to collect real-time data from energy devices and artificial intelligence (AI) technology to prognostic faults in the devices becomes essential to achieve power system security. Although real-time data contains rich descriptions of relevant equipment faults and solutions, the complex data structure and severe imbalance of condition monitoring (CM) data may lead to poor performance of AI models. In this study, we propose a multitask correlation constrained topology learning model for smart prognostic and health management in IoT. In particular, the proposed model mainly includes a feature extraction module, a multitask topology network (MTTN) module, and a class balance loss (CBL) algorithm module. First, the Bi-LSTM is employed to extract word collocation features and numerical features of the data, focusing on the important semantic features in complex structured data. Second, the MTTN is constructed to efficiently utilize the topological dependency between multiple tasks. Finally, a CBL loss function is applied to enhance the focus on the minority classes. Experiment and evaluation results demonstrate that our model has superior learning efficiency and prediction performance, especially on extremely imbalanced data, which can correctly predict the faulty equipment, the fault cause and the fault severity level, providing a rapid and precise reference for advance repair or replacement of energy system equipment in IoT environments.

With the flourish of digital technologies and rapid development of 5G and beyond networks, Metaverse has become an increasingly hotly discussed topic, which offers users with multiple roles for diversified experience interacting with virtual services. How to capture and model users’ multi-platform or cross-space data/behaviors become essential to enrich people with more realistic and immersed experience in Metaverse-enabled smart applications over 5G and beyond networks. In this study, we propose a Personalized Federated Learning with Model-Contrastive Learning (PFL-MCL) framework, which may efficiently enhance the communication and interaction in human-centric Metaverse environments by making use of the large-scale, heterogeneous, and multi-modal Metaverse data. Differing from the conventional Federated Learning (FL) architecture, a multi-center aggregation structure to learn multiple global models based on the changes of dynamically updated local model weights, is developed in global, while a hierarchical neural network structure which includes a personalized module and a federated module to tackle both issues on data heterogeneity and model heterogeneity, is designed in local, so as to enhance the performance of PFL with unique characteristics of Metaverse data. In particular, a two-stage iterative clustering algorithm with a more precise initialization is developed to facilitate the personalized global aggregation with dynamically updated multiple aggregation centers. A personalized multi-modal fusion network is constructed to greatly reduce the computational cost and feature dimensions from the high-dimensional heterogeneous inputs for more efficient cross-modal fusion, based on a hierarchical shift-window attention mechanism and a newly designed bridge attention mechanism. A MCL scheme is then incorporated to speed up the model convergence with less communication overload between the local federated module and global model, while an embedding layer which effectively enables the delivered global model to better adapt to the local personality in each client is further integrated. Compared with five baseline methods, experiment and evaluation results based on two different real-world datasets demonstrate the excellent performance of our proposed PFL-MCL model in a fine-grain personalized training strategy, toward more efficient communication and networking among human-centric Metaverse enabled smart applications.

The high-speed mobile networks offer great potentials to many future intelligent applications, such as autonomous vehicles in smart transportation systems. Such networks provide the possibility to interconnect mobile devices to achieve fast knowledge sharing for efficient collaborative learning and operations, especially with the help of distributed machine learning, e.g., Federated Learning (FL), and modern digital technologies, e.g., Digital Twin (DT) systems. Typically, FL requires a fixed group of participants that have Independent and Identically Distributed (IID) data for accurate and stable model training, which is highly unlikely in real-world mobile network scenarios. In this paper, in order to facilitate the lightweight model training and real-time processing in high-speed mobile networks, we design and introduce an end-edge-cloud structured three-layer Federated Reinforcement Learning (FRL) framework, incorporated with an edge-cloud structured DT system. A dual-Reinforcement Learning (dual-RL) scheme is devised to support optimizations of client node selection and global aggregation frequency during FL via a cooperative decision-making strategy, which is assisted by a two-layer DT system deployed in the edge-cloud for real-time monitoring of mobile devices and environment changes. A model pruning and federated bidirectional distillation (Bi-distillation) mechanism is then developed locally for the lightweight model training, while a model splitting scheme with a lightweight data augmentation mechanism is developed globally to separately optimize the aggregation weights based on a splitted neural network structure (i.e., the encoder and classifier) in a more targeted manner, which can work together to effectively reduce the overall communication cost and improve the non-IID problem. Experiment and evaluation results compared with three baseline methods using two different real-world datasets demonstrate the usefulness and outstanding performance of our proposed FRL model in communication-efficient model training and non-IID issue alleviation for high-speed mobile network scenarios.

With the development of the Internet of Things (IoT) and 5G technologies, IoT devices deployed on roads are able to collect a large amount of traffic data at any time. Road networks can be easily constructed into a graph structure with spatial-temporal features, and how to use these spatial-temporal features for dynamic traffic flow forecasting has become a heated issue. Although existing studies bring in the consideration of periodicity to deal with spatial-temporal sequence dependence, the similarity of time-varying relationships among cross-spatial nodes has not been well discussed. In this paper, we propose a Graph Attention Network with Spatial-Temporal Clustering (GAT-STC), which considers the so-called recent-aware features and periodic-aware features, to improve the Graph Neural Network (GNN)-based traffic flow forecasting in Intelligent Transportation System (ITS). Specifically, for the recent-aware feature extraction, a distance-based Graph Attention Network (GAT) is improved and constructed to better utilize the hidden features of neighbor nodes within a reliable distance during the recent time interval, thus can effectively capture the dynamic changes in spatial feature representation. For the periodic-aware feature extraction, a spatial-temporal clustering algorithm, in which both features in terms of nodes’ current traffic states and similar trends in terms of their dynamic changes are taken into account, is developed and applied to achieve better learning efficiency. Experiments using three public traffic datasets demonstrate the higher accuracy and better efficiency of our proposed model for traffic flow forecasting, compared with five baseline methods in ITS.