The demand for disruption-free fault diagnosis of mechanical equipment under a constantly changing operation environment poses a great challenge to the deployment of data-driven diagnosis models in practice. Extant continual learning-based diagnosis models suffer from consuming a large number of labeled samples to be trained for adapting to new diagnostic tasks and failing to account for the diagnosis of heterogeneous fault types across different machines. In this paper, we use a representative mechanical equipment -rotating machinery – as an example and develop an uncertainty-aware continual learning framework (UACLF) to provide a unified interface for fault diagnosis of rotating machinery under various dynamic scenarios: class continual scenario, domain continual scenario, and both. The proposed UACLF takes a three-step to tackle fault diagnosis of rotating machinery with homogeneous-heterogeneous faults under dynamic environments. In the first step, an inter-class classification loss function and an intra-class discrimination loss function are devised to extract informative feature representations from the raw vibration signal for fault classification. Next, an uncertainty-aware pseudo labeling mechanism is developed to select unlabeled fault samples that we are able to assign pseudo labels confidently, thus expanding the training samples for faults arising in the new environment. Thirdly, an adaptive prototypical feedback mechanism is used to enhance the decision boundary of fault classification and diminish the model misclassification rate. Experimental results on three datasets suggest that the proposed UACLF outperforms several alternatives in the literature on fault diagnosis of rotating machinery across various working conditions and different machines.Note to Practitioners—This paper presents a continual fault diagnosis methodology for mechanical equipment under various working conditions across different machines with homogeneous-heterogeneous faults. On the application side, the proposed UACLF can be applied to facilitate diagnosis across a broad range of complex industrial equipment, including aerospace, automobile transmission, and wind turbines, among others. With the uncertainty-aware pseudo labeling, the proposed framework is empowered to select the samples in the new phase that we are able to reliably assign their labels. Hence, it can effectively improve mechanical equipment fault classification accuracy in the case that only a small portion of labeled fault samples is available. When training the model, given the model architecture, fault samples collected from multiple accelerometers are fed into the developed model. Four different loss functions, supervision loss, inter-class classification loss, intra-class discrimination loss, and uncertainty estimation loss, are employed to train the diagnostic model. Experiments conducted on three different laboratory datasets have demonstrated the effectiveness of the proposed framework, but have not been tested in the practical industrial applications. We will consider testing the proposed UACLF in an actual plant in future research.

Deep learning holds significant potential for bearing fault diagnosis; however, its effectiveness is often hindered by the pervasive issue of imbalanced data in industrial settings, where fault events are inherently rare. To address this widespread challenge, we propose the Class-Aware Supervised Contrastive Quadratic Neural Network (CCQNet), a novel framework combining a class-aware supervised contrastive learning scheme with a quadratic neural network backbone. Our approach introduces two key components to tackle data imbalance: a class-weighted contrastive loss and a logit adjusted cross-entropy loss, which work in tandem to ensure the model pays equal attention to both majority and minority classes. Additionally, we enhance feature extraction through a quadratic convolutional residual network, and provide a novel theoretical analysis linking the function of the quadratic neuron to the principle of autocorrelation in signal processing. Comprehensive experiments on both public and proprietary datasets demonstrate that CCQNet substantially outperforms state-of-the-art methods, particularly in scenarios with extreme data imbalance. The source code is publicly available at https://github.com/yuweien1220/CCQNet for evaluation and validation.

Neural networks that overlook the underlying causal relationships among observed variables pose significant risks in high-stakes decision-making contexts due to concerns about the robustness and stability of model performance. To tackle this issue, we present a general approach for embedding hierarchical causal structure among observed variables into a neural network to inform its learning. The proposed methodology, termed causality-informed neural network (CINN), exploits hierarchical causal structure learned from observational data as a structurally informed prior to guide the layer-to-layer architectural design of the neural network while maintaining the orientation of causal relationships in the discovered causal graph. The proposed method involves three steps. First, CINN mines causal relationships from observational data via directed acyclic graph (DAG) learning, where causal discovery is recast as a continuous optimization problem to circumvent the combinatorial nature of DAG learning. Second, we encode the discovered hierarchical causal graph among observed variables into a neural network via a dedicated architecture and loss function. By classifying observed variables in the DAG as root, intermediate, and leaf nodes, we translate the hierarchical causal DAG into CINN by creating a one-to-one correspondence between DAG nodes and certain CINN neurons. For the loss function, both intermediate and leaf nodes in the DAG are treated as target outputs during CINN training, facilitating the co-learning of causal relationships among the observed variables. Finally, as multiple loss components emerge in CINN, we leverage the projection of conflicting gradients (PCGrads) to mitigate the gradient interference among the multiple learning tasks. Computational studies indicate that CINN outperforms several state-of-the-art methods across a broad range of datasets. In addition, an ablation study that incrementally incorporates structural and quantitative causal knowledge into the neural network is conducted to highlight the pivotal role of causal knowledge in enhancing neural network’s prediction performance.

The development of time-based flow management has significantly enhanced the safety, reliability, and predictability of air traffic control (ATC). Actual flight paths often deviate from these standard terminal arrival routes due to pilots requesting shortcut arrivals or ATC officers implementing holding procedures to alleviate congestion. These deviations exacerbate the dynamic complexity of air traffic management (ATM). To address these challenges, we propose a novel online learning Transformer-bidirectional gated recurrent unit (Transformer-BiGRU) framework for tactical spatiotemporal flight trajectory prediction. BiGRU further obtains bidirectional sequence information to improve the Transformer’s spatiotemporal prediction. The proposed research utilises image processing techniques to produce ATC aeronautical holding instructions from historical automatic dependent surveillance-broadcast data. The framework significantly improves real-time prediction ability and environment adaptability by integrating holding instructions and online learning. Experiment results demonstrate that incorporating holding instructions with the proposed Transformer-BiGRU reduces the mean absolute error by approximately 10% in latitude, 8.9% in longitude, and 13.1% in flight level compared to the best baseline model. Furthermore, the mean deviation error of horizontal distance decreases from 0.49 to 0.42 nautical miles (a 13% improvement). These results confirm that the methodology benefits real-time ATC decision-making in various ATM scenarios and provides valuable insights to assure airspace safety.

The robustness of recommender systems under node injection attacks has garnered significant attention. Recently, GraphRfi, a Graph-Neural-Network-based (GNN-based) recommender system, was proposed and shown to effectively mitigate the impact of injected fake users. However, we demonstrate that GraphRfi remains vulnerable to attacks due to the supervised nature of its fraudster detection component, where obtaining clean labels is challenging in practice. In particular, we propose a powerful poisoning attack, MetaC, against both GNN-based and Martix-Faxtorization-based recommender systems. Furthermore, we analyze why GraphRfi fails under such an attack. Then, based on our insights obtained from vulnerability analysis, we design an adaptive fraudster detection module that explicitly considers label uncertainty. This module can serve as a plug-in for different recommender systems, resulting in a robust framework named Posterior-Detection Recommender (PDR). Comprehensive experiments show that our defense approach outperforms other benchmark methods under attacks. Overall, our research presents an effective framework for integrating fraudster detection into recommendation systems to achieve adversarial robustness.

Signed social networks are widely used to model the trust relationships among online users in security-sensitive systems such as cryptocurrency trading platforms, where trust prediction plays a critical role. In this paper, we investigate how attackers could mislead trust prediction by secretly manipulating signed networks. To this end, we first design effective poisoning attacks against representative trust prediction models. The attacks are formulated as hard bi-level optimization problems, for which we propose several efficient approximation solutions. However, the resultingbasic attackswould severely change the structural semantics (in particular, both local and global balance properties) of a signed network, which makes the attacks prone to be detected by the powerful attack detectors we designed. Given this, we further refine the basic attacks by integrating someconflicting metricsas penalty terms into the objective function. Therefined attacksbecome secrecy-aware, i.e., they can successfully evade attack detectors with high probability while sacrificing little attack performance. We conduct comprehensive experiments to demonstrate that the basic attacks can severely disrupt trust prediction but could be easily detected, and the refined attacks perform almost equally well while evading detection. Overall, our results significantly advance the knowledge in designing more practical attacks, reflecting more realistic threats to current trust prediction models. Moreover, the results also provide valuable insights and guidance for building up robust trust prediction systems.

Accurate and reliable prediction of bearing remaining useful life (RUL) is crucial to the prognostics and health management of rotation machinery. Despite the rapid progress of data-driven methods, the generalizability of data-driven models remains an open issue to be addressed. In this article, we tackle this challenge by resolving the feature misalignment problem that arises in extracting features from the raw vibration signals. Toward this goal, we introduce a logarithmic cumulative transformation (LCT) operator consisting of cumulative, logarithmic, and another cumulative transformation for feature extraction. In addition, we propose a novel method to estimate the reliability associated with each RUL prediction by integrating a linear regression model and an auxiliary exponential model. The linear regression model rectifies bias from neural network's point predictions while the auxiliary exponential model fits the differential slopes of the linear models and generates the upper and lower bounds for building the reliability indicator. The proposed approach comprised of LCT, an attention GRU-based encoder–decoder network, and reliability evaluation is validated on the FEMETO-ST dataset. Computational results demonstrate the superior performance of the proposed approach several other state-of-the-art methods.

As safety is a top priority in mission-critical engineering applications, uncertainty quantification emerges as a linchpin to the successful deployment of AI models in these high-stakes domains. In this article, we seamlessly encode a simple and principled uncertainty quantification module spectral-normalized neural Gaussian process (SNGP) into GoogLeNet to detect various defects in steel wire ropes (SWRs) accurately and reliably. To this end, the developed methodology consists of three coherent steps. In the first step, raw magnetic flux leakage (MFL) signals in waveform associated with normal and defective SWRs that are manifested in the number of broken wires are collected via a dedicated experimental setup. Next, the proposed approach utilizes Gramian angular field to transform the MFL signal in 1-D time series into 2-D images while preserving key spatial and temporal structures in the data. Third, built atop the backbone of GoogLeNet, we systematically integrate SNGP by adding the spectral normalization (SN) layer to normalize the weights and replacing the output layers with a Gaussian process (GP) in the main network and auxiliary classifiers of GoogLeNet accordingly, where SN enables to preserve the distance in data transformation and GP makes the output layer of neural network distance aware when assigning uncertainty. Comprehensive comparisons with the state-of-the-art models highlight the advantages of the developed methodology in classifying SWR defects and identifying out-of-distribution (OOD) SWR instances. In addition, a thorough ablation study is performed to quantitatively illustrate the significant role played by SN and GP in the principledness of the estimated uncertainty toward detecting SWR instances with varying OODness.

Measuring the uncertainty of information is a crucial problem in many fields. Recent studies have found a new uncertainty measure for probabilities called “extropy” as a complementary dual function of classical Shannon entropy. In this article, the extropy measure of randomness is generalized to the case of information with epistemic uncertainty by means of a framework of Dempster-Shafer evidence theory. Specifically, a novel measure called plausibility extropy is proposed, which inherits the intriguing properties of original extropy. Moreover, the duality and complementarity between the proposed plausibility extropy and existing plausibility entropy are proved strictly, which constitutes an entropy-extropy combination for mass functions to measure the epistemic uncertainty. In addition, the maximum plausibility extropy is also studied in this article. Through comparing with existing extropy-like measures in Dempster-Shafer evidence theory, the rationality of proposed plausibility extropy is further demonstrated.

Graph neural networks (GNNs) have transformed network analysis, leading to state-of-the-art performance across a variety of tasks. Especially, GNNs are increasingly been employed as detection tools in the AIoT environment in various security applications. However, GNNs have also been shown vulnerable to adversarial graph perturbation. We present the first approach for certifying robustness of general GNNs against attacks that add or remove graph edges either at training or prediction time. Extensive experiments demonstrate that our approach significantly outperforms prior art in certified robust predictions. In addition, we show that a noncertified adaptation of our method exhibits significantly better robust accuracy against state-of-the-art attacks that past approaches. Thus, we achieve both the best certified bounds and best practical robustness of GNNs to structural attacks to date.

Landing is generally cited as one of the riskiest phases of a flight, as indicated by the much higher accident rate than other flight phases. In this paper, we focus on the hard landing problem (which is defined as the touchdown vertical speed exceeding a predefined threshold), and build a probabilistic predictive model to forecast the aircraft’s vertical speed at touchdown, using DASHlink data. Previous work has treated hard landing as a classification problem, where the vertical speed is represented as a categorical variable based on a predefined threshold. In this paper, we build a machine learning model to numerically predict the touchdown vertical speed during aircraft landing. Probabilistic forecasting is used to quantify the uncertainty in model prediction, which in turn supports risk-informed decision-making. A Bayesian neural network approach is leveraged to construct the predictive model. The overall methodology consists of five steps. First, a clustering method based on the minimum separation between different airports is developed to identify flights in the dataset that landed at the same airport. Secondly, identifying the touchdown point itself is not straightforward; in this paper, it is determined by comparing the vertical speed distributions derived from different candidate touchdown indicators. Thirdly, a forward and backward filtering (filtfilt) approach is used to smooth the data without introducing phase lag. Next, a minimal-redundancy-maximal-relevance (mRMR) analysis is used to reduce the dimensionality of input variables. Finally, a Bayesian recurrent neural network is trained to predict the touchdown vertical speed and quantify the uncertainty in the prediction. The model is validated using several flights in the test dataset, and computational results demonstrate the satisfactory performance of the proposed approach.

Resilience is an important capability for many complex systems to mitigate the impact of extreme events as well as timely restoration of system performance in the aftermath of a disruptive event. In this article, we investigate a bilevel predisaster resilience-based design optimization approach for the configuration of logistics service centers. In the bilevel program, the upper level model considers the impact of potential disruptive events, and characterizes system planners’ decision regarding possible service center configuration that consists of two decision variables—construction of service center at candidate sites and their specific capacities. The optimization in the upper level model considers both the travel time of each customer from their origins to the service centers and the within-center service time including average waiting time in the queue and mean processing time. The lower level model captures customers’ behavior in choosing the distribution center to fulfill their requests with the goal of minimizing the cumulative travel time for all the customers. The objective of the formulated bilevel program is to maximize the resilience of the service center configuration, thereby increasing the ability of the system to withstand unexpected events. To tackle this NP-hard optimization problem, an adaptive importance sampling approach—cross-entropy-based method—is leveraged to generate samples that gradually concentrates all its mass in the proximity of the optimal solution in an iterative way. A numerical example is used to illustrate the procedures of the developed method and demonstrate the effectiveness of the proposed methodology.

As a novel computational paradigm, Physarum solver has received increasing attention from the researchers in tackling a plethora of network optimization problems. However, the convergence of Physarum solver is grounded by solving a system of linear equations iteratively, which often leads to low computational performance. Two factors have been highlighted along the process: 1) high time complexity in solving the system of linear equations and 2) extensive iterations required for convergence. Thus, Physarum solver has been largely restricted by its unsatisfactory computational performance. In this paper, we aim to address these two issues by developing two enhancement strategies: 1) pruning inactive nodes and 2) terminating Physarum solver in advance. First, extensive nodes and edges become and stay inactive after a few iterations in identifying the shortest path. Removing these inactive nodes and edges significantly decreases the graph size, thereby reducing computational complexity. Second, we define a transition phase for edges. All of the paths experiencing such a transition phase are dynamically aggregated to form a set of near-optimal paths among which the optimal path is included. Depth-first search is then leveraged to identify the optimal path from the near-optimal paths set. Earlier termination of Physarum solver saves considerable iterations while guaranteeing the optimality of the found solution. Empirically, 20 randomly generated sparse and complete graphs with network sizes ranging from 50 to 2000 as well as two real-world traffic networks are used to compare the performance of accelerated Physarum solver to the other two state-of-the-art algorithms.

Finding an equilibrium state of the traffic assignment plays a significant role in the design of transportation networks. We adapt the path finding mathematical model of slime mold Physarum polycephalum to solve the traffic equilibrium assignment problem. We make three contributions in this paper. First, we propose a generalized Physarum model to solve the shortest path problem in directed and asymmetric graphs. Second, we extend it further to resolve the network design problem with multiple source nodes and sink nodes. At last, we demonstrate that the Physarum solver converges to the user-optimized (Wardrop) equilibrium by dynamically updating the costs of links in the network. In addition, convergence of the developed algorithm is proved. Numerical examples are used to demonstrate the efficiency of the proposed algorithm. The superiority of the proposed algorithm is demonstrated in comparison with several other algorithms, including the Frank-Wolfe algorithm, conjugate Frank-Wolfe algorithm, biconjugate Frank-Wolfe algorithm, and gradient projection algorithm.

This paper evaluates the network reliability from a game theory perspective. We formulate a network game consisting of two players-router and attacker, where the router seeks to minimize his total expected trip cost, while the attacker attempts to maximize the expected trip cost by undermining some of the network links. Each link has a probabilistic cost in accordance with its state (normal or damaged). Two different scenarios are considered: link cost independent of the flow and link cost dependent on the flow. We are interested in the link use and damage probabilities at system equilibrium for both cases, and these probabilities are derived in a four-step procedure. First, for the router, Dijkstra and the Frank-Wolfe (FW) algorithms are used to optimize his strategy under the two scenarios, respectively. Second, we model the attacker's problem as a constrained optimization problem, in which all the decision variables are binary. A probabilistic solution discovery algorithm (PSDA) is integrated with stochastic ranking to determine the attacker's optimal strategy. Third, we leverage the Method of Successive Averages (MSA) to approximate the router's link use probabilities and attacker's link damage probabilities at the mixed Nash equilibrium of the game. Finally, given the router's probability of traveling through each link and attacker's probability of undermining each link, we use Monte Carlo Simulations (MCS) to estimate the network reliability as the router arriving the destination node within a prescribed time. Two numerical examples are used to illustrate the procedures and effectiveness of the proposed method.