Anomaly detection in multivariate time series (MTS) from sensor data is critical in many industrial applications. The challenge lies in managing massive unlabeled datasets with complex spatio-temporal correlations, diverse anomalies, and noise. While several unsupervised methods have been proposed, they are often limited to specific applications. In this paper, we introduce a probabilistic self-supervised framework, Autoregressive Density Estimation Transformer (ADET). ADET integrates an efficient transformer for learning spatio-temporal representations with density estimation networks for multi-score anomaly detection, focusing on point-to-point, point-to-distribution, and distribution-to-distribution distances. ADET improves noise resilience using optimal truncated singular value decomposition (OT-SVD) in an end-to-end optimization process. We conducted experiments by employing several encoders and performed an ablation study to examine the effect of OT-SVD.

• Model agnostic XAI approach for time series forecasting based on score aggregation. • Visual explanations of temporal patterns learned by Transformers and LTSF-Linear. • Analysis over four diverse real-world datasets and two models of each family. • Transformers focus on the recent past or learn wrong and inaccurate patterns. • NLinear and DLinear learn the right periodicities even in partial look-back windows. The surge of Transformers’ popularity has motivated their usage in a multitude of domains with sequential data, as can be seen in time series forecasting. It is assumed that Transformers can easily handle time dependencies, encouraged by their ability to handle contextual information from texts. However, this hypothesis has been challenged by the uncovering of performance issues that arise from comparing Transformers with simpler models, such as the autoregressive LTSF-Linear. Moreover, no attempts have been made to extract interpretable knowledge on the patterns learned by these models that can help understand why they underperform in time series forecasting. This study proposes ChronoSHAP, an explainable AI approach based on Shapley additive explanations (SHAP) combined with a novel aggregation procedure along the temporal dimension. ChronoSHAP aims to provide visual explanations of the global temporal dependencies in time series forecasting, using them to reveal why Transformers are underperforming. ChronoSHAP is used with two Transformers and two LTSF-Linear models across a variety of datasets in a multivariate, multi-step prediction setting. The explanations are analyzed to extract interpretable knowledge about temporal patterns compared to those learned by simpler models with better performance, and those observed in the data. The results show that Transformers are prone to focus on the recent past, while tending to disregard or misunderstand the long-term dependencies observed in the data, while the simpler LTSF-Linear models learn clearer long-term dependencies more consistently, which explains why Transformers struggle to keep up with simpler models’ performance.

Anomaly detection in multivariate time series (MTS) is crucial in domains such as industrial monitoring, cybersecurity, healthcare, and autonomous driving. Deep learning approaches have improved anomaly detection but lack interpretability. We propose an explainable anomaly detection (XAD) framework using a sparse non-linear vector autoregressive network (SNL-VAR-Net). This framework combines neural networks with vector autoregression for non-linear representation learning and interpretable models. We employ regularization to enforce sparsity, enabling efficient handling of long-range dependencies. Additionally, augmented Lagrange multiplier-based techniques for low-rank and sparse decomposition reduce the impact of noise. Evaluation on publicly available datasets shows that SNL-VAR-Net offers comparable performance to deep learning methods with better interpretability.

This article presents an algorithmic framework for real-time condition monitoring and state forecasting using multivariate data demonstrated on thermal imagery data of a ship's engine. The proposed method aims to improve the accuracy, efficiency, and robustness of condition monitoring and state predictions by identifying the most informative sampling locations of high-dimensional datasets and extracting the underlying dynamics of the system. The method is based on a combination of Proper Orthogonal Decomposition (POD), Optimal Sampling Location (OSL), and Dynamic Mode Decomposition (DMD), allowing the identification of key features in the system's behavior and predicting future states. Based on thermal imagery data, it is shown how thermal areas of interest can be classified via POD. By extracting the POD modes of the data, dimensions can be drastically reduced and via OSL, optimal sampling locations are found. In addition, nonlinear kernel-based Support Vector Regression (SVR) is used to build models between the optimal locations, enabling the imputation of erroneous data to improve the overall robustness. To build predictive data-driven models, DMD is applied on the subspace obtained by OSL, which leads to an intensive lower demand of computational resources, making the proposed method real-time applicable. Furthermore, an unsupervised approach for anomaly detection is proposed using OSL. The anomaly detection framework is coupled with the state prediction framework, which extends the capabilities to real-time anomaly predictions. In summary, this study proposes a robust predictive condition monitoring framework for real-time risk assessment.

Modeling complex physical processes such as the extraction of aluminum is mainly done using pure physics-based models derived from first principles. However, the accuracy of these models can often suffer due to a partial understanding of the process, uncertainty in the input parameters, and numerous modeling assumptions. More recently, with the ever-increasing availability of data, there has been an explosion of interest in applying modern machine learning methods because of their ability to learn complex mappings directly from data. Unfortunately, these models tend to be black boxes, require an enormous amount of data, and do not utilize existing domain knowledge. In this work, we develop a novel approach combining physics-based and data-driven modeling approaches while eliminating some weaknesses. We use a data-driven model to correct a misspecified physics-based model of the Hall–Héroult process in an aluminum electrolysis cell using a corrective source term added to the set of governing ordinary differential equations. Our approach ensures that the existing knowledge is utilized to the maximum extent possible while relying on the data-driven models only to model those aspects which the physics-based model does not represent well. We compare this approach with an end-to-end learning approach and an ablated physics-based model, showing that the proposed hybrid method is more accurate, consistent, and stable for long-term predictions.

Autonomous systems are becoming ubiquitous and gaining momentum within the marine sector. Since the electrification of transport is happening simultaneously, autonomous marine vessels can reduce environmental impact, lower costs, and increase efficiency. Although close monitoring is still required to ensure safety, the ultimate goal is full autonomy. One major milestone is to develop a control system that is versatile enough to handle any weather and encounter that is also robust and reliable. Additionally, the control system must adhere to the International Regulations for Preventing Collisions at Sea (COLREGs) for successful interaction with human sailors. Since the COLREGs were written for the human mind to interpret, they are written in ambiguous prose and therefore not machine-readable or verifiable. Due to these challenges and the wide variety of situations to be tackled, classical model-based approaches prove complicated to implement and computationally heavy. Within machine learning (ML), deep reinforcement learning (DRL) has shown great potential for a wide range of applications. The model-free and self-learning properties of DRL make it a promising candidate for autonomous vessels. In this work, a subset of the COLREGs is incorporated into a DRL-based path following and obstacle avoidance system using collision risk theory. The resulting autonomous agent dynamically interpolates between path following and COLREG-compliant collision avoidance in the training scenario, isolated encounter situations, and AIS-based simulations of real-world scenarios.

The success of the current wave of artificial intelligence can be partly attributed to deep neural networks, which have proven to be very effective in learning complex patterns from large datasets with minimal human intervention. However, it is difficult to train these models on complex dynamical systems from data alone due to their low data efficiency and sensitivity to hyperparameters and initialisation. This work demonstrates that injection of partially known information at an intermediate layer in a DNN can improve model accuracy, reduce model uncertainty, and yield improved convergence during the training. The value of these physics-guided neural networks has been demonstrated by learning the dynamics of a wide variety of nonlinear dynamical systems represented by five well-known equations in nonlinear systems theory: the Lotka-Volterra, Duffing, Van der Pol, Lorenz, and Henon-Heiles systems.