Combining system knowledge with deep learning for fault diagnosis in industrial applications offers the potential to reduce the dependency of deep learning algorithms on extensive labeled datasets. However, existing methods often rely on highly specialized, problem-specific knowledge or demand detailed physical insights into the system, which limits their generalizability. Additionally, inconsistencies in knowledge representation hinder the ability to compare and build upon prior approaches. In this work, we address these challenges by leveraging commonly available knowledge about the phase structure of systems and the hierarchical organization of condition spaces. This information is systematically represented using knowledge graphs (KGs) based on the Resource Description Framework (RDF). To integrate this knowledge into deep learning, we transform the input data and the corresponding labels based on the KGs, and employ a graph neural network (GNN) trained with a semantic loss function informed by the knowledge about the condition space. The proposed approach is evaluated on three diverse datasets with varying characteristics under the two scenarios of domain generalization and novel fault detection.

Accurate uncertainty quantification is critical for robust and trustworthy predictions in many real-world applications. Bayesian Neural Networks (BNNs) provide a principled approach for modeling uncertainty but are often limited by the computational complexity of Bayesian inference. In this paper, we introduce a statistical linearization approach for multilayer feedforward BNNs. We demonstrate that this statistical linearization is equivalent to the Kalman Bayesian Neural Networks (KBNN) framework. This equivalence unifies these methodologies, providing a theoretical foundation for understanding the relationship between different BNN training approaches.

To gain deeper insights into a complex sensor system through the lens of causality, we present common and individual causal mechanism estimation (CICME), a novel threestep approach to inferring causal mechanisms from heterogeneous data collected across multiple domains. By leveraging the principle of Causal Transfer Learning (CTL), CICME is able to reliably detect domain-invariant causal mechanisms when provided with sufficient samples. The identified common causal mechanisms are further used to guide the estimation of the remaining causal mechanisms in each domain individually. The performance of CICME is evaluated on linear Gaussian models under scenarios inspired from a manufacturing process. Building upon existing continuous optimization-based causal discovery methods, we show that CICME leverages the benefits of applying causal discovery on the pooled data and repeatedly on data from individual domains, and it even outperforms both baseline methods under certain scenarios.

Many works have been devoted to evaluating the robustness of a classifier in the neighborhood of single points of input data. Recently, in particular, probabilistic settings have been considered, where robustness is defined in terms of random perturbations of input data. In this paper, we consider robustness on the entire input domain as opposed to single points of input. For the first time, we provide formal guarantees on the probability of robustness, given a random input and a random perturbation, based only on sampling or in combination with existing pointwise methods. We prove that the error becomes arbitrarily small for enough input data. This is applicable to any classification or regression model and any random input perturbation. We then illustrate the resulting bounds and compare them against the state of the art for models trained on the MNIST, California Housing, and ImageNet datasets.

Bayesian Neural Networks (BNNs) offer a sophisticated framework for extending classical neural network point estimates to encompass predictive distributions. Despite the high potential of BNNs, established BNN training methods such as Variational Inference (VI) and Markov Chain Monte Carlo (MCMC) grapple with issues such as scalability and hyperparameter dependence. In addressing these issues, our research focuses on the fundamental elements of BNNs, in particular perceptrons and their predictive capabilities. We introduce a new perspective on the closed-form solution for backward-pass computation for the Bayesian perceptron and prove that the state-of-the-art solution is equivalent to statistical linearization. To assess the efficacy of Bayesian perceptrons and provide insights into their performance in distinct input space regions, a novel methodology utilizing k-d trees as a space partitioning method is introduced to evaluate prediction quality within specific input space regions.

Data fusion deals with combining information from multiple sensors to support decision making. In such settings, machine learning methods, that principally only take correlation into account, have been applied widely due to their strong predictive and computational capabilities. In this paper, we investigate potential benefits of introducing causal knowledge in machine learning-based data fusion to address two common downstream tasks, namely, quality prediction and root cause analysis (RCA). To resemble the complex relationships typically associated with sensor data, we create simulation data with explicit modeling of latent confounding. The results of this study indicate that taking into account true causal knowledge significantly improves the performance of RCA, and leads to prediction models that are more robust to severe distribution shifts in the presence of latent confounding. Furthermore, if causal knowledge needs to be inferred from observational data using existing causal discovery methods, we propose a selection criterion to choose the best causal structure. We show that given a sufficient amount of data, the selected causal structure can be used as reliable input to solve the downstream tasks.

In a surveillance environment with high clutter, finding the correct measurement to track associations becomes extremely important for efficient target tracking. This study offers a novel algorithm to retrodict the data association probabilities at any past time instant, when the batch set of measurements is kept in memory. For the retrodiction procedure, the batch association cost is first written explicitly as a binary integer optimization problem with a quadratic cost function and it is shown that the relaxed form of the problem is convex. From the relaxed problem, a lower bound for the optimal association cost is derived, and this lower bound is used as the data association probabilities pertaining to that selected time instant in the past. Due to its consideration of the batch set of data in a retrospective manner, we will call this algorithm as Retrodictive Probabilistic Data Association, RPDA. For simplification of the mathematical analysis, a single point target with no missing measurements, i.e. PD= 1, is taken into account.

In nonlinear Bayesian estimation it is generally inevitable to incorporate approximate descriptions of the exact estimation algorithm. There are two possible ways to involve approximations: Approximating the nonlinear stochastic system model or approximating the prior probability density function. The key idea of the introduced novel estimator called Hybrid Density Filter relies on approximating the nonlinear system, thus approximating conditional densities. These densities nonlinearly relate the current system state to the future system state at predictions or to potential measurements at measurement updates. A hybrid density consisting of both Dirac delta functions and Gaussian densities is used for an optimal approximation. This paper addresses the optimization problem for treating the conditional density approximation. Furthermore, efficient estimation algorithms are derived based upon the special structure of the hybrid density, which yield a Gaussian mixture representation of the system state's density.

This paper addresses the problem of model-based reconstruction and parameter identification of distributed phenomena characterized by partial differential equations. The novelty of the proposed method is the systematic approach and the integrated treatment of uncertainties, which naturally occur in the physical system and arise from noisy measurements. The main challenge of accurate reconstruction is that model parameters, i.e., diffusion coefficients, of the physical model are not known in advance and usually need to be identified. Generally, the problem of parameter identification leads to a nonlinear estimation problem. Hence, a novel efficient recursive procedure is employed. Unlike other estimators, the so-called Hybrid Density Filter not only assures accurate estimation results for nonlinear systems, but also offers an efficient processing. By this means it is possible to reconstruct and identify distributed phenomena monitored by autonomous wireless sensor networks. The performance of the proposed estimation method is demonstrated by means of simulations.