Abstract Today, the use of Artificial Intelligence (AI) is rapidly increasing in many areas of society. While model performance on various tasks continue to impress, it does so at the cost of increased model complexity, such that most state-of-the-art AI models are effectively black boxes. Where human-made decisions typically are accompanied by human-understandable explanations detailing the reasoning behind the decision, incorporating advanced AI as part of a decision-making process reduces the transparency of that process significantly. Yet, the ability to explain decisions is essential for there to be understanding and trust. As a response to this, Explainable Artificial Intelligence (XAI) has emerged as a field that aims to provide explanations of model behaviour. Methods categorised as post-hoc are designed to generate explanations for black box models after training, at no cost to model performance. In parallel with this, extensive work has been done in the field of causality to formalise the structure of human-understandable, causal explanations. This work presents a comprehensive literature review of the current state of the subfield of XAI that consist of causality-motivated post-hoc XAI methods. In order to clearly define causal XAI, a causal framework for categorising XAI is introduced, and three types of post-hoc XAI methods are identified: observational methods, internally causal methods and externally causal methods. Finally, externally causal XAI is argued a promising direction for reliable and understandable post-hoc XAI, with the ability to generate counterfactual explanations using a meaningful vocabulary, in line with the definition of counterfactual used in causal theory.

Structural causal models are a powerful framework for causal and counterfactual inference, extending the capabilities of traditional Bayesian networks. These models comprise endogenous and exogenous variables, where the exogenous variables frequently lack clear semantic interpretation. Exogenous variables are typically unobservable, rendering certain counterfactual queries unidentifiable. In such cases, standard inference algorithms for Bayesian networks are insufficient. Recent methods attempt to bound unidentifiable queries through imprecise estimation of exogenous probabilities. However, these methods become computationally infeasible as the cardinality of the exogenous variables increases, thereby constraining the complexity of applicable models. In this paper we study a divide-and-conquer approach that decomposes a general causal model into a set of submodels with low-cardinality exogenous variables, enabling exact calculation of any query within these submodels. By aggregating results from the submodels, efficient approximations of bounds for queries in the original model are obtained. Our proposal is able to handle models with variables of any cardinality assuming that there are no unobserved confounders. We show that the method is theoretically robust, and experimental results demonstrate that it achieves more accurate bounds with lower computational costs compared to existing techniques.

Abstract Deep reinforcement learning policies perform exceptionally well in applications like Atari games, chess, Go, and poker. However, they are incomprehensible, making the process of extracting new knowledge and understanding policy behavior difficult. For the same reason, deploying these policies in high-stakes applications like healthcare, finance, and criminal justice is infeasible. To rectify the incomprehensibility issue, we propose a new concept-based policy distillation method for convolutional neural network-based policies. Our method transforms raw image states into human-interpretable concepts using non-negative matrix factorization on the policy’s activations. The concepts express features in an interpretable way and detail how the policy represents the world internally. We use the concepts to train a distilled policy represented using sparse linear models. The distilled policy chooses one linear model from a set of linear models to make action predictions. Employing a single sparse linear model reduces the complexity, making it easier for humans to understand policy behavior. Experimentally, we show the effectiveness of our distilled policy in four environments: Car Racing, Pong, Breakout, and Ms Pacman. We illustrate that inspecting these linear models gives local and global insight into how the black box policy works. Furthermore, we demonstrate that these linear models perform well by faithfully using the same features as the black box policy and capturing the black box policy’s behavior in critical states. The code, data, trained models, and TensorBoard logs with hyperparameters used are provided ( https://github.com/observer4599/interpretable-concept-based-policy-distillation ).

Detecting anomalous events in time series data, ranging from manufacturing processes to health care monitoring, is important. The problem of uncertainty in real-world datasets and its anomalies makes it challenging to validate and compare results between algorithms. With this in mind, we present two benchmark suites to fill gaps in today’s landscape of datasets for anomaly detection in multivariate time series data. Here, one suite focuses only on fully synthetic sequences to provide a playground for testing algorithms with complete knowledge of the sequences. The second suite bridges between fully synthetic and real-world sequences. It provides a few extensive sequences with close-to-reality complexity but synthetic injected anomalies. The paper provides a detailed overview of the suites content and complexities. It further includes a concise overview and showcases the strengths and weaknesses of 34 algorithms in the evaluation. The benchmark suites highlight issues regarding algorithms and metrics and should support new research directions.

An emerging definition of fairness in machine learning requires that models are oblivious to demographic user information, e.g., a user’s gender or age should not influence the model. Personalized recommender systems are particularly prone to violating this definition through their explicit user focus and user modelling. Explicit user modelling is also an aspect that makes many recommender systems incapable of providing hitherto unseen users with recommendations. We propose novel approaches for mitigating discrimination in Variational Autoencoder-based recommender systems by limiting the encoding of demographic information. The approaches are capable of, and evaluated on, providing users that are not represented in the training data with fair recommendations.

Prediction intervals are a machine- and human-interpretable way to represent predictive uncertainty in a regression analysis. In this paper, we present a method for generating prediction intervals along with point estimates from an ensemble of neural networks. We propose a multi-objective loss function fusing quality measures related to prediction intervals and point estimates, and a penalty function, which enforces semantic integrity of the results and stabilizes the training process of the neural networks. The ensembled prediction intervals are aggregated as a split normal mixture accounting for possible multimodality and asymmetricity of the posterior predictive distribution, and resulting in prediction intervals that capture aleatoric and epistemic uncertainty. Our results show that both our quality-driven loss function and our aggregation method contribute to well-calibrated prediction intervals and point estimates.

In this paper, we present a method for exploratory data analysis of streaming data based on probabilistic graphical models (latent variable models). This method is illustrated by concept drift tracking, using financial client data from a European regional bank. For this particular setting, the anal yzed data spans the period from April 2007 to March 2014 and therefore starts before the beginning of the financial crisis of 2008. The implied changes in the economic climate during this period manifests itself as concept drift in the underlying data generating distribution. We explore and analyze this financial client data using a probabilistic graphical modeling framework that provides an explicit representation of concept drift as an integral part of the model. We show how learning these types of models from data provides additional insight into the hidden mechanisms governing the drift in the domain. We present an iterative approach for identifying disparate factors that jointly account for the drift in the domain. This includes a semantic characterization of one of the main influencing drift factors. Based on the experiences and results obtained from analyzing the financial data, we discuss the applicability of the framework within a more general context.

Keeping the electricity production in balance with the actual demand is becoming a difficult and expensive task in spite of an involvement of experienced human operators. This is due to the increasing complexity of the electric power grid system with the intermittent renewable production as one of the contributors. A beforehand information about an occurring imbalance can help the transmission system operator to adjust the production plans, and thus ensure a high security of supply by reducing the use of costly balancing reserves, and consequently reduce undesirable fluctuations of the 50 Hz power system frequency. In this paper, we introduce the relatively new problem of an intra-hour imbalance forecasting for the transmission system operator (TSO). We focus on the use case of the Norwegian TSO, Statnett. We present a complementary imbalance forecasting tool that is able to support the TSO in determining the trend of future imbalances, and show the potential to proactively alleviate imbalances with a higher accuracy compared to the contemporary solution.

Deep neural networks, including recurrent networks, have been successfully applied to human activity recognition. Unfortunately, the final representation learned by recurrent networks might encode some noise (irrelevant signal components, unimportant sensor modalities, etc.). Besides, it is difficult to interpret the recurrent networks to gain insight into the models' behavior. To address these issues, we propose two attention models for human activity recognition: temporal attention and sensor attention. These two mechanisms adaptively focus on important signals and sensor modalities. To further improve the understandability and mean Fl score, we add continuity constraints, considering that continuous sensor signals are more robust than discrete ones. We evaluate the approaches on three datasets and obtain state-of-the-art results. Furthermore, qualitative analysis shows that the attention learned by the models agree well with human intuition.

Hybrid Bayesian networks have received an increasing attention during the last years. The difference with respect to standard Bayesian networks is that they can host discrete and continuous variables simultaneously, which extends the applicability of the Bayesian network framework in general. However, this extra feature also comes at a cost: inference in these types of models is computationally more challenging and the underlying models and updating procedures may not even support closed-form solutions. In this paper we provide an overview of the main trends and principled approaches for performing inference in hybrid Bayesian networks. The methods covered in the paper are organized and discussed according to their methodological basis. We consider how the methods have been extended and adapted to also include (hybrid) dynamic Bayesian networks, and we end with an overview of established software systems supporting inference in these types of models.

Making inferences from data streams is a pervasive problem in many modern data analysis applications. But it requires to address the problem of continuous model updating, and adapt to changes or drifts in the underlying data generating distribution. In this paper, we approach these problems from a Bayesian perspective covering general conjugate exponential models. Our proposal makes use of non-conjugate hierarchical priors to explicitly model temporal changes of the model parameters. We also derive a novel variational inference scheme which overcomes the use of non-conjugate priors while maintaining the computational efficiency of variational methods over conjugate models. The approach is validated on three real data sets over three latent variable models.

This paper considers a parallel algorithm for Bayesian network structure learning from large data sets. The parallel algorithm is a variant of the well known PC algorithm. The PC algorithm is a constraint-based algorithm consisting of five steps where the first step is to perform a set of (conditional) independence tests while the remaining four steps relate to identifying the structure of the Bayesian network using the results of the (conditional) independence tests. In this paper, we describe a new approach to parallelization of the (conditional) independence testing as experiments illustrate that this is by far the most time consuming step. The proposed parallel PC algorithm is evaluated on data sets generated at random from five different real-world Bayesian networks. The algorithm is also compared empirically with a process-based approach where each process manages a subset of the data over all the variables on the Bayesian network. The results demonstrate that significant time performance improvements are possible using both approaches.

In this paper we propose a method for scaling up filter-based feature selection in classification problems. We use the conditional mutual information as filter measure and show how the required statistics can be computed in parallel avoiding unnecessary calculations. The distribution of the calculations between the available computing units is determined based on balanced incomplete block designs, a strategy first developed within the area of statistical design of experiments. We show the scalability of our method through a series of experiments on synthetic and real-world datasets.

The overall goal of this research is to study reasoning under uncertainty by combining Bayesian Networks and Case-Based Reasoning through constructing an experimental decision support system for classification of cancer pain. We have experimentally analysed a medical dataset in order to reveal properties of the data with respect to properties of the two reasoning methods. We also preprocessed our medical data with help from a clinical expert, which resulted in four data sets with different characteristics. This culminates in a hybrid system architecture, where CBR handles the exceptions or outliers with respect to the distribution of the data and the target class, while BN handles the more common situations. Through a set of experiments under varying conditions we show that a hybrid BN+CBR system is favorable over each single method.