Structural causal models are a powerful framework for causal and counterfactual inference, extending the capabilities of traditional Bayesian networks. These models comprise endogenous and exogenous variables, where the exogenous variables frequently lack clear semantic interpretation. Exogenous variables are typically unobservable, rendering certain counterfactual queries unidentifiable. In such cases, standard inference algorithms for Bayesian networks are insufficient. Recent methods attempt to bound unidentifiable queries through imprecise estimation of exogenous probabilities. However, these methods become computationally infeasible as the cardinality of the exogenous variables increases, thereby constraining the complexity of applicable models. In this paper we study a divide-and-conquer approach that decomposes a general causal model into a set of submodels with low-cardinality exogenous variables, enabling exact calculation of any query within these submodels. By aggregating results from the submodels, efficient approximations of bounds for queries in the original model are obtained. Our proposal is able to handle models with variables of any cardinality assuming that there are no unobserved confounders. We show that the method is theoretically robust, and experimental results demonstrate that it achieves more accurate bounds with lower computational costs compared to existing techniques.

Abstract Software organisations aim to develop and maintain high‐quality software systems. Due to large amounts of behaviour data available, software organisations can conduct data‐driven software maintenance. Indeed, software quality assurance and improvement programs have attracted many researchers' attention. Bayesian Networks (BNs) are proposed as a log analysis technique to discover poor performance indicators in a system and to explore usage patterns that usually require temporal analysis. For this, an action research study is designed and conducted to improve the software quality and the user experience of a web application using BNs as a technique to analyse software logs. To this aim, three models with BNs are created. As a result, multiple enhancement points have been identified within the application ranging from performance issues and errors to recurring user usage patterns. These enhancement points enable the creation of cards in the Scrum process of the web application, contributing to its data‐driven software maintenance. Finally, the authors consider that BNs within quality‐aware and data‐driven software maintenance have great potential as a software log analysis technique and encourage the community to deepen its possible applications. For this, the applied methodology and a replication package are shared.

In this paper, we present a method for exploratory data analysis of streaming data based on probabilistic graphical models (latent variable models). This method is illustrated by concept drift tracking, using financial client data from a European regional bank. For this particular setting, the anal yzed data spans the period from April 2007 to March 2014 and therefore starts before the beginning of the financial crisis of 2008. The implied changes in the economic climate during this period manifests itself as concept drift in the underlying data generating distribution. We explore and analyze this financial client data using a probabilistic graphical modeling framework that provides an explicit representation of concept drift as an integral part of the model. We show how learning these types of models from data provides additional insight into the hidden mechanisms governing the drift in the domain. We present an iterative approach for identifying disparate factors that jointly account for the drift in the domain. This includes a semantic characterization of one of the main influencing drift factors. Based on the experiences and results obtained from analyzing the financial data, we discuss the applicability of the framework within a more general context.

Hybrid Bayesian networks have received an increasing attention during the last years. The difference with respect to standard Bayesian networks is that they can host discrete and continuous variables simultaneously, which extends the applicability of the Bayesian network framework in general. However, this extra feature also comes at a cost: inference in these types of models is computationally more challenging and the underlying models and updating procedures may not even support closed-form solutions. In this paper we provide an overview of the main trends and principled approaches for performing inference in hybrid Bayesian networks. The methods covered in the paper are organized and discussed according to their methodological basis. We consider how the methods have been extended and adapted to also include (hybrid) dynamic Bayesian networks, and we end with an overview of established software systems supporting inference in these types of models.

Making inferences from data streams is a pervasive problem in many modern data analysis applications. But it requires to address the problem of continuous model updating, and adapt to changes or drifts in the underlying data generating distribution. In this paper, we approach these problems from a Bayesian perspective covering general conjugate exponential models. Our proposal makes use of non-conjugate hierarchical priors to explicitly model temporal changes of the model parameters. We also derive a novel variational inference scheme which overcomes the use of non-conjugate priors while maintaining the computational efficiency of variational methods over conjugate models. The approach is validated on three real data sets over three latent variable models.

This paper considers a parallel algorithm for Bayesian network structure learning from large data sets. The parallel algorithm is a variant of the well known PC algorithm. The PC algorithm is a constraint-based algorithm consisting of five steps where the first step is to perform a set of (conditional) independence tests while the remaining four steps relate to identifying the structure of the Bayesian network using the results of the (conditional) independence tests. In this paper, we describe a new approach to parallelization of the (conditional) independence testing as experiments illustrate that this is by far the most time consuming step. The proposed parallel PC algorithm is evaluated on data sets generated at random from five different real-world Bayesian networks. The algorithm is also compared empirically with a process-based approach where each process manages a subset of the data over all the variables on the Bayesian network. The results demonstrate that significant time performance improvements are possible using both approaches.

In this paper we propose a method for scaling up filter-based feature selection in classification problems. We use the conditional mutual information as filter measure and show how the required statistics can be computed in parallel avoiding unnecessary calculations. The distribution of the calculations between the available computing units is determined based on balanced incomplete block designs, a strategy first developed within the area of statistical design of experiments. We show the scalability of our method through a series of experiments on synthetic and real-world datasets.

Probabilistic graphical models constitute a fundamental tool for the development of intelligent systems.They provide a sound and well-founded approach for performing inference and belief updating in complex domains endowed with uncertainty.A probabilistic graphical model is the result of the combination of a qualitative component (a graph) encoding conditional independence relationships among the variables in the system and a quantitative component consisting of a collection of local probability distributions matching the independence properties specified by the graph.The union of the two components provides a compact representation of the joint probability distribution over the domain being modeled.Bayesian networks are the most prominent type of probabilistic graphical models and have experienced a remarkable methodological development during the past two decades.This has come along with a wide variety of successful applications in different domains.Regardless of the increasing interest in the area, probabilistic graphical models are still facing a number of challenges, covering modeling, inference and learning.This special issue contains seven papers that contribute to the methodological development beyond the current state-of-the-art knowledge.Five of the papers were selected among the contributions presented at the 15th Conference of the Spanish Association of Artificial Intelligence (CAEPIA'2013, Madrid, Spain, September 17-20, 2013).These papers were substantially extended and went through a new and strict review process.The other two papers were invited contributions not presented at the conference and went through the same review process.The first three papers deal with hybrid models, where both discrete and continuous variables coexist.Lucas and Hommersom propose a framework for handling causal independence covering discrete and continuous variables simultaneously.The methodology is based on the convolution concept from probability theory, and the

In this paper, we discuss some practical issues that arise in solving hybrid Bayesian networks that include deterministic conditionals for continuous variables. We show how exact inference can become intractable even for small networks due to the difficulty in handling deterministic conditionals (for continuous variables). We propose some strategies for carrying out the inference task using mixtures of polynomials (MOPs) and mixtures of truncated exponentials. MOPs can be defined on hypercubes or hyperrhombuses. We compare these two methods. A key strategy is to reapproximate large potentials with potentials consisting of fewer pieces and lower degrees/number of terms. We discuss several methods for reapproximating potentials. We illustrate our methods in a practical application consisting of solving a stochastic program evaluation and review technique (PERT) network.

Recursive probability trees (RPTs) are a data structure for representing several types of potentials involved in probabilistic graphical models. The RPT structure improves the modeling capabilities of previous structures (like probability trees or conditional probability tables). These capabilities can be exploited to gain savings in memory space and/or computation time during inference. This paper describes the modeling capabilities of RPTs as well as how the basic operations required for making inference on Bayesian networks operate on them. The performance of the inference process with RPTs is examined with some experiments using the variable elimination algorithm.

We present an efficient procedure for factorising probabilistic potentials represented as probability trees. This new procedure is able to detect some regularities that cannot be captured by existing methods. In cases where an exact decomposition is not achievable, we propose a heuristic way to carry out approximate factorisations guided by a parameter called factorisation degree, which is fast to compute. We show how this parameter can be used to control the tradeoff between complexity and accuracy in approximate inference algorithms for Bayesian networks.

The Averaged One-Dependence Estimators (AODE) classifier is one of the most attractive semi-naive Bayesian classifiers and hence a good alternative to Naive Bayes (NB), as it obtains fairly low error rates maintaining under control the computational complexity. Unfortunately, as most of the methods designed within the framework of Bayesian networks, AODE is exclusively defined to deal with discrete variables. Several approaches to avoid the use of discretization pre-processing techniques have already been presented, all of them involving in lower or greater degree the assumption of (conditional) Gaussian distributions. In this paper, we propose the use of Mixture of Truncated Exponentials (MTEs), whose expressive power to accurately approximate the most commonly used distributions for hybrid networks has already been demonstrated. We perform experiments on the use of MTEs over a large group of datasets for the first time, and we analyze the importance of selecting a proper number of points when learning MTEs for NB and AODE, as we believe, it is decisive to provide accurate results.

In this paper we analyze the use of hybrid Bayesian networks in domains that include deterministic conditionals for continuous variables. We show how exact inference can become infeasible even for small networks, due to the difficulty in handling functional relationships. We compare two strategies for carrying out the inference task, using mixtures of polynomials (MOPs) and mixtures of truncated exponentials (MTEs).

In this paper we address the problem of inducing Bayesian network models for regression from incomplete databases. We use mixtures of truncated exponentials (MTEs) to represent the joint distribution in the induced networks. We consider two particular Bayesian network structures, the so-called naïve Bayes and TAN, which have been successfully used as regression models when learning from complete data. We propose an iterative procedure for inducing the models, based on a variation of the data augmentation method in which the missing values of the explanatory variables are filled by simulating from their posterior distributions, while the missing values of the response variable are generated using the conditional expectation of the response given the explanatory variables. We also consider the refinement of the regression models by using variable selection and bias reduction. We illustrate through a set of experiments with various databases the performance of the proposed algorithms.

Naive Bayes models have been successfully used in classification problems where the class variable is discrete. These models have also been applied to regression or prediction problems, i.e. classification problems where the class variable is continuous, but usually under the assumption that the joint distribution of the feature variables and the class is multivariate Gaussian. In this paper we are interested in regression problems where some of the feature variables are discrete while the others are continuous. We propose a Naive Bayes predictor based on the approximation of the joint distribution by a Mixture of Truncated Exponentials (MTE). We have followed a filter-wrapper procedure for selecting the variables to be used in the construction of the model. This scheme is based on the mutual information between each of the candidate variables and the class. Since the mutual information can not be computed exactly for the MTE distribution, we introduce an unbiased estimator of it, based on Monte Carlo methods. We test the performance of the proposed model in artificial and real-world datasets.

In this article we introduce some modifications over the Penniless propagation algorithm. When a message through the join tree is approximated, the corresponding error is quantified in terms of an improved information measure, which leads to a new way of pruning several values in a probability tree (representing a message) by a single one, computed from the value stored in the tree being pruned but taking into account the message stored in the opposite direction. Also, we have considered the possibility of replacing small probability values by zero. Locally, this is not an optimal approximation strategy, but in Penniless propagation many different local approximations are carried out in order to estimate the posterior probabilities and, as we show in some experiments, replacing by zeros can improve the quality of the final approximations. © 2003 Wiley Periodicals, Inc.

Abstract In this paper, we investigate the application of the ideas behind Lazy propagation to the Penniless propagation scheme. Probabilistic potentials attached to the messages and the nodes of the join tree are represented in a factorized way as a product of (approximate) probability trees, and the combination operations are postponed until they are compulsory for the deletion of a variable. We tested two variations of the basic Lazy scheme: One is based on keeping a hash table for the operations with probabilistic potentials that are carried out more than once during the propagation, to avoid repeating computations; the other uses a heuristic method to determine the order of the operations when combining a set of potentials. © 2002 Wiley Periodicals, Inc.

This paper presents non-random algorithms for approximate computation in Bayesian networks. They are based on the use of probability trees to represent probability potentials, using the Kullback-Leibler cross entropy as a measure of the error of the approximation. Different alternatives are presented and tested in several experiments with difficult propagation problems. The results show how it is possible to find good approximations in short time compared with Hugin algorithm. © 2000 John Wiley & Sons, Inc.

In this paper we propose an operational interpretation of general fuzzy measures. On the basis of this interpretation, we define the concept of coherence with respect to a partial information, and propose a rule of inference similar to the natural extension 7. We also suggest a definition of independence for fuzzy measures.