We discuss some recent advances on combining data and knowledge in the context of supervised learning using Bayesian networks. A first set of advances concern the computational efficiency of learning and inference, and they include a software-level boost based on compiling Bayesian network structures into tractable circuits in the form of tensor graphs, and algorithmic improvements based on exploiting a type of knowledge called unknown functional dependencies. The used tensor graphs capitalize on a highly optimized tensor operation (matrix multiplication) which brings orders of magnitude speedups in circuit training and evaluation. The exploitation of unknown functional dependencies yields exponential reductions in the size of tractable circuits and gives rise to the notion of causal treewidth for offering a corresponding complexity bound. Beyond computational efficiency, we discuss empirical evidence showing the promise of learning from a combination of data and knowledge, in terms of data hungriness and robustness against noise perturbations. Sometimes, however, an accurate Bayesian network structure may not be available due to the incompleteness of human knowledge, leading to modeling errors in the form of missing dependencies or missing variable values. On this front, we discuss another set of advances for recovering from certain types of modeling errors. This is achieved using Testing Bayesian networks which dynamically select parameters based on the input evidence, and come with theoretical guarantees on full recovery under certain conditions.

We consider the problem of supervised learning with Bayesian Networks when the used dependency structure is incomplete due to missing edges or missing variable states. These modeling errors induce independence constraints on the learned model that may not hold in the true, data-generating distribution. We provide a unified treatment of these modeling errors as instances of state-space abstractions. We then identify a class of Bayesian Networks and queries which allow one to fully recover from such modeling errors if one can choose Conditional Probability Tables (CPTs) dynamically based on evidence. We show theoretically that the recently proposed Testing Bayesian Networks (TBNs), which can be trained by compiling them into Testing Arithmetic Circuits (TACs), provide a promising construct for emulating this CPT selection mechanism. Finally, we present empirical results that illustrate the promise of TBNs as a tool for recovering from certain modeling errors in the context of supervised learning.

Testing Bayesian Networks (TBNs) were introduced recently to represent a set of distributions, one of which is selected based on the given evidence and used for reasoning. TBNs are more expressive than classical Bayesian Networks (BNs): Marginal queries correspond to multi-linear functions in BNs and to piecewise multi-linear functions in TBNs. Moreover, TBN queries are universal approximators, like neural networks. In this paper, we study conditional independence in TBNs, showing that it can be inferred from d-separation as in BNs. We also study the role of TBN expressiveness and independence in dealing with the problem of learning with incomplete models (i.e., ones that miss nodes or edges from the data-generating model). Finally, we illustrate our results on a number of concrete examples, including a case study on Hidden Markov Models.