In recent years, Bagging method has been applied to learn Bayesian networks (BNs), especially on limited datasets. However, the BNs learned using Bagging method from limited datasets can be biased towards complex models. We present an efficient approach to produce more accurate BNs from limited datasets. Based on the Markov condition of BN learning, we proposed a novel sampling method, called Root Nodes based Sampling (RNS), and a BNs fusion method. The experimental results reveal that our ensemble method can achieve more accurate results in terms of accuracy on limited datasets.

Currently, there are few efficient methods in practice for learning Bayesian networks from incomplete data, which affects their use in real world data mining applications. We present a general-duty method that estimates the (conditional) mutual information directly from incomplete datasets, EMI. EMI starts by computing the interval estimates of a joint probability of a variable set, which are obtained from the possible completions of the incomplete dataset. And then computes a point estimate via a convex combination of the extreme points, with weights depending on the assumed pattern of missing data. Finally, based on these point estimates, EMI gets the estimated (conditional) mutual information. We also apply EMI to the dependency analysis based learning algorithm by J. Cheng so as to efficiently learn BNs with incomplete data. The experimental results on Asia and Alarm networks show that EMI based algorithm is much more efficient than two search & scoring based algorithms, SEM and EM-EA algorithms. In terms of accuracy, EMI based algorithm is more accurate than SEM algorithm, and comparable with EM-EA algorithm.

For simplicity of calculation, dynamic Bayesian networks (DBNs) make assumptions that their evolvement follows Markov process and the transition probabilities in the evolvement are time-invariant. While this is not the case in many real complex systems. For the purpose of modeling these complex systems with DBNs, we attempt to add hidden variables to the evolutional process so as to build Markov models and expand the EM-EA algorithm to the DBNs learning in the presence of hidden variables. Moreover, as for the time-variant transition probabilities, we estimate the suffcient statistics of posterior time slices using polynomial fitting algorithm, and then learns the time-variant transition probabilities with both current sufficient statistics and estimated sufficient statistics. The theoretical analysis demonstrates the validity of the methods. The future work is to do experimental analysis with real complex systems.

An incremental method for learning Bayesian networks based on evolutionary computing, IEMA, is put forward. IEMA introduces the evolutionary algorithm and EM algorithm into the process of incremental learning; it can avoid getting into local maxima, and also incrementally learn Bayesian networks with high accuracy in the presence of missing values and hidden variables. In addition, we improved the incremental learning process by N. Friedman and M. Goldschmidt (1997). The experimental results verified the validity of IEMA. In terms of storage cost, IEMA is comparable with the incremental learning method of Friedman et al, while it is more accurate.