The fuzzy broad learning system (FBLS) is a recently proposed neuro-fuzzy model that shares the similar structure of a broad learning system (BLS). It shows high accuracy in both classification and regression tasks and inherits the fast computational nature of a BLS. However, the ensemble of several fuzzy subsystems in an FBLS decreases the possibility of understanding the fuzzy model since the fuzzy rules from different fuzzy systems are difficult to combine together while keeping the consistence. To balance the model accuracy and complexity, a synthetically simplified FBLS with better interpretability, named compact FBLS (CFBLS), is developed in this article, which can generate much fewer and more explainable fuzzy rules for understanding. In such a way, only one traditional Takagi–Sugeno–Kang fuzzy system is employed in the feature layer of a CFBLS, and the input universe of discourse is equally partitioned to obtain the fuzzy sets with proper linguistic labels accordingly. The random feature selection matrix and rule combination matrix are employed to reduce the total number of fuzzy rules and to avoid the “curse of dimensionality.” The enhancement layer is kept in the CFBLS that helps to add a nonlinear transformation of input features to the traditional first-order polynomial used in the consequent part of a fuzzy rule. The pseudoinverse is also used to determine the parameters of CFBLS guaranteeing its fast computational nature. The experiments on the popular UCI and KEEL datasets indicate that the CFBLS can generate a smaller set of comprehensible fuzzy rules and achieve much higher accuracy than some state-of-the-art neuro-fuzzy models. Moreover, the advantage of CFBLS is also verified in a real-world application.

The broad learning system (BLS) has been proved to be effective and efficient lately. In this article, several deep variants of BLS are reviewed, and a new adaptive incremental structure, Stacked BLS, is proposed. The proposed model is a novel incremental stacking of BLS. This invariant inherits the efficiency and effectiveness of BLS that the structure and weights of lower layers of BLS are fixed when the new blocks are added. The incremental stacking algorithm computes not only the connection weights between the newly stacking blocks but also the connection weights of the enhancement nodes within the BLS block. The Stacked BLS is considered as the increment of “layers” and “neurons” dynamically during the training for multilayer neural networks. The proposed architecture along with the training algorithms that utilizes the residual characteristic is very versatile in comparison with traditional fixed architecture. Finally, experimental results on UCI datasets, MNIST dataset, NORB dataset, CIFAR-10 dataset, SVHN dataset, and CIFAR-100 dataset indicate that the proposed method outperforms the selected state-of-the-art methods on both accuracy and training speed, such as deep residual networks. The results also imply that the proposed structure could highly reduce the number of nodes and the training time of the original BLS in the classification task of some datasets.

The restricted Boltzmann machine (RBM) is an excellent generative learning model for feature extraction. By extending its parameters from real numbers to fuzzy ones, we have developed the fuzzy RBM (FRBM) which is demonstrated to possess better generative capability than RBM. In this paper, we first propose a generative model named Gaussian FRBM (GFRBM) to deal with real-valued inputs. Then, motivated by the fact that the discriminative variant of RBM can provide a self-contained framework for classification with competitive performance compared with some traditional classifiers, we establish the discriminative FRBM (DFRBM) and discriminative GFRBM (DGFRBM) that combine both the generative and discriminative facility by adding extra neurons next to the input units. Specifically, they can be trained into excellent stand-alone classifiers and retain outstanding generative capability simultaneously. The experimental results including text and image (both clean and noisy) classification indicate that DFRBM and DGFRBM outperform discriminative RBM models in terms of reconstruction and classification accuracy, and they behave more stable when encountering noisy data. Moreover, the proposed learning models show some promising advantages over other standard classifiers.

A novel neuro-fuzzy model named fuzzy broad learning system (BLS) is proposed by merging the Takagi-Sugeno (TS) fuzzy system into BLS. The fuzzy BLS replaces the feature nodes of BLS with a group of TS fuzzy subsystems, and the input data are processed by each of them. Instead of aggregating the outputs of fuzzy rules produced by every fuzzy subsystem into one value immediately, all of them are sent to the enhancement layer for further nonlinear transformation to preserve the characteristic of inputs. The defuzzification outputs of all fuzzy subsystem and the outputs of enhancement layer are combined together to obtain the model output. The k -means method is employed to determine the centers of Gaussian membership functions in antecedent part and the number of fuzzy rules. The parameters that need to be calculated in a fuzzy BLS are the weights connecting the outputs of enhancement layer to model output and the randomly initialized coefficients of polynomials in consequent part in fuzzy subsystems, which can be calculated analytically. Therefore, fuzzy BLS retains the fast computational nature of BLS. The proposed fuzzy BLS is evaluated by some popular benchmarks for regression and classification, and compared with some state-of-the-art nonfuzzy and neuro-fuzzy approaches. The results indicate that fuzzy BLS outperforms other models involved. Moreover, fuzzy BLS shows advantages over neuro-fuzzy models regarding to the number of fuzzy rules and training time, which can ease the problem of rule explosion to some extent.

We establish a fuzzy deep model called the fuzzy deep belief net (FDBN) based on fuzzy restricted Boltzmann machines (FRBMs) due to their excellent generative and discriminative properties. The learning procedure of an FDBN is divided into a pretraining phase and a subsequent fine-tuning phase. In the pretraining phase, a group of FRBMs is trained in a greedy layerwise way: the first FRBM is trained by original samples, and the average values of the left and right probabilities produced by its hidden units are treated as the training data for subsequent FRBMs. The resulting FDBN is either a generative or a discriminative model depending on the choice of training a generative or a discriminative type of FRBM on top. Then, a hybrid learning approach is proposed to fine-tune this novel fuzzy deep model: the well pretrained fuzzy parameters are first defuzzified, and the FDBN with defuzzified parameters is fine-tuned by the wake-sleep or stochastic gradient descent algorithm. This hybrid strategy not only avoids learning an intractable fuzzy neural network, but also greatly improves the classification capability of the FDBN. The experimental results on MNIST, NORB, and 15 Scene databases indicate that the FDBN with the hybrid learning approach can handle high-dimensional raw images directly. It inherits the fine nature of the FRBM and outperforms some state-of-the-art discriminative models in classification accuracy. Moreover, it shows better capability of robustness than a deep belief net when encountering noisy data.

After a very fast and efficient discriminative broad learning system (BLS) that takes advantage of flatted structure and incremental learning has been developed, here, a mathematical proof of the universal approximation property of BLS is provided. In addition, the framework of several BLS variants with their mathematical modeling is given. The variations include cascade, recurrent, and broad-deep combination structures. From the experimental results, the BLS and its variations outperform several exist learning algorithms on regression performance over function approximation, time series prediction, and face recognition databases. In addition, experiments on the extremely challenging data set, such as MS-Celeb-1M, are given. Compared with other convolutional networks, the effectiveness and efficiency of the variants of BLS are demonstrated.

We apply the Broad Learning System (BLS) to control of nonlinear dynamic systems by proposing an iterative learning algorithm using the gradient descent method. The parameters in the enhancement layer and the top layer consisting of weights connecting the feature nodes to the enhancement nodes and the output nodes, the weights connecting the enhancement nodes to the output nodes, as well as the corresponding bias terms, are adjusted to reduce the error between the plant output and desired signal, while the remain weights and bias terms are randomly generated. The convergence of this learning algorithm is guaranteed by proving a Lyapunov stability theorem. The control performance of BLS with the proposed iterative learning algorithm on two dynamic plants indicates its effectiveness and practicability. Moreover, the BLS based controller produces correct control inputs much faster with fewer parameters, and behaves more accurate than some neuro and fuzzy-neuro models.

A fuzzy restricted Boltzmann machine (FRBM) is extended from a restricted Boltzmann machine (RBM) by replacing all the real-valued parameters with fuzzy numbers. A new FRBM that employs the crisp possibilistic mean value of a fuzzy number to defuzzify the fuzzy free energy function is presented. This approach is much clearer and easier to obtain the expression of the defuzzified free energy function and its approximation than the centroid method. Several theorems that discuss the error bounds of the approximation to ensure the rationality and validity are also investigated. Learning algorithms are given for the designed FRBM with symmetric triangular fuzzy numbers (STFNs), asymmetric triangular fuzzy numbers, and Gaussian fuzzy numbers. By appropriately choosing the parameters, a theorem is concluded that all FRBMs with symmetric fuzzy numbers will have identical learning algorithm to that of FRBMs with STFNs. This is illustrated by a case of FRBM with Gaussian fuzzy numbers. Two experiments including the MNIST handwriting recognition and the Bars-and-Stripes benchmark are carried out. The results show that the proposed FRBMs significantly outperform RBMs in learning accuracy and generalization ability, especially when encountering unlearned samples and recovering incomplete images.

The fuzzy restricted Boltzmann machine (FRBM) is demonstrated to have better generative and discriminative capabilities than traditional RBM. We now further investigate and compare the generative ability of DBN with FRBM on image reconstruction. The DBN is pre-trained by stacking RBMs layer by layer and then fine-tuned by the wake-sleep algorithm. Then the FRBM, RBM and DBN are compared in detail under different conditions on the MNIST and Extended Yale B data sets. The experiment results again indicate that the FRBM outperforms RBM: it can achieve smaller average reconstruction errors (AREs) given the same number of hidden units and learning time. When compared to DBNs, the FRBM can achieve smaller AREs in less learning time than the two-layer DBN with equal hidden size. Moreover, when we increase the training epochs, the FRBM shows a better ARE and a slight increase of (or still less) learning time than corresponding two-layer and three-layer DBNs with double number of hidden units. Hence we make a preliminary conclusion that the FRBM with m hidden units possesses the close generative capability to DBN with 2m hidden units in reconstructing images.