A novel reinforced dual-flow neural network based on attention and a polynomial-based radial basis function network (DFBTP) is proposed to enhance classification performance on tabular data. DFBTP consists of two types of architectures such as advanced Transformer model based on Tabular Prior data Fitted Network (TabPFN) and SV-clustering driven radial basis function neural network (SV-PRBFNN). The conventional PRBFNN model may encounter local minima and noise issues during training, which negatively impacts its performance. Local minima result from initial parameter choices, and noise issues are inherent in the dataset. By introducing the Transformer model and the Whale Optimization Algorithm (WOA), these challenges can be mitigated. Within the dual-flow architecture, the attention flow is trained using Bayesian inference capabilities and structural causal models, and it uses the self-attention mechanism to capture global features. This approach mitigates the problem of local minima. SV-PRBFNN flow uses the fuzzy clustering algorithm based on support vectors to replace the original radial basis function for training. Fuzzy clustering based on support vectors can alleviate the negative impact of outliers on model performance and also reduce the number of fuzzy rules. During neural network hyperparameter optimization, WOA is used to identify the global optimal values for hyperparameters. In the experiments, DFBTP demonstrated its superiority in classification accuracy in comparative experiments on 18 datasets and 13 models, and also performed well on real-world datasets. The robust performance of DFBTP was further validated through statistical analysis of the experimental results.

Polynomial neural networks (PNN) have emerged as an effective regression modeling methodology in computational intelligence, relying on its interpretable polynomial nodes to fit complex nonlinear data relationships and the adaptive nature of self-organizing networks. To break the bottleneck of PNN structure in the field of multi-classification, this study designs a dynamical multiple polynomial-based neural networks (DMPNN) classifier, focusing on developing a flexible polynomial network classification methodology that enhances predictive capabilities without sacrificing the advantages of PNN structures. Our approach effectively addresses the challenges of multi-class classification with uncertain class boundaries and reduces computational complexity, which is achieved through the synergy of several proposed techniques. Three key issues underpin the proposed DMPNN: (a) The integration of PNN regression models using the one-against-all strategy can provide effective and scalable solutions to multi-class classification problems, especially for uncertain class boundary issues. (b) The dual statistical selection (DSS) approach aims to eliminate redundant inputs during data processing, reduce the computational burden, and increase the variety of neural network nodes in the model neuron selection stage. (c) The synergy of regularization methods including the ℓ2 norm-based method (ℓ2-LSM) and the DropFilter, is exploited to mitigate potential overfitting in coefficient estimation and enhance the generalization capabilities of the proposed classifier. A series of ablation experiments and parameter analysis were conducted to demonstrate the stability and reliability of the proposed model. Then, we applied DMPNN to 17 publicly available datasets and two engineering applications: Recycling of black plastic wastes and phased resolved partial discharge. The performance results show that the DMPNN model outperforms five classical classifiers and four state-of-the-art (SOTA) classifiers on 78.94% of the datasets. This highlights the unique ability of the proposed DMPNN to enhance predictive accuracy while maintaining model simplicity and interpretability.

In order to represent the influences of different semantics on targets and improve the prediction with interpretability ability for multi-dimensional time series, we integrate Axiomatic Fuzzy Set (AFS) and Fuzzy Cognitive Map (FCM) with memory for fuzzy knowledge representation and prediction in this paper. The AFS is used to extract semantics of concepts for fuzzy representation using data distribution. The FCM with memory is trained to model the influence relationships between different semantics of concepts and multiple targets based on multi-dimensional time series data. And a multi- dimensional learning algorithm of AFS-FCM with memory based on gradient descent is developed to investigate the influences of different semantics of concepts on multiple targets. Finally, we validate our model by comparing with other FCMs, intrinsic interpretable models and machine learning methods for prediction of air quality multidimensional time series data, and discuss the performance of AFS-FCM with different transformation functions. The model can not only predict air quality accurately, but also explicitly reveal the specific quantitative relationship of different semantics of meteorology on air quality.

In this paper, we propose a hybrid ensemble polynomial neural network (HEPNN) with the aid of polynomial neural network (PNN) and hybrid ensemble polynomials neurons (HEPNs). Two types of HEPNs including ensemble radial-based-function polynomial neuron (ERPN) and ensemble polynomial neuron (EPN) are proposed. ERPN and EPN are generalized polynomial neurons based on ensemble architecture. To address the problem of multiple covariance in traditional PNN neural networks, correlation coefficients and performance are utilized to select neurons replacing the original selection of nodes by performance only. The main strategies of HEPNN design are as follows: First, the first layer of the network consists of ERPN that are utilized to reflect the structure encountered between the data, while the second and higher layers consists of EPN, which reflect higher polynomial-order relationships between input and output data. Second, particle swarm optimization (PSO) is adopted to optimize the architecture of HEPNN. A comparative study shows the proposed HEPNN has better performance than other state-of-art models reported in literature.

A novel fuzzy adaptive knowledge-based inference neural network (FAKINN) is proposed in this study. Conventional fuzzy cluster-based neural networks (FCBNNs) suffer from the challenge of a direct extraction of fuzzy rules that can capture and represent the interclass heterogeneity and intraclass homogeneity when the data possess complex structures. Moreover, the capability of the cluster-based rule generator in FCBNNs may decrease with the increase of data dimensionality. These drawbacks impede the generation of desired fuzzy rules, and affect the inference results depending on the fuzzy rules, thereby limiting their generalization ability. To address these drawbacks, an adaptive knowledge generator (AKG), consisting of the observation paradigm (OP) and clustering strategy (CS), is effectively designed to improve the generalization ability in FAKINN. The OP distills the characteristic information (CI) from data to highlight the homogeneity and heterogeneity of objects, and the CS, viz., the weighted condition-driven fuzzy clustering method (WCFCM), is proposed to summarize the CI to construct fuzzy rules. Moreover, the feedback between the OP and CS can control the dimensionality of CI, which endows FAKINN with the potential to tackle high-dimensional data. The main originality of the study focuses on the AKG and WCFCM that are proposed to develop the structural design methodology of FNNs. The performance of FAKINN is evaluated on various benchmarks with 27 comparative methods, and two real-world problems are adopted to validate its effectiveness. Experimental results show that FAKINN outperforms the comparison methods.

Fuzzy clustering-based neural networks (FCNNs) based on information granulation techniques have been shown to be effective Takagi–Sugeno (TS)-type fuzzy models. However, the existing FCNNs could not cope well with sequential learning tasks. In this study, we introduce incremental FCNNs (IFCNNs), which could dynamically update themselves whenever new learning data (e.g., single datum or block data) are incorporated into the dataset. Specifically, we employ dynamic (incremental) fuzzy C-means (FCMs) clustering algorithms to reveal a structure in data and divide the entire input space into several subregions. In the aforementioned partition, the dynamic FCM adaptively adjusts the position of its prototypes by using sequential data. Due to the time-sharing arrival of training data, compared with batch learning models, incremental learning methods may lose classification (prediction) accuracy. In order to tackle this challenge, we utilize quasi-fuzzy local models (QFLMs) based on modified Schmidt neural networks to replace the popular linear functions in TS-type fuzzy models to refine and enhance the ability to represent the behavior of fuzzy subspaces. Meanwhile, the recursive least square error (LSE) estimation is utilized to update the weights of QFLMs from one-by-one or block-by-block (fixed or varying block size) learning data. In addition, the$L_{2}$regularization is considered to ameliorate the deterioration of generalization abilities caused by potential overfitting when carrying out weight estimation. The proposed method leads to the construction of FCNNs in a new way, which can effectively deal with incremental data as well as deliver sound generalization capability. Extensive machine-learning datasets and a real-world application are employed to show the validity and performance of the presented methods. From the experimental results, we show that the proposal can maintain sound classification accuracy when effectively processing sequential data.

Intraclass heterogeneity of medical diagnostic objects poses a challenge for accurate intraclass classification of medical fine-grained images (MFGIs) within deep learning. To accurately classify MFGIs, we propose a novel approach termed fuzzy channel filter-based separable convolution neural networks (FSCNN). The original design of FSCNN comprises the following components: 1) Designing the fuzzy channel filter (FCF) module, devised to establish long-distance feature dependencies for each feature channel with the input image by formulating fuzzy rules “IF–THEN”. 2) The FCF-based separable convolution (FSC) block uses depth-wise and point-wise convolutions to extract and mix feature channels. Then, the internal information of each feature channel is reintegrated through fuzzy weighted averaging in FCF to enhance fine-grained feature information. 3) Creating the deep fuzzy learning architecture FSCNN through the superimposition of FSC blocks. This architectural arrangement enables more effective learning of fine-grained feature distinctions within MFGIs, thereby enhancing classification accuracy. Compared to other advanced fine-grained classification models, including state-of-the-art models, our model outperforms by 2%–6% and 3%–9% on brain MRI and pneumonia CT datasets, respectively.

In this study, a novel semantic concept-based inference neural network (SCINN) is proposed to develop the design methodology of the explainable deep neuro-fuzzy models and improve their generalization performance in high-dimensional problems. Traditional neuro-fuzzy models exhibit outstanding interpretability in the problems with lower dimensionality. However, when faced with high-dimensional scenarios, the long rule and rule explosion problems damage their interpretability and result in poor generalization performance (e.g., accuracy), even making them unusable. Although deep neuro-fuzzy models show enhanced performance in handling high-dimensional problems compared to traditional neuro-fuzzy models, they often come at the expense of interpretability. In order to establish the neuro-fuzzy model that is capable of addressing the high-dimensional problems while preserving the interpretability, the SCINN is proposed with the aid of the concept-based measure generation paradigm (CMGP) and the multi-view information augmentation strategy (MIAS). The CMGP is designed to adaptively define the membership functions (MFs) that correspond to the human-understandable semantic concepts based on the given data; the defined MFs contribute to the construction of the explainable fuzzy rule that can directly process high-dimensional data. The MIAS is structured to develop a unified paradigm for implementing consequence functions in the fuzzy rules, which enhances the approximation ability of the SCINN. The performance of SCINN is evaluated on various image datasets using different comparison methods, including neuro-fuzzy-based approaches and deep structure-based neural networks. Furthermore, a real-world application is adopted to evaluate its effectiveness. The experimental results show that SCINN outperforms the compared neuro-fuzzy models and is comparable to the deep structure-based neural network.

In this article, a novel attention-based reinforced interval type-2 fuzzy clustering neural network (ARIT2FCN) is developed to improve the generalization performance of fuzzy clustering-based neural networks (FCNNs). Commonly, fuzzy rules in FCNNs are generated through the clustering-based rule generator. However, the generated fuzzy rules may not be able to fully describe the given data, because the clustering-based rule generator does not simultaneously consider the intracluster homogeneity and intercluster heterogeneity for both of data characteristics and label information when defining membership functions (MFs) of fuzzy rules. This negatively affects fuzzy rules to accurately quantify the interclass heterogeneity and intraclass homogeneity and degrades the performance of FCNNs. The ARIT2FCN is proposed with the aid of the attention-based clustering mechanism and the successive learning method. The attention-based clustering mechanism is designed to define MFs by simultaneously considering data characteristics and label information. The successive learning method is adopted to construct the desired fuzzy rules that can capture the interclass heterogeneity and intraclass homogeneity. Moreover,L$_{2}$norm regularization is used to alleviate the overfitting effect. The performance of ARIT2FCN is evaluated on machine learning datasets with 16 comparative methods. In addition, two real-world problems are adopted to validate the effectiveness of ARIT2FCN. Experimental results demonstrate that the ARIT2FCN outperforms the comparative methods, and the statistical tests also support the superiority of ARIT2FCN.

In this article, we present a dimensionality reduction methodology of reinforced fuzzy-rule-based neural networks (FRNNs) realized with the help of determination/correlation coefficient-based streamlined feature selection strategy and fuzzy clustering with standard deviation to cope with high-dimensional data. This approach aims to reduce the design process of the proposed networks and to curb the computational overhead inherently associated with the increasing volume of data both in terms of their number and the dimensionality of the feature space. The overall architecture and learning mechanism of the FRNNs are based on radial basis function neural networks. However, we design the hidden layer of RBFNNs differently by using fuzzy clustering, which makes it easy to determine the parameters, such as centers and widths of the receptive fields (activation functions). Unlike conventional neural networks, the RBFNNs do not have a feature to support dimensionality reduction. To overcome this limitation, FRNNs select input variables by evaluating the adjusted determination coefficient of the model. To reduce the computational burden of finding an appropriate combination of inputs, we propose a simplified feature selection and elimination technique, in which the variables are selected or eliminated by correlation coefficients. A linear function expresses the connection weight, and we apply L2-norm regularization to least-square-error-based learning to estimate stable coefficients (weights), which is expected to significantly improve the generalization ability. The superiority of the proposed FRNNs was demonstrated by using 28 real-world benchmark datasets. The networks are also compared with the conventional models associated with the FRNNs and several related models previously published in the literature.

This study presents an innovative approach to the design of a hybrid fuzzy classifier, with a focus on exploring the classification capability of a conventional fuzzy polynomial neural network (CFPNN). The proposed novel self-organizing hybrid fuzzy polynomial NN classifier (HFPNNC) improves performance while maintaining model interpretability and fixability by synergistically combining an adaptive network structure and the compound regularization technique (CRT). Recent studies have focused on exploring the potential of CFPNN structures for addressing regression issues. To effectively introduce the CFPNN framework to multiclassification tasks, a resilient fuzzy polynomial neural network structure was designed as a basic subclassifier to construct the proposed HFPNNC. The proposed HFPNNC employs a dynamically adaptive structure comprising of two types of critical layers: 1) the fuzzy set-based polynomial neurons layers and 2) the polynomial neurons layers. This allows the classifier to adapt to the complexity of classification tasks. To further strengthen the robustness and generalization ability of the HFPNNC, we incorporate the synergistic combination of probabilistic constrained competitive response selection (PCCRS) and ℓ2-norm regularization least squares estimation (ℓ2-LSE) methods to manage the generation of network layers and the estimation of neuron weights. As key constituents of the CRT approach, the PCCRS and ℓ2-LSE method strike a balance between model complexity and performance. The effectiveness of the proposed HFPNNC is thoroughly evaluated against classical classifiers, state-of-the-art fuzzy classifiers and deep learning baseline models using 17 public datasets, two real-world datasets, and three large-scale datasets. HFPNNC achieves the best prediction in 72.7% of the data. The experimental results and the statistical analysis show a remarkable advantage of the HFPNNC over existing methods, confirming its potential as a flexible, interpretable solution for classification tasks.

Time series forecasting is an essential and challenging task, especially for large-scale time-series (LSTS) forecasting, which plays a crucial role in many real-world applications. Due to the instability of time series data and the randomness (noise) of their characteristics, it is difficult for polynomial neural network (PNN) and its modifications to achieve accurate and stable time series prediction. In this study, we propose a novel structure of hierarchical neural networks (HNN) realized by long short-term memory (LSTM), two classes of self-organizing dynamical fuzzy neural network architectures of fuzzy rule-based polynomial neurons (FPN) and polynomial neurons (PN) constructed by variant generation of nodes as well as layers of networks. The proposed HNN combines the deep learning method with the PNN method for the first time and extends it to time series prediction as a modification of PNN. LSTM extracts the temporal dependencies present in each time series and enables the model to learn its representation. FPNs are designed to capture the complex non-linear patterns present in the data space by utilizing Fuzzy C-Means clustering (FCM) and least square error (LSE)-based learning of polynomial functions. The self-organizing hierarchical network architecture generated by the Elitism-based Roulette Wheel Selection (ERWS) strategy ensures that candidate neurons exhibit sufficient fitting ability while enriching the diversity of heterogeneous neurons, addressing the issue of multicollinearity and providing opportunities to select better prediction neurons. In addition, L2-norm regularization is applied to mitigate the overfitting problem. Experiments are conducted on 9 real-world LSTS datasets including three practical applications. The results show that the proposed model exhibits high prediction performance, outperforming many state-of-the-art models.

Fuzzy min-max neural network (FMNN) is one kind of three-layer models based on hyperboxes that are constructed in a sequential way. Such a sequential mechanism inevitably leads to the input order and overlap region problem. In this study, we propose a deep FMNN (DFMNN) based on initialization and optimization operation to overcome these limitations. Initialization operation that can solve the input order problem is to design hyperboxes in a simultaneous way, and side parameters have been proposed to control the size of hyperboxes. Optimization operation that can eliminate overlap region problem is realized by means of deep layers, where the number of layers is immediately determined when the overlap among hyperboxes is eliminated. In the optimization process, each layer consists of three sections, namely, the partition section, combination section, and union section. The partition section aims to divide the hyperboxes into a nonoverlapping hyperbox set and an overlapping hyperbox set. The combination section eliminates the overlap problem of overlapping hyperbox set. The union section obtains the optimized hyperbox set in the current layer. DFMNN is evaluated based on a series of benchmark datasets. A comparative analysis illustrates that the proposed DFMNN model outperforms several models previously reported in the literature.

In this article, we propose the concept of random polynomial neural networks (RPNNs) realized based on the architecture of polynomial neural networks (PNNs) with random polynomial neurons (RPNs). RPNs exhibit generalized polynomial neurons (PNs) based on random forest (RF) architecture. In the design of RPNs, the target variables are no longer directly used in conventional decision trees, and the polynomial of these target variables is exploited here to determine the average prediction. Unlike the conventional performance index used in the selection of PNs, the correlation coefficient is adopted here to select the RPNs of each layer. When compared with the conventional PNs used in PNNs, the proposed RPNs exhibit the following advantages: first, RPNs are insensitive to outliers; second, RPNs can obtain the importance of each input variable after training; third, RPNs can alleviate the overfitting problem with the use of an RF structure. The overall nonlinearity of a complex system is captured by means of PNNs. Moreover, particle swarm optimization (PSO) is exploited to optimize the parameters when constructing RPNNs. The RPNNs take advantage of both RF and PNNs: it exhibits high accuracy based on ensemble learning used in the RF and is beneficial to describe high-order nonlinear relations between input and output variables stemming from PNNs. Experimental results based on a series of well-known modeling benchmarks illustrate that the proposed RPNNs outperform other state-of-the-art models reported in the literature.

A novel structure of reinforced two-stream fuzzy neural networks (TSFNNs) realized with the aid of fuzzy logic and transfer learning method is presented. This architecture consists of a TSFNN and a fusion strategy. TSFNN architecture consists of two combined networks of both fuzzy rules-based radial basis function neural networks (FRBFNN) and convolutional neural networks (CNNs). In the TSFNN architecture, one stream employs the deep CNN to extract the spatial information of images and effectively learn the high-level features and another stream uses the FRBFNN to analyze the distribution of data points over the input space and learn to capture complex relationships in data. In the fusion strategy, the outputs of two streams are concatenated by a softmax function, which normalizes the output to a probability distribution. A transfer learning method is considered to reconstruct new data representation as the inputs of CNN to mine potential spatial features of data. Moreover, L2-norm regularization is used to alleviate the possible overfitting and enhance the generalization ability. The proposed method not only inherits the advantages of FRBFNN and CNN such as global feature extraction ability, good local approximating performance, ability of handling uncertainty by fuzzy logic but also improves the classification performance under the synergy between two-stream architecture and the fusion strategy. Experimental results obtained for a diversity of datasets as well as partial discharge datasets be using in the real life of fault diagnosis and black plastic wastes datasets for recycling confirm the effectiveness of the proposed TSFNN. A comprehensive comparative analysis is covered. This design can simultaneously capture different level information of inputs and easing the insufficient problem of extracting features from a single steam. Especially, we show that the synergistic effect of FRBFNN, CNN, enabling deep learning for generic classification tasks and multipoint crossover, and L2-norm regularization can effectively improve the performance of the TSFNNs.

In this study, a novel promotive particle swarm optimizer with double hierarchical structures is proposed. It is inspired by successful mechanisms present in social and biological systems to make particles compete fairly. In the proposed method, the swarm is first divided into multiple independent subpopulations organized in a hierarchical promotion structure, which protects subpopulation at each hierarchy to search for the optima in parallel. A unidirectional communication strategy and a promotion operator are further implemented to allow excellent particles to be promoted from low-hierarchy subpopulations to high-hierarchy subpopulations. Furthermore, for the internal competition within each subpopulation of the hierarchical promotion structure, a hierarchical multiscale optimum controlled by a tiered architecture of particles is constructed for particles, in which each particle can synthesize a set of optima of its different scales. The hierarchical promotion structure can protect particles that just fly to promising regions and have low fitness from competing with the entire swarm. Also, the double hierarchical structures increase the diversity of searching. Numerical experiments and statistical analysis of results reported on 30 benchmark problems show that the proposed method improves the accuracy and convergence speed especially in solving complex problems when compared with several variations of particle swarm optimization.

In this article, we introduce a design methodology of reinforced fuzzy models based both on univariate analysis and multivariable analysis to cope with high-dimensional problems. This approach is aimed at reducing the design process and curbing computing overhead inherently associated with the increasing volume of data in terms of both their number and the dimensionality of the feature space. The critical features of the proposed fuzzy models are highlighted as follows: First, the essential input variables of the model are selected by running the univariable and multivariable analyses. In univariate analysis, input variables with a strong linear relationship with the output variable are selected through correlation analysis completed for each input space and output space. On the contrary, in multivariable analysis, input variables are chosen by comparing the determination coefficients obtained from the subsets of input variables. Second, according to the analysis of the input variable, we construct two different kinds of fuzzy models. The first fuzzy model comprises the design of the univariable-based fuzzy model (UFM) and its aggregation. The UFMs are made by the individual input variables selected from the univariable analysis using a correlation coefficient. The subspaces formed by correlation analysis are applied for determining the centers of the membership function (MF). The results produced by individual fuzzy models are aggregated through somet-conorms. The second fuzzy model is with the fuzzy clustering for improving the form of fuzzy space in the premise part of a fuzzy rule. To curb the dramatic increase in size of fuzzy rule in high-dimensional problems, the clustering space is employed as the fuzzy space, and the partition matrix produced by fuzzy provided the required degrees of the MF. Experimental studies include a suite of synthetic and publicly available data. The superiority of the proposed design methodology was demonstrated by using 34 publicly available datasets and also compared with the conventional models associated with the fuzzy rule-based models as well as the state-of-the-art models reported in the literature.

In this article, a reinforced iterative fuzzy radial basis function neural networks (IFRBFNN) is introduced as an augmented FRBFNN architecture generated through the iterative refinement process of weighted fuzzy C-means (WFCM) clustering and weighted least square error estimation. It is well known that the location of the clusters have an effect on the classification performance of the FRBFNN. The underlying idea behind this article is how to define the center points of clusters based on the data distribution analysis as well as the improvement of classification performance. In a nutshell, while the fuzzy C-means clustering to define the positions of radial basis functions (RBFs) based in the unsupervised learning manner is usually used in a FRBFNN, the parameters (cluster centers and their ensuing polynomial coefficients) refinement of FRBFNN through the proposed iterative method lead to superb classification performance by relocating the positions of RBFs over the input space implied by supervised learning. The idea of the proposed approach is to relocate the centers (prototypes) of the fuzzy clusters by using WFCM clustering algorithm and to re-estimate the coefficients by weighted least square estimation with the aid of the cross-entropy loss values of data so that the classification performance is improved. The weight related to each data is determined by some auxiliary information (i.e., the modified version of the cross-entropy loss value of each data). When it comes to the estimation of the coefficients of the consequent polynomial in IFRBFNN, they are estimated by the weighted least square error estimation technique, where the weights of the coefficients are defined based on the cross-entropy loss values. The auxiliary information such as the weights for relocation of the centers of RBFs and the weights for the re-estimation of the polynomial coefficients should be defined from the viewpoint of enhancement of the classification performance. The cross-entropy loss value of each data is able to meet the necessity of the required supervision signal. Several numerical experiments are provided to demonstrate the usefulness of the proposed design method for the FRBFNN for classification problems.

The two issues on dynamically generated hierarchical neural networks such as the sort of basic neurons and how to compose a layer are considered in this article. On the first issue, a variant version of the least-square support vector regression (SVR) is chosen as a basic neuron. Support vector machine (SVM) is a representative classifier which usually shows good classification performance. Along with the SVMs, SVR was introduced to deal with the regression problem. Especially, least-square SVR has the advantages of high learning speed due to the substitution of the inequality constraints by the equality constraint in the formulation of the optimization problem. Based on the least-square SVR, the multiple least-square (MLS) SVR, which is a type of a linear combination of least-square SVRs with fuzzy clustering, is proposed to improve the modeling performance. In addition, a hierarchical neural network, where the MLS SVR is utilized as the generic node instead of the conventional polynomial, is developed. The key issues of hierarchical neural networks, which are generated dynamically layer by layer, are discussed on how to retain the diversity of the nodes located at the same layer according to the increase of the layer. In order to maintain the diversity of the nodes, various selection methods such as truncation selection and roulette wheel selection (RWS) to choose the nodes among candidate nodes are proposed. In addition, in order to reduce the computational overhead to determine all candidates which exhibit all compositions of the input variables, a new implementation method is proposed. From the viewpoint of the diversity of the selected nodes and the computational aspects, it is shown that the proposed method is preferred over the conventional design methodology.

Neural networks (NNs) with least square error (LSE) estimation form a certain type of single hidden layer feed-forward NNs. In this class of networks, the input connections (weights) and the biases of hidden neurons are generated randomly and fixed after being generated. The output connections are estimated by the LSE method rather than the back-propagation method. The random generation of the input connection weights and the hidden biases results in the larger number of hidden neurons to assure the quality of classification performance. To reduce the number of neurons in the hidden layer while maintaining the classification performance, we apply a “divide and conquer” strategy in this article. In other words, we divide an overall input space into several subspaces by using information granulation technique (Fuzzy C-Means clustering algorithm) and determine the local decision boundaries among related subspaces. A decision boundary defined in the input space can be considered as being composed of several decision boundaries defined in subspaces that form the entire input space. For the decision boundaries defined in the subspaces, their nonlinearity becomes lower in comparison with the one being encountered when considering the entire input space. Through the weighted LSE estimation instead of using the LSE estimation method, the connections of several NNs can be estimated without interfering with each other. After estimating the weights, the decision boundaries defined in the related subspaces are merged to a single decision boundary by using fuzzy ensemble technique. Several machine learning datasets and one real world application dataset are used to evaluate and validate the proposed fuzzy ensemble classifier. Based on the experimental results, the proposed classifier shows better classification performance when compared with the performance of some selected classifiers.

In this article, a reinforced fuzzy radial basis function neural network (R-FRBFNN) classifier is proposed. It focuses on the development of methodologies of reinforced architecture to improve classification accuracy and enhance the robust capability based on two learning strategies. The two learning strategies are summarized: 1) R-FRBFNN designed via support vector (SV)-based fuzzy C-means (FCM) clustering and softmax-based iterative reweighted least square (IRLS), which concentrate on improving the classification performance of R-FRBFNN; and 2) R-FRBFNN designed via SV-based FCM and softmax-based iterative quadratic programming (IQP), which focus on improving the robust abilities of the R-FRBFNN and reducing the effects of noise and outliers. The essential points of the proposed R-FRBFNN classifier are summarized as follows. a) The proposed R-FRBFNN consists of three phases: condition, conclusion, and inference. b) An SV-based FCM is considered for prioritizing the classification boundary and improving the classification performance of the proposed classifier. c) Three types of polynomials construct the conclusion phase. Two learning techniques are designed to update the coefficients of the polynomials. Softmax-based IRLS is a type of iterative learning technique based on Newton's method. Softmax-based IQP is more robust and avoids the degradation of generalization capabilities caused by outliers and noisy data. d) In the concept of reinforced architecture, SV-based FCM imposes compensation (membership degrees) on learning techniques according to the data characteristics encountered in the inference phase. Experimental results reported for benchmark data and outliers/noisy datasets demonstrate that the proposed classifier shows improved classification performance compared with other previously studied methods.

In this article, we introduce novel reinforced hybrid fuzzy rule-based neural networks (RHFNNs). This article is concerned with the development of the design methodologies of hybrid fuzzy rule-based model for constructing the network structure and enhancing its predictive abilities through the combination of inhomogeneous neurons [i.e., clustering-based polynomial neurons (CPNs) and polynomial neurons (PNs)] and tournament selection. The key points of the proposed RHFNN are enumerated as follows: The first layer of the proposed network consists of CPNs. CPN can effectively reflect the complex nonlinear structure encountered in the data space, and refine (granulate) it with the help of the clustering algorithm. Two types of CPNs including hard C-means (HCM) clustering-based polynomial neuron (HCPN) and fuzzy C-means (FCM) clustering-based polynomial neuron (FCPN) are designed. According to the type of CPN used in the first layer, RHFNN can be categorized into two types, namely, RHFNN based on HCPN (HRHFNN) and RHFNN based on FCPN (FRHFNN). We use PNs to construct the second and consecutive layers. PN can identify and approximate the nonlinear relationship among system's inputs and outputs. A tournament-based performance selection (TPS) algorithm stemming from evolutionary computation is used for selection of neuron. TPS not only ensures that the candidate nodes have sufficient fitting ability but also enhances the individual diversity in the node set and provides the abilities to generate better prediction nodes. In addition,L2-norm regularization is considered to reduce the deviation between coefficients and ameliorate overfitting as well as boost generalization ability. The performance of RHFNN is discussed through a variety of publicly available machine learning datasets. From the experimental results, we conclude that RHFNN achieves the best prediction accuracy on 13 of 15 datasets; the statistical analysis also confirms the superiority of RHFNN.

In this paper, we propose reinforced fuzzy clustering-based ensemble neural networks (FCENNs) classifier. The objective of this paper is focused on the development of the design methodologies of ensemble neural networks classifier for constructing the network structure and enhancing the learning methods of fuzzy clustering-based neural networks through the combination of the probabilistic model and its learning mechanism. The proposed FCENNs classifier takes into consideration a cross-entropy error function to improve learning while L2norm regularization is used to reduce overfitting as well as enhance generalization abilities. The essential points of the proposed reinforced FCENNs classifier can be enumerated as follows: First, in the proposed classifier, the cross-entropy error function is used as a cost function; to do this, a softmax function is applied to represent a categorical distribution located at the nodes of the output layer. Second, the learning mechanism is composed of two parts. First, fuzzy C-means clustering forms the connections (weights) of the hidden layer while the connections of the output layer are adjusted with the aid of the nonlinear least squares method using Newton's method-based learning. Third, L2norm-regularization is considered to avoid the degradation of generalization ability caused by overfitting. The learning mechanism similar to ridge regression is realized by adding L2penalty term to the cross-entropy error function. From the viewpoint of performance improvement achieved through the proposed novel learning method, the design methodology for the ensemble neural networks classifier is discussed and analyzed with the aid of a diversity of two-dimensional synthetic data and machine learning datasets.

This paper is concerned with a new design methodology of a reinforced interval type-2 fuzzy c-means (FCM) based fuzzy classifier (FC). The key point of this study is to reduce the computational complexity of type-2 fuzzy set-based models and to alleviate the deterioration of its generalization abilities through the synergistic effect of two algorithms: First, interval type-2 FCM (IT2FCM) is used in the hidden layer of the network and connections (weights) are adjusted by invoking the least squares error estimation method. Second, an L2-norm regularization is considered in the cost function to avoid the construction of the network suffering from overfitting. In more detail, the hidden layer of the proposed FC is realized by interval type-2 FCM clustering to deal with the factor of uncertainty involved in the problem. This type of clustering is realized by using two values of the fuzzification coefficient resulting in the interval type-2 membership functions. Once completing type reduction, the membership grades of IT2FCM are used as the outputs of the hidden layer. Instead of the backpropagation training, least squares estimator based learning is applied to adjust the functional connection being regarded as linear functions mapping the hidden layer to the output layer. In order to reduce potential overfitting, L2-norm regularization is taken into account. The effectiveness of the proposed classifier is analyzed with the aid of a number of machine learning datasets as well as face image datasets. Thorough comparative studies are also included.

The performance enhancement of system identification of various plastic materials to effectively recycle the waste plastics arises as a key issue studied here. For black plastics, which contain carbon black, one is unable to discriminate it from other materials. To facilitate the identification process, Fourier transform-infrared with attenuated total reflectance is used to carry out qualitative as well as quantitative analysis of black plastics. Since a spectrum obtained in this manner constitutes highly dimensional data, feature reduction becomes necessary to extract sound features and reduce the dimensionality of the original spectrum. In this study, three types of feature extraction techniques are considered: peak detection technique, feature extraction based on the chemical characteristics, and fuzzy transform-based feature extraction to determine sound discriminative features. In order to enhance classification process, fuzzy radial basis function neural networks classifier is constructed; these architectures of the classifiers take advantage of the hybrid technologies. Based upon experimental studies, it is shown that the proposed classification system with the feature extraction techniques exhibits superior performance over the performance reported for the already studied classifiers.

This paper presents a hybrid fuzzy wavelet neural network (HFWNN) realized with the aid of polynomial neural networks (PNNs) and fuzzy inference-based wavelet neurons (FIWNs). Two types of FIWNs including fuzzy set inference-based wavelet neurons (FSIWNs) and fuzzy relation inference-based wavelet neurons (FRIWNs) are proposed. In particular, a FIWN without any fuzzy set component (viz., a premise part of fuzzy rule) becomes a wavelet neuron (WN). To alleviate the limitations of the conventional wavelet neural networks or fuzzy wavelet neural networks whose parameters are determined based on a purely random basis, the parameters of wavelet functions standing in FIWNs or WNs are initialized by using the C-Means clustering method. The overall architecture of the HFWNN is similar to the one of the typical PNNs. The main strategies in the design of HFWNN are developed as follows. First, the first layer of the network consists of FIWNs (e.g., FSIWN or FRIWN) that are used to reflect the uncertainty of data, while the second and higher layers consist of WNs, which exhibit a high level of flexibility and realize a linear combination of wavelet functions. Second, the parameters used in the design of the HFWNN are adjusted through genetic optimization. To evaluate the performance of the proposed HFWNN, several publicly available data are considered. Furthermore a thorough comparative analysis is covered.

In this study, we propose a concept of fuzzy wavelet polynomial neural networks (FWPNNs) based on concepts and constructs of polynomial neural networks and fuzzy wavelet neurons (FWNs). These networks exhibit a rule-based architecture while each rule in the FWN consists of the premise part and consequence part. The premise part is realized by using C-means clustering method, while the consequence part is realized by means of wavelet functions whose parameters are estimated with the aid of the least square method. In some sense, the FWPNN can be regarded as a generalized fuzzy wavelet neural network (FWNN). Unlike Gaussian membership functions that are commonly utilized to implement the premise part of the rules in typical FWNNs, C-means method is employed here to overcome a possible curse of dimensionality. Polynomial neural networks (PNNs) are used to express the nonlinearity of a complex system. Furthermore, the particle swarm optimization is used to optimize the design parameters of the proposed network. Based on the PNNs and FWNNs, the proposed FWPNNs take advantages of these two neural networks: it exhibits the abilities to describe high-order nonlinear relations between input and output variables and it is beneficial to describe models impacted by uncertainty. The proposed FWPNNs are applied for time-series prediction and regression problems (e.g., control of dynamic plants). Several well-known modeling benchmarks including regression and time series are considered to evaluate the performance of the proposed FWPNNs. A comparative analysis shows that the proposed FWPNNs result in better performance when comparing with some previous models reported in the literature.

In this paper, we introduce the design methodology of interval type-2 fuzzy neural networks (IT2FNN). And to optimize the network we use a real-coded genetic algorithm. IT2FNN is the network of combination between the fuzzy neural network (FNN) and interval type-2 fuzzy set with uncertainty. The antecedent part of the network is composed of the fuzzy division of input space and the consequence part of the network is represented by polynomial functions. The parameters such as the apexes of membership function, uncertainty parameter, the learning rate and the momentum coefficient are optimized using genetic algorithm (GA). The proposed network is evaluated with the performance between the approximation and the generalization abilities.

In this paper, we introduce new architectures of genetically oriented fuzzy relation neural networks (FrNNs) and offer a comprehensive design methodology that supports their development. The proposed FrNNs are based on ldquoif-thenrdquo-rule-based networks, with the extended structure of the premise and the consequence parts of the individual rules. We consider two types of the FrNN topologies, which are called FrNN-I and FrNN-II here, depending upon the usage of inputs in the premise and the consequence of fuzzy rules. Three different forms of regression polynomials (namely, constant, linear, and quadratic) are used to construct the consequence of the rules. In order to develop optimal FrNNs, the structure and the parameters are optimized using genetic algorithms (GAs). The proposed methodology is compared when the two development strategies, with separate and simultaneous optimization schemes that involve structure and parameters, are carried out. Given the large search space associated with these FrNN models, we enhance the search capabilities of the GAs by introducing the dynamic variants of genetic optimization. It fully exploits the processing capabilities of the FrNNs by supporting their structural and parametric optimization. To evaluate the performance of the proposed FrNNs, we exploit a suite of several representative numerical examples. A comparative analysis shows that the FrNNs exhibit higher accuracy and predictive capabilities as well as better modeling stability, when compared with some other models that exist in the literature.

In this study, we present a new architecture of a granular neural network and provide a comprehensive design methodology as well as elaborate on an algorithmic setup supporting its development. The proposed neural network relates to a broad category of radial basis function neural networks (RBFNNs) in the sense that its topology involves a collection of receptive fields. In contrast to the standard architectures encountered in RBFNNs, here we form individual receptive fields in subspaces of the original input space rather than in the entire input space. These subspaces could be different for different receptive fields. The architecture of the network is fully reflective of the structure encountered in the training data which are granulated with the aid of clustering techniques. More specifically, the output space is granulated with use of K-means clustering while the information granules in the multidimensional input space are formed by using the so-called context-based fuzzy C-means, which takes into account the structure being already formed in the output space. The innovative development facet of the network involves a dynamic reduction of dimensionality of the input space in which the information granules are formed in the subspace of the overall input space which is formed by selecting a suitable subset of input variables so that this subspace retains the structure of the entire space. As this search is of combinatorial character, we use the technique of genetic optimization [genetic algorithms (GAs), to be more specific] to determine the optimal input subspaces. A series of numeric studies exploiting synthetic data and data coming from the Machine Learning Repository, University of California at Irvine, provide a detailed insight into the nature of the algorithm and its parameters as well as offer some comparative analysis.

In this study, we introduce an advanced architecture of genetically optimized Hybrid Fuzzy Neural Networks (gHFNN) and develop a comprehensive design methodology supporting their construction. A series of numeric experiments is included to illustrate the performance of the networks. The construction of gHFNN exploits fundamental technologies of Computational Intelligence (CI), namely fuzzy sets, neural networks, and genetic algorithms (GAs). The architecture of the gHFNNs results from a synergistic usage of the genetic optimization-driven hybrid system generated by combining Fuzzy Neural Networks (FNN) with Polynomial Neural Networks (PNN). In this tandem, a FNN supports the formation of the condition part of the rule-based structure of the gHFNN. The conclusion part of the gHFNN is designed using PNNs. We distinguish between two types of the simplified fuzzy inference rule-based FNN structures showing how this taxonomy depends upon the type of a fuzzy partition of input variables. As to the conclusion part of the gHFNN, the development of the PNN dwells on two general optimization mechanisms: the structural optimization is realized via GAs whereas in case of the parametric optimization we proceed with a standard least square method-based learning. To evaluate the performance of the gHFNN, we experimented with three representative numerical examples. A comparative analysis demonstrates that the proposed gHFNN come with higher accuracy as well as superb predictive capabilities when compared with other neurofuzzy models.

We introduce a new architecture of Information granulation based genetically optimized hybrid self-organizing fuzzy polynomial neural networks (IG_gHSOFPNN) that is based on a genetically optimized multi-layer perceptron and develop their comprehensive design methodology involving mechanisms of genetic optimization, especially information granulation and genetic algorithms. The architecture of the resulting IG_gHSOFPNN results from a synergistic usage of the hybrid system generated by combining fuzzy polynomial neurons (FPNs)-based self-organizing fuzzy polynomial neural networks(SOFPNN) with polynomial neurons (PNs)-based self-organizing polynomial neural networks (SOPNN). The augmented IG_gHSOFPNN results in a structurally optimized structure and comes with a higher level of flexibility in comparison to the one we encounter in the conventional HSOFPNN. The GA-based design procedure being applied at each layer of IG_gHSOFPNN leads to the selection of preferred nodes (FPNs or PNs) available within the HSOFPNN. In the sequel, two general optimization mechanisms are explored. First, the structural optimization is realized via GAs whereas the ensuing detailed parametric optimization is carried out in the setting of a standard least square method-based learning. The obtained results demonstrate superiority of the proposed networks over the existing fuzzy and neural models.

In this paper, we introduce a new topology of fuzzy polynomial neural networks (FPNNs) that is based on a genetically optimized multilayer perceptron with fuzzy polynomial neurons (FPNs). The study offers a comprehensive design methodology involving mechanisms of genetic optimization, especially those exploiting genetic algorithms (GAs). Let us recall that the design of the "conventional" FPNNs uses an extended group method of data handling (GMDH) and uses a fixed scheme of fuzzy inference (such as simplified, linear, and regression polynomial fuzzy inference) in each FPN of the network. It also considers a fixed number of input nodes (as being selected in advance by a network designer) at FPNs (or nodes) located in each layer. However such design process does not guarantee that the resulting FPNs will always result in an optimal networks architecture. Here, the development of the FPNN gives rise to a structurally optimized topology and comes with a substantial level of flexibility which becomes apparent when contrasted with the one we encounter in the conventional FPNNs. The design of each layer of the FPNN deals with its structural optimization involving a selection of preferred nodes (or FPNs) with specific local characteristics (such as the number of input variables, the order of the polynomial forming a consequent part of fuzzy rules and a collection of the specific subset of input variables) and addresses detailed aspects of parametric optimization. Along this line, two general optimization mechanisms are explored. The structural optimization is realized via GAs. In case of the parametric optimization we proceed with a standard least square method-based learning. Through the consecutive process of such structural and parametric optimization, an optimized and flexible fuzzy neural network becomes generated in a dynamic fashion. To evaluate the performance of the genetically optimized FPNN (gFPNN), we experimented with two time series data (gas furnace and chaotic time series) as well as some synthetic data. A comparative analysis reveals that the proposed FPNN exhibits higher accuracy and superb predictive capability in comparison to some previous models available in the literature.

In this study, we introduce a comprehensive design methodology of hybrid fuzzy controllers (HFCs). The hybrid facet of the proposed architecture of the controller manifests in the form of a convex combination of a standard proportional integral derivative (PID) controller and a fuzzy controller. The design procedure dwells on the use of evolutionary computing (genetic algorithms) and an autotuning algorithm based on estimation modes. The tuning of the scaling factors of the HFC is an essential component of the entire optimization process. Numerical studies are presented and a detailed comparative analysis is included as well.

Experimental software datasets describing software projects in terms of their complexity and development time have been a subject of intensive modeling. A number of various modeling methodologies and modeling designs have been proposed including such development frameworks as neural networks, fuzzy and neurofuzzy models. In this study, we introduce a concept of self-organizing neurofuzzy networks (SONFN), a hybrid modeling architecture combining neurofuzzy networks (NFN) and polynomial neural networks (PNN). For these networks we develop a comprehensive design methodology. The construction of SONFNs takes advantage of the well-established technologies of computational intelligence (CI), namely fuzzy sets, neural networks and genetic algorithms. The architecture of the SONFN results from a synergistic usage of NFNs and PNNs. NFN contributes to the formation of the premise part of the rule-based structure of the SONFN. The consequence part of the SONFN is designed using PNNs. We discuss two types of SONFN architectures whose taxonomy is based on the NFN scheme being applied to the premise part of SONFN. We introduce a comprehensive learning algorithm. It is shown that this network exhibits a dynamic structure as the number of its layers as well as the number of nodes in each layer of the SONFN are not predetermined (as this is the case in a popular topology of a multilayer perceptron). The experimental results include a well-known NASA dataset concerning software cost estimation.

In this study, we propose a hybrid identification algorithm for a class of fuzzy rule-based systems. The rule-based fuzzy modeling concerns structure optimization and parameter identification using the fuzzy inference methods and hybrid structure combined with two methods of optimization theories for nonlinear systems. Two types of inference methods of a fuzzy model concern a simplified and linear type of inference. The proposed hybrid optimal identification algorithm is carried out using a combination of genetic algorithms and an improved complex method. The genetic algorithms determine initial parameters of the membership function of the premise part of the fuzzy rules. In the sequel, the improved complex method (being in essence a powerful auto-tuning algorithm) leads to fine-tuning of the parameters of the respective membership functions. An aggregate performance index with a weighting factor is proposed in order to achieve a balance between performance of the fuzzy model obtained for the training and testing data. Numerical examples are included to evaluate the performance of the proposed model. They are also contrasted with the performance of the fuzzy models existing in the literature. © 2002 John Wiley & Sons, Inc.

We propose a hybrid architecture based on a combination of fuzzy systems and polynomial neural networks. The resulting Hybrid Fuzzy Polynomial Neural Networks (HFPNN) dwells on the ideas of fuzzy rule-based computing and polynomial neural networks. The structure of the network comprises of fuzzy polynomial neurons (FPNs) forming the nodes of the first (input) layer of the HFPNN and polynomial neurons (PNs) that are located in the consecutive layers of the network. In the FPN (that forms a fuzzy inference system), the generic rules assume the form "if A then y = P(x) " where A is fuzzy relation in the condition space while P(x) is a polynomial standing in the conclusion part of the rule. The conclusion part of the rules, especially the regression polynomial uses several types of high-order polynomials such as constant, linear, quadratic, and modified quadratic. As the premise part of the rules, both triangular and Gaussian-like membership functions are considered. Each PN of the network realizes a polynomial type of partial description (PD) of the mapping between input and out variables. HFPNN is a flexible neural architecture whose structure is based on the Group Method of Data Handling (GMDH) and developed through learning. In particular, the number of layers of the PNN is not fixed in advance but is generated in a dynamic way. The experimental part of the study involves two representative numerical examples such as chaotic time series and Box-Jenkins gas furnace data.

We introduce a concept of fuzzy polynomial neural networks (FPNNs), a hybrid modeling architecture combining polynomial neural networks (PNNs) and fuzzy neural networks (FNNs). The development of the FPNNs dwells on the technologies of computational intelligence (CI), namely fuzzy sets, neural networks, and genetic algorithms. The structure of the FPNN results from a synergistic usage of FNN and PNN. FNNs contribute to the formation of the premise part of the rule-based structure of the FPNN. The consequence part of the FPNN is designed using PNNs. The structure of the PNN is not fixed in advance as it usually takes place in the case of conventional neural networks, but becomes organized dynamically to meet the required approximation error. We exploit a group method of data handling (GMDH) to produce this dynamic topology of the network. The performance of the FPNN is quantified through experimentation that exploits standard data already used in fuzzy modeling. The obtained experimental results reveal that the proposed networks exhibit high accuracy and generalization capabilities in comparison to other similar fuzzy models.