Abstract Class imbalance is a substantial challenge in classifying many real-world cases. Synthetic over-sampling methods have been effective to improve the performance of classifiers for imbalance problems. However, most synthetic over-sampling methods generate synthetic instances within the convex hull formed by the existing minority instances as they only concentrate on the minority class and ignore the vast information provided by the majority class. They also often do not perform well for extremely imbalanced data, as fewer minority instances mean less information with which to generate synthetic instances. Moreover, existing methods that generate synthetic instances using the majority class distributional information cannot perform effectively when the majority class has a multi-modal distribution. We propose a new method to generate diverse and adaptable synthetic instances using Synthetic Over-sampling with Minority and Majority classes (SOMM). SOMM generates synthetic instances diversely within the minority data space. It updates the generated instances adaptively to the neighbourhood including both classes. Thus, SOMM performs well for imbalance problems. We examine the performance of SOMM for binary multiclass imbalance classification problems for different imbalance levels. The empirical results and nonparametric statistical testing show the superiority of SOMM compared to existing methods. We also discuss the strengths and limitations of SOMM through visualisations.

Intelligent techniques, including artificial intelligence and deep learning, normally perform on complete data without missing data. Multiple imputation is indispensable for addressing missing data resulting in unbiased estimates and dealing with uncertainty by providing more valid results. Most state-of-the-art techniques focus on high-missing rates (around 50%–60%) and short missing gaps, while imputation for extreme missing gaps and missing rates is an important challenge for multivariate time-series data generated through the Internet of Things (IoT). Hence, we propose an lightweight-window-portion-based multiple imputation (LWPMI) based on multivariate variables, correlation, data fusion, regression, and multiple imputations. We conduct extensive experiments by generating extreme missing gaps and high-missing rates ranging from 10% to 90% on data generated by sensors. We also investigate different sets of feature to examine how LWPMI works when features have high, weak, or a mixture of high and weak correlation. All the obtained results prove LWPMI outperforms baseline techniques in preserving pattern, structure, and trend in both 90% extreme missing gap and missing rates.

Bagging is a common approach in ensemble learning that generates a group of classifiers through bootstrapping for classification tasks. Despite its wide applications, generating redundant classifiers remains a central challenge in bagging. In recent years, many selective bagging models have been presented to deal with this challenge. These models mostly focused on the accuracy of classifiers and the diversity among them. Despite the importance of uncertainty in the performance of ensemble classifiers, this criterion has been neglected in selective bagging models. In this paper, we propose a two-stage selective bagging model. In the first stage, we formalize the selective bagging problem as a bi-objective optimization model considering both the uncertainty and accuracy of classifiers. We propose an adaptive evolutionary Two-Arch2 algorithm, named Diverse-Two-Arch2, to solve the bi-objective model. The output of this stage is a subset of classifiers that are diverse, certain about correct predictions, and uncertain about incorrect predictions. While most selective bagging models focus on the selection of a fixed subset of classifiers for all test samples (static approach), our proposed model has a dynamic approach to the selection process. So, in the second stage of the model, we select only certain classifiers to make an ensemble prediction for each test sample. Experimental results on twenty data sets and comparing with two ensemble models, and five state-of-the-art dynamic selective bagging models show the outperformance of the proposed model. We also compare the performance of the proposed Diverse-Two-Arch2 to alternative evolutionary computation methods.

We are very pleased to introduce this special issue on multiobjective evolutionary optimization for machine learning (MOML). Optimization is at the heart of many machine-learning techniques. However, there is still room to exploit optimization in machine learning. Every machine-learning technique has hyperparameters that can be tuned using evolutionary computation and optimization, considering normally multiple criteria, such as bias, variance, complexity, and fairness in model selection. Multiobjective evolutionary optimization can help meet these criteria for optimizing machine-learning models. Some of the existing approaches address these multiple criteria by transforming the problem into a single-objective optimization problem. However, multiobjective optimization models are able to outperform single-objective ones in contributing to multiple intended objectives (criteria). In recent years, evolutionary computation has been shown to be the premier method for solving multiobjective optimization problems (MOPs), producing both optimal and diverse solutions beyond the capabilities of other heuristics. This is particularly true for very large solution spaces, which is the case in real-world machine-learning problems with many features.

In many real-life classification tasks, the issue of imbalanced data is commonly observed. The workings of mainstream machine learning algorithms typically assume the classes amongst underlying datasets are relatively well-balanced. The failure of this assumption can lead to a biased representation of the models’ performance. This has encouraged the incorporation of re-sampling techniques to generate more balanced datasets. However, mainstream re-sampling methods fail to account for the distribution of minority data and the diversity within generated instances. Therefore, in this paper, we propose a data-generation algorithm, Cluster-based Diversity Over-sampling (CDO), to consider minority instance distribution during the process of data generation. Diversity optimisation is utilised to promote diversity within the generated data. We have conducted extensive experiments on synthetic and real-world datasets to evaluate the performance of CDO in comparison with SMOTE-based and diversity-based methods (DADO, DIWO, BL-SMOTE, DB-SMOTE, and MAHAKIL). The experiments show the superiority of CDO.

This study concentrates on clustering problems and aims to find compact clusters that are informative regarding the outcome variable. The main goal is partitioning data points so that observations in each cluster are similar and the outcome variable can be predicted using these clusters simultaneously. We model this semi-supervised clustering problem as a multi-objective optimization problem with considering deviation of data points in clusters and prediction error of the outcome variable as two objective functions to be minimized. For finding optimal clustering solutions, we employ a non-dominated sorting genetic algorithm II approach and local regression is applied as the prediction method for the output variable. For comparing the performance of the proposed model, we compute seven models using five real-world data sets. Furthermore, we investigate the impact of using local regression for predicting the outcome variable in all models and examine the performance of the multi-objective models compared to single-objective models.

This paper primarily presents two methods of ranking aggregated fuzzy numbers from intervals using the Interval Agreement Approach (IAA). The two proposed ranking methods within this study contain the combination and application of previously proposed similarity measures, along with attributes novel to that of aggregated fuzzy numbers from interval-valued data. The shortcomings of previous measures, along with the improvements of the proposed methods, are illustrated using both a synthetic and real-world application. The real-world application regards the Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS) algorithm, modified to include both the previous and newly proposed methods.

Machine learning techniques have been developed to learn from complete data. When missing values exist in a dataset, the incomplete data should be preprocessed separately by removing data points with missing values or imputation. In this paper, we propose an online approach to handle missing values while a classification model is learnt. To reach this goal, we develop a multi-objective optimization model with two objective functions for imputation and model selection. We also propose three formulations for imputation objective function. We use an evolutionary algorithm based on NSGA II to find the optimal solutions as the Pareto solutions. We investigate the reliability and robustness of the proposed model using experiments by defining several scenarios in dealing with missing values and classification. We also describe how the proposed model can contribute to medical informatics. We compare the performance of three different formulations via experimental results. The proposed model results get validated by comparing with a comparable literature.

This contribution reviews critically the existing entropy measures for probabilistic hesitant fuzzy sets (PHFSs), and demonstrates that these entropy measures fail to effectively distinguish a variety of different PHFSs in some cases. In the sequel, we develop a new axiomatic framework of entropy measures for probabilistic hesitant fuzzy elements (PHFEs) by considering two facets of uncertainty associated with PHFEs which are known as fuzziness and nonspecificity. Respect to each kind of uncertainty, a number of formulae are derived to permit flexible selection of PHFE entropy measures. Moreover, based on the proposed PHFE entropy measures, we introduce some entropy-based distance measures which are used in the portion of comparative analysis. Eventually, the proposed PHFE entropy measures and PHFE entropy-based distance measures are applied to decision making in the strategy initiatives where their reliability and effectiveness are verified.

This paper proposes a new iterative fuzzy clustering (IFC) algorithm to impute missing values of datasets. The information provided by fuzzy clustering is used to update the imputed values through iterations. The performance of the IFC algorithm is examined by conducting experiments on three commonly used datasets and a case study on a city mobility database. Experimental results show that the IFC algorithm not only works well for datasets with a small number of missing values but also provides an effective imputation result for datasets where the proportion of missing data is high.

System reliability modeling needs a large amount of data to estimate the parameters. In addition, reliability estimation is associated with uncertainty. This paper aims to propose a new method to evaluate the failure behavior and reliability of a large system using failure modes, effects, and criticality analysis (FMECA). Therefore, qualitative data based on the judgment of experts are used when data are not sufficient. The subjective data of failure modes and causes have been aggregated through the system to develop an overall failure index (OFI). This index not only represents the system reliability behavior, but also prioritizes corrective actions based on improvements in system failure. In addition, two optimization models are presented to select optimal actions subject to budget constraint. The associated costs of each corrective action are considered in risk evaluation. Finally, a case study of a manufacturing line is introduced to verify the applicability of the proposed method in industrial environments. The proposed method is compared with conventional FMECA approach. It is shown that the proposed method has a better performance in risk assessment. A sensitivity analysis is provided on the budget amount and the results are discussed.

A reliability optimization model is proposed in this paper for multi-state weighted k-out-of-n systems. In this model, income generated by components through each functioning period is used as a reliability index. Therefore, time value of money is used in the presented optimization model to estimate both system's reliability and cost. The system reliability is evaluated by Universal Generating Function (UGF). The model's objective function is to maximize the net present value (NPV) of the system. Therefore, it would maximize the system reliability and minimize the system cost simultaneously. A numerical example is presented in this paper to illustrate the model by finding the optimal design of the system, and the best time for maintenance plan. Also, a discussion is provided based on the result.