ABSTRACT Instance selection is an important data preprocessing step in data mining that removes noisy data from training datasets to build more effective models and reduces dataset size to improve computational efficiency during model training. Traditional instance selection algorithms have varying strengths and weaknesses, making it difficult for any single method to consistently achieve optimal performance across diverse datasets. Therefore, in this paper, two novel ensemble instance selection methods are proposed, namely Single‐Stage Parallel Ensemble (SSPE) and Two‐Stage Serial Ensemble (TSSE), which are based on combining multiple instance selection algorithms in the parallel and serial manners. The proposed ensemble approaches aim to improve data reduction and classification performance. Extensive experiments are conducted on 17 medical datasets of varying sizes, evaluating four instance selection algorithms, CNN, ENN, IPF, and GA, alongside three classifiers: KNN, SVM, and RF. Results demonstrate that certain SSPE combinations, particularly the union of ENN and IPF, outperform single baseline algorithms in classification accuracy and AUC while maintaining effective data reduction. Although TSSE achieves higher reduction rates, its classification performance is inferior to SSPE. Overall, the proposed methods serve as effective preprocessing tools for medical data and provide a strong baseline for future ensemble instance selection research.

ABSTRACT Dimensionality reduction plays a vital role in enhancing the performance of data classification tasks by reducing the complexity of the feature space. This study examines the effectiveness of integrating dimensionality reduction techniques with classification algorithms across four strategic configurations: (1) machine learning (ML)‐based dimensionality reduction with ML classifiers, (2) deep learning (DL)‐based dimensionality reduction with DL classifiers, and two heterogeneous combinations that mix ML and DL methods. Using 20 benchmark datasets from diverse domains, with feature dimensions ranging from 44 to 19,993, we systematically evaluate and compare these configurations. The dimensionality reduction methods include three ML‐based feature selection techniques, Genetic Algorithm (GA), Information Gain (IG), and the C4.5 decision tree, and four DL‐based feature extraction approaches, Autoencoder (AE), Sparse Autoencoder (SAE), Denoising Autoencoder (DAE), and Variational Autoencoder (VAE). For classification, Support Vector Machine (SVM) and k‐Nearest Neighbours (KNN) are used as ML classifiers, while Multilayer Perceptron (MLP) and Deep Belief Network (DBN) serve as DL classifiers. Experimental results show that SAE consistently produces the most compact feature sets and improves classification performance, with the SAE + MLP combination achieving the best overall results. Furthermore, we explore ensemble dimensionality reduction strategies that integrate multiple algorithms. Although the best ensemble approach slightly outperforms the SAE + MLP model, the observed performance improvements are not statistically significant, that is, 0.839 versus 0.836 for AUC rates. In addition, SAE achieves a significantly higher dimensionality reduction rate compared to the best ensemble method (63% vs. 18%).

Many real world domain problem datasets are class imbalanced where the number of data in a given class is much less than in the other classes. In related literatures, under- and over-sampling techniques are widely used techniques to re-balance the class imbalanced datasets. However, their limitations include the risk of removing representative majority class data samples and the overfitting problem because of generating a large number of synthetic minority class data samples. Therefore, a novel approach, namely Majority Re-sampling visa Sub-class Clustering (MRSC) is introduced. It uses a clustering algorithm to group the majority class data into several clusters, i.e. sub-classes. Then, a new training set containing multiple sub-classes and a minority class is produced, after which the classifier is trained using this new multi-class dataset which has a lower imbalance ratio than the original dataset. The experimental results obtained using 44 two-class imbalanced datasets show that MRSC combined with the k-NN classifiers, including single and ensemble classifiers, significantly outperforms the other classifiers as well as seven state-of-the-art re-sampling approaches. Moreover, for the clustering algorithms based on affinity propagation and k-means, very similar results can be produced, without significant differences in performance, which indicate the stability of MRSC.

In this paper, we focus on comparing the imputation performance of different deep and machine learning techniques on nine related datasets containing different missing rates ranging from 10% to 50%. Moreover, since each feature value ranges differently, such as total liability/equity ratio and earnings per share, the effect of feature normalisation on the imputation results is also examined to see whether normalising the feature values after missing value imputation can improve the prediction model performance. The experimental results show that the deep neural network technique does not necessarily perform better than the traditional machine learning technique for missing value imputation. In particular, the random forest imputation model performs the best, whereas the k-nearest neighbour method is the second best imputation model in terms of the AUC rates and type II errors. The performance in improvement of prediction models after performing feature normalisation is heavily dependent on the chosen classification technique. There is thus no need to consider the normalisation step when the random forest classifier is used. It is found that the deep neural network and support vector machine classifiers can significantly outperform those without feature normalisation.

Abstract Instance is important in data analysis and mining; it filters out unrepresentative, redundant, or noisy data from a given training set to obtain effective model learning. Various instance selection algorithms are proposed in the literature, and their potential and applicability in data cleaning and preprocessing steps are demonstrated. For multiclass classification datasets, the existing instance selection algorithms must deal with all the instances across the different classes simultaneously to produce a reduced training set. Generally, every multiclass classification dataset can be regarded as a complex domain problem, which can be effectively solved using the divide‐and‐conquer principle. In this study, the one‐versus‐all (OVA) and one‐versus‐one (OVO) decomposition approaches were used to decompose a multiclass dataset into multiple binary class datasets. These approaches have been widely employed when constructing the classifier but have never been considered in instance selection. The results of instance selection performance obtained with the OVA, OVO, and baseline approaches were assessed and compared for 20 different domain multiclass datasets as the first study and five medical domain datasets as the validation study. Furthermore, three instance selection algorithms were compared, including IB3, DROP3, and GA. The results demonstrate that using the OVO approach to perform instance selection can make the support vector machine (SVM) and k‐nearest neighbour (k‐NN) classifiers perform significantly better than the OVA and baseline approaches in terms of the area under the ROC curve (AUC) rate, regardless of the instance selection algorithm used. Moreover, the OVO approach can provide reasonably good data reduction rates and processing times, which are all better than those of the OVA approach.

Online review helpfulness prediction is an important research issue in electronic commerce and data mining. However, the collected datasets used for the analysis and prediction of the helpfulness of online reviews often contain some missing attribute values, such as reviewer background and rating information. In related literatures, many studies have either used the case deletion approach to remove the data containing missing values or considered the imputation of missing values by the mean/mode method. However, none of them consider the direct handling approach without missing value imputation for online review datasets by decision tree-related techniques. Therefore, in this paper, we investigate the suitability of different types of approaches to solve the incomplete dataset problem of online reviews. Specifically, for missing value imputation, several supervised learning techniques including MICE, KNN, SVM, and CART are examined. Moreover, for the direct handling approach without missing value imputation, CART is also performed for this task. The experimental results based on the TripAdvisor dataset for review helpfulness prediction show that the approach where incomplete online review datasets are handled directly without imputation by CART significantly outperforms the other approaches, including case deletion and missing value imputation approaches.

Abstract Feature selection is an important data preprocessing step for the construction of an effective bankruptcy prediction model. The prediction performance can be affected by the employed feature selection and classification techniques. However, there have been very few studies of bankruptcy prediction that identify the best combination of feature selection and classification techniques. In this study, two types of feature selection methods, including filter‐ and wrapper‐based methods, are considered, and two types of classification techniques, including statistical and machine learning techniques, are employed in the development of the prediction methods. In addition, bagging and boosting ensemble classifiers are also constructed for comparison. The experimental results based on three related datasets that contain different numbers of input features show that the genetic algorithm as the wrapper‐based feature selection method performs better than the filter‐based one by information gain. It is also shown that the lowest prediction error rates for the three datasets are provided by combining the genetic algorithm with the naïve Bayes and support vector machine classifiers without bagging and boosting.

Abstract While there is an ample amount of medical information available for data mining, many of the datasets are unfortunately incomplete – missing relevant values needed by many machine learning algorithms. Several approaches have been proposed for the imputation of missing values, using various reasoning steps to provide estimations from the observed data. One of the important steps in data mining is data preprocessing, where unrepresentative data is filtered out of the data to be mined. However, none of the related studies about missing value imputation consider performing a data preprocessing step before imputation. Therefore, the aim of this study is to examine the effect of two preprocessing steps, feature and instance selection, on missing value imputation. Specifically, eight different medical‐related datasets are used, containing categorical, numerical and mixed types of data. Our experimental results show that imputation after instance selection can produce better classification performance than imputation alone. In addition, we will demonstrate that imputation after feature selection does not have a positive impact on the imputation result.

When the hospitalization periods of an unexpectedly high number of patients are extended, the income of a hospital is substantially affected and the rate of hospital bed occupancy increases. Because none of the currently available score systems can be used to evaluate the possibility that patients who require urgent surgery in a single department prolong their length of stay (LOS), this study attempts to build a prolonged LOS prediction model utilizing a number of supervised learning techniques. This study involved analyzing the complete historical medical records and lab data of 897 clinical cases in which surgeries were performed by general surgery physicians. These clinical cases were divided into an urgent operation (UO) group comprising 462 cases and a non-UO group comprising 434 cases to develop a prolonged LOS prediction model by using several supervised learning techniques. The results indicated that the random forest method constituted the most accurate and stable prediction model. This study demonstrated that supervised learning techniques can be used to analyze patient medical records to accurately predict a prolonged LOS, thus, supervised learning techniques can serve as valuable reference tools for patient prognoses. The developed prediction models can facilitate the decision making of physicians when patients require surgery and increase patient safety.

Abstract Companies can use customer segmentation to group customers with similar characteristics together and identify the differences between groups to develop marketing strategies. This study investigates the problem of customer segmentation in relation to automotive customer relationship management and presents a real case study of an automobile dealer in Taiwan. Although several past studies have adopted different clustering techniques with which to group customer attributes, few have simultaneously considered customer transaction behaviour and customer satisfaction variables. In addition, most previous work has used only a single clustering method for customer segmentation, which results in unreliable results and leads to inadequate marketing decisions. Therefore, in this study, we consider two clustering techniques, k‐means and expectation maximization, and compare their results for correctness. The experimental results show that four customer groups are identified with both clustering methods: loyal, potential, VIP and churn customer groups. Based on the segmentation results, several customized marketing strategies aimed at each of the four customer groups are suggested to improve the quality of services for effective customer relationship management.

Pseudorelevance feedback (PRF) was proposed to solve the limitation of relevance feedback (RF), which is based on the user‐in‐the‐loop process. In PRF, the top‐k retrieved images are regarded as PRF. Although the PRF set contains noise, PRF has proven effective for automatically improving the overall retrieval result. To implement PRF, the Rocchio algorithm has been considered as a reasonable and well‐established baseline. However, the performance of Rocchio‐based PRF is subject to various representation choices (or factors). In this article, we examine these factors that affect the performance of Rocchio‐based PRF, including image‐feature representation, the number of top‐ranked images, the weighting parameters of Rocchio, and similarity measure. We offer practical insights on how to optimize the performance of Rocchio‐based PRF by choosing appropriate representation choices. Our extensive experiments on NUS‐WIDE‐LITE and Caltech 101 + Corel 5000 data sets show that the optimal feature representation is color moment + wavelet texture in terms of retrieval efficiency and effectiveness. Other representation choices are that using top‐20 ranked images as pseudopositive and pseudonegative feedback sets with the equal weight (i.e., 0.5) by the correlation and cosine distance functions can produce the optimal retrieval result.

For financial institutions, the ability to predict or forecast business failures is crucial, as incorrect decisions can have direct financial consequences. Bankruptcy prediction and credit scoring are the two major research problems in the accounting and finance domain. In the literature, a number of models have been developed to predict whether borrowers are in danger of bankruptcy and whether they should be considered a good or bad credit risk. Since the 1990s, machine-learning techniques, such as neural networks and decision trees, have been studied extensively as tools for bankruptcy prediction and credit score modeling. This paper reviews 130 related journal papers from the period between 1995 and 2010, focusing on the development of state-of-the-art machine-learning techniques, including hybrid and ensemble classifiers. Related studies are compared in terms of classifier design, datasets, baselines, and other experimental factors. This paper presents the current achievements and limitations associated with the development of bankruptcy-prediction and credit-scoring models employing machine learning. We also provide suggestions for future research.

Abstract:Bankruptcy prediction and credit scoring are the two important problems facing financial decision support. The multilayer perceptron (MLP) network has shown its applicability to these problems and its performance is usually superior to those of other traditional statistical models. Support vector machines (SVMs) are the core machine learning techniques and have been used to compare with MLP as the benchmark. However, the performance of SVMs is not fully understood in the literature because an insufficient number of data sets is considered and different kernel functions are used to train the SVMs. In this paper, four public data sets are used. In particular, three different sizes of training and testing data in each of the four data sets are considered (i.e. 3:7, 1:1 and 7:3) in order to examine and fully understand the performance of SVMs. For SVM model construction, the linear, radial basis function and polynomial kernel functions are used to construct the SVMs. Using MLP as the benchmark, the SVM classifier only performs better in one of the four data sets. On the other hand, the prediction results of the MLP and SVM classifiers are not significantly different for the three different sizes of training and testing data.

Many users of image retrieval systems would prefer to express initial queries using keywords. However, manual keyword indexing is very time-consuming. Therefore, a content-based image retrieval system which can automatically assign keywords to images would be very attractive. Unfortunately, it has proved very challenging to build such systems, except where either the image domain is restricted or the keywords relate only to low-level concepts such as color. This article presents a novel image indexing and classification system, called CLAIRE (CLAssifying Images for REtrieval), composed of one image processing module and three modules of support vector machines for color, texture, and high-level concept classification for keyword assignment. The experimental prototype system described here assigns up to five keywords selected from a controlled vocabulary of 60 terms to each image. The system is trained offline by 1639 examples from the Corel stock photo library. For evaluation, five judges reviewed a sample of 800 unknown images to identify which automatically assigned keywords were actually relevant to the image. The system proved to have an 80% probability to assign at least one relevant keyword to an image.

Use of semantic content is one of the major issues which needs to be addressed for improving image retrieval effectiveness. We present a new approach to classify images based on the combination of image processing techniques and hybrid neural networks. Multiple keywords are assigned to an image to represent its main contents, i.e. semantic content. Images are divided into a number of regions and colour and texture features are extracted. The first classifier, a self-organising map (SOM) clusters similar images based on the extracted features. Then, regions of the representative images of these clusters were labeled and used to train the second classifier, composed of several support vector machines (SVMs). Initial experiments on the accuracy of keyword assignment for a small vocabulary are reported.