Abstract Software managers constantly look out for methods that ensure cost effective development of good quality software products. An important means of accomplishing this is by allocating more resources to weak classes of a software product, which are prone to changes. Therefore, correct prediction of these change‐prone classes is critical. Though various researchers have investigated the performance of several algorithms for identifying them, the search for an optimum classifier still persists. To this end, this study critically investigates the use of six Heterogenous Ensemble Classifiers (HEC) for Software Change Prediction (SCP) by empirically validating datasets obtained from 12 open‐source software systems. The results of the study are statistically assessed using three robust performance indicators (AUC, F‐measure and Mathew Correlation Coefficient) in two different validation scenarios (within project and cross‐project). They indicate the superiority of Average Probability Voting Ensemble, a heterogenous classifier for determining change‐proneness in the investigated systems. The average AUC values of software change prediction models developed using this ensemble classifier exhibited an improvement of 3%‐9% and 3%‐11% respectively when compared with its base learners and homogeneous counter parts. Similar observations were inferred using other investigated performance measures. Furthermore, the evidence obtained from the results suggests that the change in number of base learners or type of meta‐learner does not exhibit significant change in the performance of corresponding heterogenous ensemble classifiers.

Class Imbalance Problem is the major issue in machine intelligence producing biased classifiers that work well for the majority class but have a relatively poor performance for the minority class. To ensure the development of accurate prediction models, it is essential to deal with the class imbalance problem. In this paper, the class imbalance problem is handled using focused undersampling techniques viz. Cluster Based, Tomek Link and Condensed Nearest Neighbours which equalize the number of instances of the two types of classes by undersampling the majority class based on some particular criteria. This is in contrast to random undersampling where the data samples are selected randomly from the majority class leading to underfitting and loss of some important datapoints. To fairly compare and evaluate the performance of focused undersampling approaches, prediction models are constructed using popular machine learning classifiers like K-Nearest Neighbor, Decision Tree and Naive Bayes. The results have shown that Decision Tree outperformed other machine learning techniques. Comparing and contrasting the undersampling approaches for Decision Tree concluded Condensed Nearest Neighbours to be best amongst others.