As an important data preprocessing technique, feature selection aims to identify useful features and therefore reduce the dimensionality of the data. Recent studies have witnessed that evolutionary computation methods show significant potential in solving feature selection tasks. However, existing methods still encounter challenges due to the high computational costs, particularly when handling high-dimensional datasets. To tackle these issues, this work proposes a new evolutionary feature selection method with a divide-and-conquer strategy. The proposed method transforms a high-dimensional feature selection task into multiple low-dimensional sub-tasks. Multiple sub-populations corresponding to the multiple sub-tasks are evolved simultaneously. To further improve the quality of the generated candidate feature subsets, a set-based population update mechanism is introduced. Furthermore, diverse collaborative coevolution strategies are systematically explored and analyzed. The experiments conducted on 12 real-world classification datasets demonstrate that the proposed method achieves superior performance compared to 9 state-of-the-art feature selection methods. The results reveal that the proposed method is able to select smaller feature subsets while achieving higher classification accuracy across the majority of the used datasets with a reasonable low training time.

In fault diagnosis tasks, samples collected under different working conditions can lead to variations in the distribution of features extracted by deep learning model. This will lead to performance degradation and weak generalization ability when a model trained under a single working condition is applied to other conditions. Domain adaptation methods have been developed to tackle the domain shift. The majority of existing methods to fault diagnosis under variable conditions focus on aligning the distribution of source and target domains, while neglecting the domain shift mitigation capabilities of pretrained networks and often resulting in suboptimal intra-class alignment. To overcome these limitations, this study proposes a novel two-stage framework for cross-domain fault diagnosis, which integrates supervised contrastive learning pre-training with downstream joint distribution alignment (SCL-JDA). During the pre-training phase, the deep learning model is trained using a supervised contrastive learning strategy, making the extracted features more discriminative for different classes. In the downstream task processing, a pseudo-labeling method is designed to assign high-confidence pseudo-labels to the target domain so that it can participate in the training to increase the generalization ability of the model. The superiority of SCL-JDA are substantiated through a series of fault diagnosis-related experiments conducted on bearing datasets.

Dynamic constrained multiobjective optimization problems (DCMOPs) are widely existed in real-world applications and emerged as a prominent research focus in the evolutionary computation community. Current studies on DCMOPs face two main challenges: limited accuracy in population prediction, and a lack of effective strategies to improve static optimizer performance. To tackle these challenges, this paper proposes a dynamic constrained multiobjective evolutionary algorithm based on multipopulation prediction and dynamic fusion ranking. Specifically, an efficient computer-vision-inspired point set registration method, named coherent point drift, is introduced to align individuals across successive environments. With the correspondences between two environments, the solution trajectory tracking problem is transformed as a point set registration problem. Based on the observed trajectory of solutions, the Pareto-optimal set or Pareto-optimal front in the new environment can be predicted. Additionally, this paper highlights the importance of task-specific multipopulation prediction. After analysis the specific tasks of each population, different initial populations tailored to the tasks are predicted by the proposed prediction method. Finally, a dynamic fusion based two-ranking environmental selection strategy is proposed for the auxiliary task. This strategy dynamically integrates experience-based and constraint-based approaches, improving the auxiliary population evolutionary efficiency and its alignment with the main task. The superiority of proposed algorithm is validated through extensive experiments on a series of benchmark problems and a real-world raw ore allocation problem.

The evaluation of objective and constraint involving expensive simulations or physical experiments with multiple optimal solutions is referred to as expensive constrained multimodal optimization problems (ECMMOPs). Under limited real function evaluations (FEs), it is challenging to find multiple optimal solutions accurately while satisfying constraints. To address these issues, this article studies a self-clustering particle swarm optimization algorithm with modal detection informed classification evaluation (MDICE) to solve ECMMOPs. To deal with multimodality, a surrogate-assisted self-clustering update mechanism is first designed to update individuals in each modality. Following that, a novel modal detection strategy is proposed based on the awareness of fitness landscapes to identify all potential modal seeds. For better utilization of FEs, a modality-guided classification evaluation strategy is designed to efficiently generate infilling samples for each constraint and modality. Moreover, to address the complex constraints, a surrogate-assisted feasibility search strategy is developed to quickly search for feasible solutions at a lower evaluation cost. Experimental results on 33 benchmark functions with various characteristics indicate that MDICE outperforms four state-of-the-art surrogate-assisted evolutionary algorithms.

Expensive constrained optimization problems (ECOPs), which frequently arise in real-world engineering optimization, are often limited by the number of evaluations. Using surrogate-assisted evolutionary algorithms (EAs) to reduce computational costs is a common approach. However, surrogate model errors are inevitable and often mislead the search direction of EAs. Most existing algorithms overlook the inevitability of such errors and attempt to minimize them through techniques like data selection, which might be ineffective for problems with highly complex constraint and objective functions. Therefore, we propose a dual-surrogate competition assisted EA with triggered constraint-first search (DC-TCFS) for ECOPs, aiming to reduce the misleading effects of surrogate models on the evolutionary process. In this study, two surrogate models are used to assist local searches through competition, effectively mitigating the impact of errors from a single surrogate model. A triggered constraint-first search method is proposed to quickly identify a feasible solution for problems with complex constraints. Additionally, an adaptive sampling criterion is designed to guide the algorithm toward solutions that are more beneficial to the evolutionary process. Experimental results demonstrate that the proposed algorithm outperforms state-of-the-art methods across various benchmark problems, real-world problems and a motor design optimization problem, highlighting its effectiveness in solving ECOPs.

Imbalanced datasets are a major challenge in industrial fault diagnosis because the majority of data belong to the non-fault class, while fault instances constitute a small minority. Genetic Programming (GP) has shown great potential in handling imbalanced classification tasks due to its ability to evolve classifiers and automatically optimize decision rules. Traditional GP methods for imbalanced classification often rely on fixed decision thresholds (e.g., 0 or 0.5). However, such thresholds fail to adapt to varying data distributions, resulting in limited accuracy in fault diagnosis. While threshold-free methods, such as GP using the Area Under the Curve (AUC) and its variants as fitness functions, have demonstrated effectiveness, practical applications in industrial systems often require explicit thresholds to generate accurate class labels. This paper introduces a GP-based approach with a simplified AUC variant as the fitness function and a dynamic threshold search mechanism. By adaptively optimizing thresholds during evolution, the method improves minority class detection. Experiments on public fault diagnosis datasets with varying imbalance ratios demonstrate that the proposed approach consistently outperforms traditional GP methods.

Dynamic multimodal optimization problems (DMMOPs) represent the multimodal optimization problems that the optimal solution changes over time. Due to the wide application of DMMOPs in reality, some related algorithms have been proposed in recent years. Most existing algorithms employ a single dynamic response mechanism and embed it in existing multi-modal evolutionary algorithms. However, these algorithms often perform limited when environmental change involves multiple types, and they fail to consider utilizing historical information to assist static multimodal optimizers. To solve these issues, this paper proposes historical information-assisted dynamic response integration and adaptive niche methods (HIA-DRI-AN) for dynamic multi-modal optimization. In HIA-DRI-AN, an dynamic response integration method with adaptive adjustment mechanism is proposed for generating the initial population when the change happens. This method considers the change types of DMMOPs, and integrates targeted dynamic response mechanisms to respond to the different change types. Also, this method can adaptively self-adjust to balance the convergence and diversity of the initial population depending on the integrated response mechanism’s performance in historical environments. Furthermore, a niching adaptive division strategy is proposed to enhance the performance of the static optimizer. The strategy dynamically divides niches based on the integrated response mechanism’s performance and the current evolutionary stage, which can adjust the preference for diversity and convergence during evolution. The comprehensive experimental results on 24 test functions show that HIADRI-AN is superior compared to some state-of-the-art dynamic multimodal algorithms.

Dynamic constrained multiobjective optimization problems (DCMOPs) are widely encountered in real-world applications and have attracted increasing attention in the evolutionary computation community. Existing studies primarily focus on the population initialization, but disregard to improve the evolution process. In DCMOPs, the knowledge is undoubtedly more abundant as the existence of historical environments. Therefore, extracting and utilizing useful knowledge from historical environments can further improve the evolution process. In this article, a history-assisted two-state auxiliary task collaboration approach is proposed to solve DCMOPs by conducting a more effective auxiliary task. Specifically, the algorithm indicates that constrained Pareto-optimal front (CPF) in similar historical environments is more suitable as an auxiliary task for current evolution, as it is closer to the current CPF. To identify the most suitable environment from extensive historical environments, a novel prediction-based similar environment identification method is proposed for the auxiliary task. To fully utilize the novel auxiliary task, an experience-driven two-state environmental selection strategy is proposed, which conditionally considers both historical and current information. In this strategy, individuals in state A are inclined to promote toward the historical CPF while disregarding the dominance relationship or constraints, to help cross infeasible region and approach CPF rapidly. For those individuals in state B, dominance relationship is taken into consideration, further bringing auxiliary task closer to the main task. The superiority of the proposed algorithm has been comprehensively demonstrated by experimental comparison results on a variety of benchmark test problems and a real-world problem with raw ore allocation in mineral processing.

Unbalanced classification problems have attracted significant academic attention due to their widespread existence in the real world. The lack of recognition accuracy of minority class samples and the “curse of dimensionality” are two major difficulties in unbalanced classification problems. Existing unbalanced classification methods run the risk of losing the original feature information and are prone to bias toward the majority class. Multiform optimization is famous for capturing useful knowledge from alternative forms to help solve the original task. Motivated by this, this article introduces a multiform evolutionary framework that addresses the issue of multiobjective feature selection in unbalanced classification scenarios. It aims to utilize the advanced experience of selecting features on balanced datasets to assist in the search for feature subsets that can more accurately identify minority classes on the original dataset. Specifically, a knowledge transfer strategy is proposed to draw on the search experience of the auxiliary task from the oversampled dataset to help the cost-sensitive learning task based on the original dataset jump out of the local optimum. In addition, an offspring repairing mechanism is proposed to filter redundant features by considering the frequency of selected features. Experimental results on 23 real-world benchmark datasets demonstrate that the proposed method can select fewer features and achieve better classification results compared to six state-of-the-art multiobjective feature selection algorithms and three classical oversampling algorithms. Furthermore, the difference in performance of four base classifiers is investigated through a series of comparative experiments.

For expensive constrained optimization problems (ECOPs), the evaluation of the objective and constraints are both expensive. Due to the low computational cost of the surrogate model and the excellent search capabilities of evolutionary algorithms, surrogate-assisted evolutionary algorithms (SAEAs) have become a popular approach for solving ECOPs. When solving ECOPs, the errors in the objective and constraint surrogates will inevitably mislead the direction of evolution, making it difficult to find feasible solutions and avoid local optima. To defeat this issue, we propose an SAEA capable of adjusting the evolutionary direction to search in the correct direction as much as possible. Specifically, the correlation between objective and constraint is first analyzed, and then adaptive adjustments are made based on this correlation to revise the evolutionary direction throughout the three stages of the evolutionary process. For reproduction, an offspring enhanced generation strategy is proposed to generate promising and diverse offspring. For sampling, a dynamic infill sampling criterion is designed to select the most suitable solutions for expensive evaluations, thereby accelerating convergence. Finally, an adaptive environment selection strategy is designed to choose parents with more potential for improvement. The proposed method is evaluated on commonly used benchmark test functions and four engineering examples, with experimental results indicating its superior performance compared to other advanced methods.

Dynamic flexible job shop scheduling (DFJSS) is an important combinatorial optimisation problem that aims to optimise machine resources to improve production efficiency. Multi-tree genetic programming (MTGP) has been widely used to learn the routing rule and the sequencing rule for DFJSS simultaneously. Unlike traditional genetic programming that only operates crossover on a single tree, MTGP has various cases to conduct crossover since a genetic programming individual consists of more than one tree. Different crossover operators may affect the performance of MTGP for DFJSS. However, the investigation into different crossover operators in MTGP for DFJSS is rare. Specifically, it is not clear what influence will have on MTGP if involving both the routing rule and the sequencing rule for crossover. To this end, this paper provides a comprehensive investigation of four possible crossover cases between multiple scheduling heuristics with MTGP for DFJSS. The four operators are designed according to the number of trees/rules that crossover operator works on, and whether swapping full trees between parents. The results show that although the compared algorithms have comparable results in most scenarios, MTGP with both rules for crossover and the swapping strategy is ranked as the best one. Further analyses show that the sizes of learned rules are highly related to the crossover operators, and crossover involving more rules can increase the rule sizes, and vice versa. In addition, the population diversity and the number of unique features in the learned rules of MTGP with both rules for crossover are increased to learn effective rules.

Feature selection in machine learning as a key data processing technique has two conflicting goals: minimizing the classification error rate and minimizing the number of features selected. However, most of the existing multi-objective feature selection methods face the problems of easily falling into local optima and slow convergence by virtue of their problem characteristics, such as partially conflicting objectives and highly discontinuous Pareto fronts. To solve these problems, this article proposes a multiform optimization framework to solve a multi-objective feature selection task together with several auxiliary single-objective feature selection tasks in a multitask environment. The proposed framework uses the problem-solving experience of single-objective tasks to assist the multi-objective feature selection task in exploring more promising regions and accelerating the convergence speed. Specifically, a knowledge transfer strategy based on the search experience of different tasks is developed to accomplish multiform optimization. In addition, a diversity enhancement mechanism is presented to improve search ability in promising decision space areas by considering historical information about the population. In most cases, the experiment results on 27 datasets demonstrate that the proposed technique can uncover more diversified feature subsets on the Pareto front in less time than existing state-of-the-art methods.

Dynamic multiobjective optimization problems (DMOPs) are widely encountered in real-world applications and have received considerable attention in recent years. During the process of solving DMOPs, tracking the constantly changing Pareto optimal set (POS) quickly is the main task, and numerous methods have been proposed to achieve this task from different perspectives. However, how to improve the evolution of static optimizers via using historical information from past environments gets little attention. In fact, with adequate historical evolution information to learn, the trend of population evolution in the historical environments is promising to guide the future evolution and thus enhance the search ability. Therefore, in this paper, a historical evolution learning based framework is proposed to assist the static optimizers in fully using the historical evolution direction and POSs distribution. Specifically, two new models are designed with a purpose of generating offsprings and environmental selection, respectively. For enhancing the contribution of offsprings, a direction guidance model is developed to guide the offsprings according to the direction tendency of historical evolution. Furthermore, to improve the robustness of static optimizers and avoid the disturbance caused by misleading solutions, a manifold revise model is proposed to produce promising solutions via consulting historical POSs distribution. The proposed framework employs these two models collaboratively, and it is flexible and readily to be embedded into various dynamic response mechanisms and static optimizers in coping with DMOPs. The superiority of the framework has been comprehensively demonstrated by experimental comparison results on a variety of benchmark test problems.

Improving classification performance and reducing the number of selected features are two conflicting objectives of feature selection, which can be well solved by multiobjective algorithms. However, as the dimensionality of the data increases, the search space for feature selection will grow exponentially, which leads to high-computational costs. Additionally, the complex interaction among features makes the population prone to falling into local optimal. To address these issues, feature grouping can treat one dimension as a group of features instead of one feature, effectively transforming the high-dimensional search space into a lower-dimensional one. Since different grouping forms can be converted into different feature combination spaces, the search directions of the population also vary. Inspired by this, a multiform optimization approach based on space transformation (MOFS-MST) is proposed in this article. Specifically, two different grouping forms are set based on the ranking of features in different evaluation criteria to construct a multiform framework, thereby increasing the diversity of the population. During the evolutionary process, a knowledge transfer strategy based on feature groups is executed between the two forms of grouping in order to help each other escape local optima. Moreover, it can dynamically adjust the state of feature grouping to enhance the potential for feature interaction. Experimental results demonstrate that this method outperforms six other state-of-the-art multiobjective high-dimensional feature selection methods on 12 high-dimensional datasets.

Genetic programming (GP) has been successfully introduced to learn scheduling heuristics for dynamic flexible job shop scheduling (DFJSS) automatically. However, the evaluations of GP individuals are normally time-consuming, especially with long DFJSS simulations. Taking k-nearest neighbour with phenotypic characterisations of GP individuals as a surrogate approach, has been successfully used to preselect GP offspring to the next generation for effectiveness improvement. However, this approach is not straightforward to improve the training efficiency, which is normally the primary goal of surrogate. In addition, there is no study on which GP individuals (samples) are good for building surrogate models. To this end, first, this paper proposes a surrogate-assisted GP algorithm to reduce the training time of learning scheduling heuristics for DFJSS. Second, this paper further proposes an effective sampling strategy for surrogate-assisted GP. The results show that our proposed algorithm can achieve comparable performance with only about a third of training time of traditional GP. With the same training time, the proposed algorithm can significantly improve the quality of learned scheduling heuristics in all examined scenarios. Furthermore, the evolved scheduling heuristics by the proposed sample-aware surrogate-assisted GP are more interpretable with smaller rule sizes than traditional GP.

As a key preprocessing step in classification, feature selection involves two conflicting objectives: maximizing the classification accuracy and minimizing the number of selected features. Therefore, multi-objective optimization is widely used in feature selection due to its excellent trade-off between the convergence of two objectives. However, most existing multi-objective feature selection methods still face the issues of the “curse of dimensionality” and high computational costs, especially when the search space is large. To solve the above issues, this paper proposes a bidirectional dynamic grouping multi-objective evolutionary approach for high-dimensional feature selection, referred to as BDGMOEA. This approach transforms a high-dimensional feature selection problem into a feature selection task with a smaller search space by the idea of feature grouping, in which one bit of an individual represents a group of features. Specifically, a grouping search strategy is developed to divide the features into different quadrants according to the importance of the features obtained by different evaluation techniques. Then, the features in each quadrant are grouped by sector. This strategy can effectively narrow the search space and quickly locate promising feature regions. In addition, a bidirectional dynamic adjustment mechanism is presented by considering the evolutionary state of the population, and it can be used to explore each feature in more detail and comprehensively to prevent good features from being ignored in unselected groups. The experimental results demonstrate that the proposed BDGMOEA method performs the best in most cases, indicating that BDGMOEA not only achieves better classification performance but also reduces the training time.

Handling constrained multiobjective optimization problems (CMOPs) is extremely challenging, since multiple conflicting objectives subject to various constraints require to be simultaneously optimized. To deal with CMOPs, numerous constrained multiobjective evolutionary algorithms (CMOEAs) have been proposed in recent years, and they have achieved promising performance. However, there has been few literature on the systematic review of the related studies currently. This article provides a comprehensive survey for evolutionary constrained multiobjective optimization. We first review a large number of CMOEAs through categorization and analyze their advantages and drawbacks in each category. Then, we summarize the benchmark test problems and investigate the performance of different constraint handling techniques (CHTs) and different algorithms, followed by some emerging and representative applications of CMOEAs. Finally, we discuss some new challenges and point out some directions of the future research in the field of evolutionary constrained multiobjective optimization.

When solving dynamic multiobjective optimization problems (DMOPs) by evolutionary algorithms, the historical moving directions of some special points along the Pareto front, such as the center and knee points, are widely employed to predict the Pareto-optimal solutions (POSs). However, special points may be impacted by certain individuals with a large direction deviation, and thus, mislead the tracking of dynamic POS. To solve this issue, a correlation-guided layered prediction approach for solving DMOPs is proposed in this article, where multiple prediction models are integrated by considering the correlation of individuals’ moving directions. To be specific, the population is clustered into three subpopulations (i.e., high, mid, and low correlation) by correlation analysis to perform different prediction behaviors. The high correlation subpopulation aims to predict the moving direction via a linear prediction model. The mid correlation subpopulation is devoted to predicting the manifold change of POS by self-adaptively using the direction and length correction models. The diversity preservation is considered by the low correlation subpopulation. While the three subpopulations focus on different optimization tasks, they also cooperate to track the dynamic POS. The comprehensive experimental results on a variety of benchmark test problems demonstrate the superiority of the proposed approach, as compared with some state-of-the-art prediction-based dynamic multiobjective algorithms.

Feature selection (FS) is an important data preprocessing technique in data mining and machine learning, which aims to select a small subset of information features to increase the performance and reduce the dimensionality. Particle swarm optimization (PSO) has been successfully applied to FS due to being efficient and easy to implement. However, most of the existing PSO-based FS methods face the problems of trapping into local optima and computationally expensive high-dimensional data. Multifactorial optimization (MFO), as an effective evolutionary multitasking paradigm, has been widely used for solving complex problems through implicit knowledge transfer between related tasks. Inspired by MFO, this study proposes a novel PSO-based FS method to solve high-dimensional classification via information sharing between two related tasks generated from a dataset. To be specific, two related tasks about the target concept are established by evaluating the importance of features. A new crossover operator, called assortative mating, is applied to share information between these two related tasks. In addition, two mechanisms, which are variable-range strategy and subset updating mechanism, are also developed to reduce the search space and maintain the diversity of the population, respectively. The results show that the proposed FS method can achieve higher classification accuracy with a smaller feature subset in a reasonable time than the state-of-the-art FS methods on the examined high-dimensional classification problems.

Feature selection (FS) is an important preprocessing technique for improving the quality of feature sets in many practical applications. Particle swarm optimization (PSO) has been widely used for FS due to being efficient and easy to implement. However, when dealing with high-dimensional data, most of the existing PSO-based FS approaches face the problems of falling into local optima and high-computational cost. Evolutionary multitasking is an effective paradigm to enhance global search capability and accelerate convergence by knowledge transfer among related tasks. Inspired by evolutionary multitasking, this article proposes a multitasking PSO approach for high-dimensional FS. The approach converts a high-dimensional FS task into several related low-dimensional FS tasks, then finds an optimal feature subset by knowledge transfer between these low-dimensional FS tasks. Specifically, a novel task generation strategy based on the importance of features is developed, which can generate highly related tasks from a dataset adaptively. In addition, a new knowledge transfer mechanism is presented, which can effectively implement positive knowledge transfer among related tasks. The results demonstrate that the proposed method can evolve a feature subset with higher classification accuracy in a shorter time than other state-of-the-art FS methods on high-dimensional classification.

Classification data are usually represented by many features, but not all of them are useful. Without domain knowledge, it is challenging to determine which features are useful. Feature selection is an effective preprocessing technique for enhancing the discriminating ability of data, but it is a difficult combinatorial optimization problem because of the challenges of the huge search space and complex interactions between features. Particle swarm optimization (PSO) has been successfully applied to feature selection due to its efficiency and easy implementation. However, most existing PSO-based feature selection methods still face the problem of falling into local optima. To solve this problem, this article proposes a novel PSO-based feature selection approach, which can continuously improve the quality of the population at each iteration. Specifically, a correlation-guided updating strategy based on the characteristic of data is developed, which can effectively use the information of the current population to generate more promising solutions. In addition, a particle selection strategy based on a surrogate technique is presented, which can efficiently select particles with better performance in both convergence and diversity to form a new population. Experimental comparing the proposed approach with a few state-of-the-art feature selection methods on 25 classification problems demonstrate that the proposed approach is able to select a smaller feature subset with higher classification accuracy in most cases.

Classification has been widely studied due to its practical applications. Feature selection aims to improve the classification accuracy by selecting a small feature subset from the original full feature set. However, identification of relevant features is not trivial due to the large search space. Particle swarm optimisation (PSO) is an efficient meta-heuristic algorithm which has shown to be promising in feature selection. However, traditional PSO uses its personal best experience and its historical best experience to determine its search direction, but this learning strategy may limit its performance for feature selection due to the premature convergence. Therefore, the potential of PSO needs to be further explored. In this paper, a new evolutionary learning algorithm termed hybridising PSO with differential evolution (HPSO-DE) is proposed to develop new feature selection methods. In HPSO-DE, differential evolution is applied to breed promising and efficient exemplars for PSO to guide its search, which is expected to not only preserve the diversity of the population but also guide particles to fly to promising areas. HPSO-DE is compared with three classic PSO variants and five traditional feature selection methods on 15 classification problems. The results show that the proposed algorithm can effectively achieve a higher classification accuracy with a smaller feature subset than the compared methods.