Surrogate-assisted evolutionary algorithms (SAEAs) have been widely developed to solve complex and computationally expensive multiobjective optimization problems (EMOPs) in recent years. However, when dealing with high-dimensional optimization problems in decision space, the performance of these surrogate-assisted multiobjective evolutionary algorithms (MOEAs) deteriorates drastically. In this work, a novel classifier-assisted rank-based learning and local model-based multiobjective evolutionary algorithm (CLMEA) is proposed for high-dimensional EMOPs. CLMEA makes full use of the uncertainty of solutions in the decision space and objective space to explore the uncertain but informative space toward high-dimensional problems. Specifically, the offspring in different ranks uses rank-based learning strategy to generate more promising and informative candidates for function evaluations (FEs). To reduce the search region of high-dimensional problems and maintain the diversity of solutions, the most uncertain sample point from the nondominated solutions measured by the crowding distance is selected as the center to conduct local search. The experimental results of benchmark problems and a real-world application on geothermal reservoir heat extraction optimization demonstrate superior performance of CLMEA compared with the state-of-the-art surrogate-assisted MOEAs. The source code for this work is available athttps://github.com/JellyChen7/CLMEA

Evolutionary transfer optimization (ETO) has been gaining popularity in research over the years due to its outstanding knowledge transfer ability to address various challenges in optimization. However, a pressing issue in this field is that the invention of new ETO algorithms has far outpaced the development of fundamental theories needed to clearly understand the key factors contributing to the success of these algorithms for effective generalization. In response to this challenge, this study aims to establish theoretical foundations for analogy-based ETO, specifically to support various algorithms that frequently reference a key concept known as similarity. First, we introduce analogical reasoning and link its subprocesses to three key issues in ETO. Then, we develop theories for analogy-based knowledge transfer, rooted in the principles that underlie the subprocesses. Afterwards, we present two theorems related to the performance gain of analogy-based knowledge transfer, namely unconditionally nonnegative performance gain and conditionally positive performance gain, to theoretically demonstrate the effectiveness of various analogy-based ETO methods. Last but not least, we offer a novel insight into analogy-based ETO that interprets its conditional superiority over traditional evolutionary optimization through the lens of the no free lunch theorem for optimization.

Despite the increasing interest in sequential transfer optimization (STO), a comprehensive benchmark suite for systematically comparing various STO algorithms remains underexplored. Existing test problems, which are often manually configured and lack scalability, can result in biased and nongeneralizable algorithm performance. In light of the above, we first introduce four concepts for characterizing STO problems (STOPs) in this study and present an important feature, namely similarity distribution, to quantitatively delineate the relationship between the optimal solutions of source and target tasks. Subsequently, we present general design guidelines for STOPs and introduce a problem generator that demonstrates strong scalability. Specifically, the similarity distribution of a problem can be easily customized through a novel inverse generation strategy, allowing for a continuous spectrum that captures the diverse similarity relationships present in real-world scenarios. Lastly, a benchmark suite comprising 12 STOPs, characterized by a range of customized similarity relationships, has been developed using the proposed generator and will serve as a platform for examining various STO algorithms. For instance, biased transferability representation, irregular mapping learning behaviors, and performance improvements unrelated to search experience are significant empirical findings that previous benchmarks failed to reveal, yet can be effectively identified through our test problems. The source code of the proposed problem generator is available at https://github.com/XmingHsueh/STOP-G.

Evolutionary sequential transfer optimization (ESTO), which aims to better optimize a target task using the knowledge extracted from a number of previously solved source tasks, has been gaining continually increasing research attention over the years. Particularly, solution-based ESTO (S-ESTO) that transfers task solutions has been receiving much popularity due to its ease of implementation and optimizer independency. However, the existing S-ESTO algorithms put much emphasis on utilizing source optimized solutions standing for global search experience without being aware of the potential of intermediate solutions that represent local optimization experience. Besides, most of them cannot take full advantage of the solution data from evolutionary search. In the light of the above, this study aims to develop a global and local search experience-based solution transfer technique to maximally release the potential of optimization experience hidden in the source tasks. First, a novel transferability metric named landscape encoding-based rank correlation (LERC) is developed. Then, we propose to divide the optimization experience into two classes: 1) global and 2) local search experience. Accordingly, by instantiating LERC into global and local versions, we develop two distinct transfer methods to exploit the global and local search experience, respectively. Finally, by combining the two transfer methods, we propose an S-ESTO algorithm that can transfer the global and local search experience simultaneously for maximum performance enhancement for the target task. Experiments conducted on a set of benchmark problems and a practical case study verify the efficacy of the proposed methods. The source code of our algorithm is available athttps://github.com/ccm831143/GL-LERC.

Expensive optimization problems (EOPs) have attracted increasing research attention over the decades due to their ubiquity in a variety of practical applications. Despite many sophisticated surrogate-assisted evolutionary algorithms (SAEAs) that have been developed for solving such problems, most of them lack the ability to transfer knowledge from previously-solved tasks and always start their search from scratch, making them troubled by the notorious cold-start issue. A few preliminary studies that integrate transfer learning into SAEAs still face some issues, such as defective similarity quantification that is prone to underestimate promising knowledge, surrogate-dependency that makes the transfer methods not coherent with the state-of-the-art in SAEAs, etc. In light of the above, a plug and play competitive knowledge transfer (CKT) method is proposed to boost various SAEAs in this article. Specifically, both the optimized solutions from the source tasks and the promising solutions acquired by the target surrogate are treated as task-solving knowledge, enabling them to compete with each other to elect the winner for expensive evaluation, thus boosting the search speed on the target task. Moreover, the lower bound of the convergence gain brought by the knowledge competition is mathematically analyzed, which is expected to strengthen the theoretical foundation of sequential transfer optimization. Experimental studies conducted on a series of benchmark problems and a practical application from the petroleum industry verify the efficacy of the proposed method. The source code of the CKT is available athttps://github.com/XmingHsueh/SAS-CKT.

Evolutionary transfer optimization (ETO), which combines evolutionary algorithms with knowledge transfer across related tasks to enhance search performance, has gained widespread attention from researchers in recent years. Multiform transfer optimization (MFTO) stands out as a representative transfer paradigm of ETO, aiming to exploit alternative formulations of the target task of interest. By leveraging useful knowledge acquired from alternative formulations to assist in solving the target task, MFTO has proven effective in tackling complex optimization problems, contributing to the growth of MFTO research. This paper provides a review of existing research progress in MFTO. Firstly, we introduce the fundamental aspects of MFTO, including the general framework and core components. Subsequently, we summarize the advances in MFTO from the perspectives of problems to be solved and the way of constructing alternative formulations. Lastly, we discuss promising future research directions. It is hoped that this survey can provide a thorough understanding of the MFTO framework and facilitate the development of more advanced MFTO algorithms and applications.

In cases of frequent problem-solving of multiobjective optimization tasks from a domain due to changing conditions or problem features, a growing number of individual tasks will be solved and stored in a database, providing an opportunity for a target task at hand to achieve better optimization performance through knowledge transfer from the previously-solved tasks, which is also known as sequential transfer optimization. Despite a variety of transfer algorithms that have been developed over the years, the research on the design of benchmark problems for evaluating such algorithms received far less attention. Oftentimes, the source and target tasks in existing test problems are manually assembled or extended from specific practical problems, limiting their ability to represent the diverse yet complex source-target similarity relationships in real-world problems. In light of this, we propose design methods to generate multiobjective sequential transfer optimization problems (MSTOPs) systematically in this work, wherein the Pareto manifolds of individual tasks and the manifold-based similarity between the tasks can be customized with ease, enabling a broad spectrum of representation of the diverse similarity relationships between the source-target Pareto manifolds of MSTOPs. Lastly, a benchmark suite with 12 test problems is developed using the proposed methods, which would serve as an arena for electing superior multiobjective sequential transfer optimization algorithms. The source code is available at https://github.com/XmingHsueh/MSTOP.

Knowledge transfer from optimized problems has emerged as a promising technique for enhancing evolutionary search. However, most studies in this domain primarily concentrate on devising knowledge transfer mechanisms for specific problem domains, often lacking the examination of the fundamental aspects of knowledge transfer, i.e., what, when and how to transfer across diverse scenarios. This not only restricts the generality of these algorithms but also hinders their practical applicability. In light of this, this paper: 1) reviews a vast array of techniques associated with the crucial aspects of solution transfer and 2) conducts a series of experiments to explore the underlying transfer mechanisms that enhance the evolutionary search. In particular, we first define solution transferability in the context of evolutionary search, which provides a new perspective in understanding what, when and how to transfer in enhancing evolutionary search. Next, through comprehensive experiments, we find that the approximation and evaluation of solution transferability is of great importance in designing what, when and how to transfer towards enhanced evolutionary search. Furthermore, our empirical study also discusses the counterintuitive performance improvements unrelated to the search experience of source tasks. The source code for reproducing our experiments is available at https://github.com/XmingHsueh/STO-EC.

The high prevalence of mental disorders gradually poses a huge pressure on the public healthcare services. Deep learning-based computer-aided diagnosis (CAD) has emerged to relieve the tension in healthcare institutions by detecting abnormal neuroimaging-derived phenotypes. However, training deep learning models relies on sufficient annotated datasets, which can be costly and laborious. Semi-supervised learning (SSL) and transfer learning (TL) can mitigate this challenge by leveraging unlabeled data within the same institution and advantageous information from source domain, respectively. This work is the first attempt to propose an effective semi-supervised transfer learning (SSTL) framework dubbed S3TL for CAD of mental disorders on fMRI data. Within S3TL, a secure cross-domain feature alignment method is developed to generate target-related source model in SSL. Subsequently, we propose an enhanced dual-stage pseudo-labeling approach to assign pseudo-labels for unlabeled samples in target domain. Finally, an advantageous knowledge transfer method is conducted to improve the generalization capability of the target model. Comprehensive experimental results demonstrate that S3TL achieves competitive accuracies of 69.14%, 69.65%, and 72.62% on ABIDE-I, ABIDE-II, and ADHD-200 datasets, respectively. Furthermore, the simulation experiments also demonstrate the application potential of S3TL through model interpretation analysis and federated learning extension.

The high prevalence of mental disorders gradually poses a huge pressure on the public healthcare services. Recently, deep learning-based computer-aided diagnosis has been introduced to relieve the tension in healthcare institutions by automatically detecting abnormal neuroimaging-derived pheno-types in patients. However, the training of deep learning models relies on sufficiently large annotated datasets, which can be costly, time-consuming, and laborious. Semi-supervised learning (SSL) can mitigate this challenge by leveraging both labeled and unlabeled samples. In this work, an effective dual-stage pseudo-labeling based classification framework dubbed DSPL is proposed to diagnose mental disorders on functional magnetic resonance imaging data. A bicriteria-based pseudo-labels selection method is developed to filter out inferior pseudo-labeled samples. Subsequently, we further propose a self-mutual learning enhanced pseudo-labeling generation approach to mitigate the adverse effects bought by the noisy pseudo-labeled samples. On real-world datasets, the proposed method achieves diagnosis accuracies of 68.09%, 67.94%, and 68.13% on ABIDE-I, ABIDE-II, and ADHD-200, respectively. Ablation study suggests that each component in DSPL makes a great contribution to performance improvement.

Evolutionary multitasking (EMT) is a newly emerging research topic in the community of evolutionary computation, which aims to improve the convergence characteristic across multiple distinct optimization tasks simultaneously by triggering knowledge transfer among them. Unfortunately, most of the existing EMT algorithms are only capable of boosting the optimization performance for homogeneous problems which explicitly share the same (or similar) fitness landscapes. Seldom efforts have been devoted to generalize the EMT for solving heterogeneous problems. A few preliminary studies employ domain adaptation techniques to enhance the transferability between two distinct tasks. However, almost all of these methods encounter a severe issue which is the so-called degradation of intertask mapping. Keeping this in mind, a novel rank loss function for acquiring a superior intertask mapping is proposed in this article. In particular, with an evolutionary-path-based representation model for optimization instance, an analytical solution of affine transformation for bridging the gap between two distinct problems is mathematically derived from the proposed rank loss function. It is worth mentioning that the proposed mapping-based transferability enhancement technique can be seamlessly embedded into an EMT paradigm. Finally, the efficacy of our proposed method against several state-of-the-art EMTs is verified experimentally on a number of synthetic multitasking and many-tasking benchmark problems, as well as a practical case study.

Evolutionary sequential transfer optimization is a paradigm that leverages search experience from solved source optimization tasks to accelerate the evolutionary search of a target task. Even though many algorithms have been developed, they mainly focus on objective-homogeneous problems, where the source and target tasks possess a similar number of objectives. In this work, we explore objective-heterogeneous problems, in which knowledge transfers across single-objective optimization problems (SOPs), multiobjective optimization problems (MOPs), and many-objective optimization problems (MaOPs). Objective-heterogeneous problems challenge the existing methods due to the diverse search and objective spaces between the source and the target task. To address this issue, we present a decision variable analysis-based transfer method that can conduct knowledge transfer across problems with the different numbers of objectives. We first separate decision variables of MOPs and MaOPs into convergence-related variables (CVs) and diversity-related variables (DVs), according to their roles while treating variables of SOPs as CVs. Then, we propose a convergence transfer module to transfer knowledge of CVs to speed up the convergence. It aligns both solutions and fitness ranks for preserving fitness rank consistency between the source and target tasks, whereby accelerating search speed. Besides, a diversity transfer module is presented to refine the distribution of DVs to maintain the population diversity. The experimental results on objective-heterogeneous test problems and a real-world case study have demonstrated the effectiveness of the proposed algorithm.