Many real-world problems involve optimizing numerous decision variables and are expensive to evaluate, known as large-scale expensive optimization problems (LSEOPs). While surrogate-assisted evolutionary algorithms have proven effective for expensive problems, training proper models for LSEOPs remains challenging due to insufficient training data. In this paper, we adopt the divide-and-conquer approach, decomposing LSEOPs into lower-dimensional sub-problems and constructing models for sub-problems, and introduce a multi-view synthetic sampling technique for new sample selection. Specifically, we propose sorting all evaluated solutions in an ascending order and dividing them into intervals, from which data are sampled to obtain informative training data for models. The population for the LSEOP is updated by employing cooperative environmental selections on the population, formed by recombining all renewed populations for sub-problems to balance exploration and exploitation. Finally, a solution is selected among the current population for the true evaluation based on its multi-view performance predicted across all sub-problems. Results on CEC’2013 benchmark problems show the effectiveness and efficiency of our proposed method compared to three prevalent large-scale expensive optimization algorithms. Additionally, results on 2000-dimensional CEC’2010 benchmark problems and a 1200-dimensional real-world problem demonstrate encouraging scalability and robustness of the proposed method for addressing higher-dimensional problems.

Surrogate-assisted evolutionary algorithms are frequently applied to solve time-consuming, resource-intensive, and black-box multiobjective optimization problems. Multiple approximation effectively identifies an approximate optimal solution set within finite exact function evaluations, which may need significant training time for the surrogate models. This article offers a model training method guided by estimation errors to assist the evolutionary algorithm in the search for optimal solutions. In model training, we dynamically use the Gaussian process (GP) and radial basis function (RBF) models following the adjacent generation discrepancy of estimation errors to reduce computational time. They are updated if only the current estimation error exceeds the previous, where the estimation error combines the minimum distance in the decision space and the prediction error of all test samples. In the model-assisted search, an autonomous function estimation method is proposed based on the preference for approximate model errors. The selection of the updated GP or RBF approximation is via a lower model estimation error; in contrast, the average is considered the function value of an individual. In infill sampling, the solution is selected based on the nondominated sorting of function estimation with the maximum angle. The uncertainty-based sampling method is to replenish when these models are not updated. The experiment investigates the effectiveness of the error preference-guided approximation method. The results of two classic benchmark problems and one practice problem show the superiority of the proposed algorithm compared to even well-performed optimization algorithms.

Traditional recommendation algorithms primarily focus on accuracy, often overlooking diversity and long-tail item recommendations, which are inherently conflicting objectives. Balancing these objectives remains a significant challenge. To address this issue, we propose refined objective functions for diversity and long-tail item recommendation, aiming to more accurately evaluate the diversity of recommendation lists and improve the exposure of long-tail items. To solve the resulting multi-objective optimization problem, we introduce a preference-guided multi-objective recommendation algorithm incorporating an adaptive Tchebycheff environmental selection strategy. This algorithm combines a preference-guided local search to enhance solution quality with an adaptive Tchebycheff selection mechanism that dynamically adjusts objective weights to improve both convergence and stability. Experimental results demonstrate that the proposed algorithm effectively balances accuracy, diversity, and long-tail item recommendation.

Neural Architecture Search (NAS) has emerged as a promising approach to automating the discovery of optimal neural network architectures. However, the computational cost of evaluating candidate architectures through full training presents a significant barrier to efficient search. While surrogate models can accelerate this process by predicting network performance, they struggle to accurately model the vast architecture search space when working with limited training data. To mitigate this challenge, we propose a Multi-Task, Multi-Surrogate Assisted Evolutionary NAS framework (MT-MSAENAS) that combines multiple surrogate models to enhance search efficiency. To fully exploit the limited training data, MT-MSAENAS constructs both strong and weak surrogate predictors, a global model (strong) that captures overall search space patterns and a local model (weak) that specializes in promising regions. Based on the two surrogate model, MT-MSAENAS employs evolutionary multi-tasking optimization by treating the strong and weak model-assisted search as two related optimization tasks to facilitate knowledge transfer between these models and improve the search efficiency. Experiments on the NAS-Bench101 and NAS-Bench201 search spaces show that the proposed algorithm outperforms state-of-the-art methods in architecture search.

The initial recommendation list in the multi-objective recommendation algorithm is generated by a specific algorithm, meaning the accuracy of the multi-objective algorithm is constrained by the performance of the algorithm used to generate the initial recommendations. To achieve better accuracy, the many-objective hybrid recommendation algorithm based on user grouping algorithm is proposed. The algorithm constructs a dual preference model, and employs a many-objective optimization approach to balance conflicts among these objectives. However, as the number of users increases, the population dimension of the many-objective hybrid recommendation algorithm also grows, resulting in challenges related to convergence. To mitigate this issue, users are grouped to reduce the population dimension. Frequency-based genes are derived from user similarity for crossover operations, generating offspring more efficiently. Experimental results demonstrate that user grouping significantly reduces computational cost and enhances the algorithm’s performance.

Over the past few years, a variety of surrogate-assisted evolutionary algorithms have emerged, aiming to tackle expensive multi-objective optimization problems. On the other hand, as the problem's dimensionality rises, the sample size needed for training surrogate models also increases significantly, making this approach impractical for large-scale scenarios. To overcome this challenge, we propose a large-scale expensive multi-objective optimization algorithm leverages collaboration among multiple subproblems. Specifically, we employ random feature selection with multiple times to partition the original decision space into multiple subproblems. For each objective, a surrogate model is constructed based on the historical data of each subproblem. After searching for the surrogate models of each subproblem, the population of original problem is updated. Additionally, to accelerate optimization, we select m individuals with the best performance for each objective, as well as the one nearest to the ideal point, for true evaluations. To illustrate the effectiveness of our approach, comparative experiments have been carried out using the DTLZ and ZDT benchmark problems, which involve up to 200 decision variables. The experimental outcomes indicate that the proposed technique delivers strong performance in addressing large-scale, expensive multi-objective optimization tasks.

Traditional large-scale evolutionary algorithms are limited in their ability to solve certain real-world applications with high-dimensional, closed-box, and computationally expensive objectives due to their need for numerous objective evaluations. Surrogate-assisted evolutionary algorithms (SAEAs) have shown effective for expensive closed-box optimization by relying on inexpensive surrogate models. However, large-scale optimization remains challenging for SAEAs due to the exponentially growing search space and the presence of multiple local optima, resulting in difficulty in training a proper model due to the lack of samples. To address these challenges, we propose constructing an initial surrogate model on randomly selected dimensions and calculating a Gaussian distribution for each sampled dimension. The surrogate then provides predictions when perturbing each sampled dimension by sampling from the distribution, enabling the identification of the most important variables for constructing an active subproblem to reduce the search space. A secondary surrogate model, built for the active subproblem, guides the offspring generation and environmental selection for a modified particle swarm optimization algorithm to effectively explores the subspace while escaping local optima in large-scale problems. Experimental results on CEC’2013 and CEC’2010 benchmark problems show that the proposed method outperforms state-of-the-art algorithms in addressing large-scale expensive optimization problems. The efficiency of the proposed method is further verified on CEC’2010 benchmark problems extended to 2000 dimensions.

In surrogate-assisted expensive multiobjective evolutionary optimization, selecting individuals for expensive function evaluations has received widespread attention to improve the model. However, most algorithms search for optimal solutions based on estimated function values obtained by the regression model, followed by choosing the infill individual to update the model following the estimated population distribution, which may mislead the location of optimal solutions. Therefore, this article proposes an extended estimation and adaptive contribution sampling approach to find optimal solutions to expensive optimization problems. In evolution, the estimation value of each individual is composed of the prediction of each objective, together with the estimated uncertainty obtained by the GP model to enhance the exploration of the optimum. Subsequently, the adaptive contribution-based sampling strategy is applied to select the promising individual for expensive function evaluations to guide the approximate model search. In this contribution mechanism, the sparseness change of each sample is used to guide the selection of the infill solution with a performance indicator or the estimated confidence level. Experiments on two multiobjective test suites and the practical problem demonstrate the effectiveness of the extended estimation and adaptive contribution sampling approach compared to seven recently prevailing algorithms.

Surrogate model-assisted multi-objective optimization has increasingly prevailed in solving expensive multi-/many-objective optimization problems. However, most research has advocated updating each model on each echo by the designed infill sampling approach. After the model-based search for the optimum, the approximated model of each objective may not need to be updated. Therefore, this paper proposes a GP model-assisted multi-objective optimization method with an adaptive modeling approach, with an angle and uncertainty-based infill sampling strategy assisting with updating. The estimated error change of all test samples is utilized in this adaptive modeling approach to assess if the surrogate model of each objective needs to be changed. The model, meanwhile, is improved by two infill individuals based on non-dominated sorting of estimations and uncertainties, respectively. One of the sampling criteria is to choose the potential individual for boosting the exploitation of the optimal solutions using estimation-based sorting and angles with samples. Following the uncertainty-based non-dominated sorting, the other aims to choose the infill individual for exploring the potential optimum position. Experiments on two multi-objective test suites and one real-world application indicate that the proposed method outperforms five contemporary classic optimization algorithms.

Large-scale expensive optimization problems (LSEOPs) are characterized by a large number of dimensions in the decision space and expensive objective functions. The curse of dimensionality poses challenges to achieving the optimal solution, especially when limited objective evaluations are allowed. In this paper, the cooperative coevolution framework is used. A PSO variant is proposed to update solutions for each sub-problem. Then, a difference-based infill criterion is proposed to select a solution for real expensive objective evaluation. A number of experiments are conducted on CEC'2013 test problems to verify the performance of the proposed method. The experimental results show that our proposed method can obtain better performance than the four representative methods presented for solving LSEOPs.

Several meta-heuristic algorithms have gained attention for addressing large-scale optimization problems (LSOPs). However, their effectiveness is limited when dealing with LSOPs that involve expensive objective functions because the significant number of objective evaluations required before identifying the optimal solution. In this paper, two sets of subproblems are op-timized, assisted by surrogate models, to collaboratively explore the optimal solution space for large-scale expensive optimization problems (LSEOPs). A variant of particle swarm optimization (PSO), engineered to accelerate convergence, is proposed to search for the optimal solution for each subproblem. Moreover, a novel infill criterion is introduced, where the solution with the maximum expected improvement in the first set of subproblems and the solution with the minimum objective value discovered so far in the second set of subproblems are selected. Then, their mean position will be evaluated using the real objective function. Experimental findings on 15 CEC2013 benchmark problems show that our proposed approach outperforms four state-of-the-art algorithms in solving expensive large-scale optimization problems with up to 1000 decision variables.

Multi-modal multi-objective optimization problems (MMOPs) are ubiquitous in real-world applications, in which multiple objective functions are conflicting and need to be optimized simultaneously. Furthermore, multiple Pareto optimal solutions of MMOPs are mapped to the same point on the Pareto front. Canonical multi-objective optimization algorithms show poor performance for MMOPs due to the lack of diversity maintenance in the decision space. In this paper, a particle swarm optimization with ring topology, denoted as PSO-RT, is proposed for solving MMOPs. The population is firstly divided into several sub-populations based on a dynamic radius to form a ring topology. Then, each solution will update its position by learning from one of its personal best positions and the best position of its neighbors. The personal best position to be learned is selected based on the special crowding distance to improve the exploration capability, and the best position of its neighbors is selected according to the weighted indicator to speed up convergence. The performance of PSO-RT is evaluated on 22 test problems. Experimental results show that our proposed PSO-RT can obtain competitive or better results than some state-of-the-art algorithms proposed for MMOPs in terms of Pareto sets proximity (PSP) and inverted generational distance (IGDX) matrics.

In surrogate-assisted multi-/many-objective evolutionary optimization, each solution normally has an approximated value on each objective, resulting in increased difficulties in selecting solutions for expensive objective evaluations due to complicated tradeoff between different objectives and accumulated uncertainty in the approximation of the objective functions. Thus, it is highly challenging to design an efficient model management strategy for surrogate-assisted expensive multi-/many-objective optimization. In this article, a surrogate model is built for each objective function, based on which a set of promising candidate solutions are found. Additionally, a Gaussian process model is constructed to approximate a newly designed performance indicator measuring both convergence and diversity properties of individual solutions. Finally, the solution of the found candidate solutions having the maximum expected improvement in terms of the performance indicator is selected for evaluation using the expensive objective functions. Comparative experiments are conducted on 3-, 5-, and 10-objective DTLZ, WFG, and MaF test functions, as well as two real-world applications. The experimental results show that the proposed method is competitive compared to five state-of-the-art surrogate-assisted evolutionary algorithms for expensive multi-/many-objective optimization.

It is particularly challenging for evolutionary algorithms to quickly converge to the Pareto front in large-scale multiobjective optimization. To tackle this problem, this article proposes a large-scale multiobjective evolutionary algorithm assisted by some selected individuals generated by directed sampling (DS). At each generation, a set of individuals closer to the ideal point is chosen for performing a DS in the decision space, and those nondominated ones of the sampled solutions are used to assist the reproduction to improve the convergence in evolutionary large-scale multiobjective optimization. In addition, elitist nondominated sorting is adopted complementarily for environmental selection with a reference vector-based method in order to maintain diversity of the population. Our experimental results show that the proposed algorithm is highly competitive on large-scale multiobjective optimization test problems with up to 5000 decision variables compared to five state-of-the-art multiobjective evolutionary algorithms.

It is challenging for optimization algorithms to obtain well converged and diverse optimal solutions in solving many-objective optimization problems, especially when the Pareto front is complex. In this paper, a new selection strategy is proposed for decomposition based evolutionary algorithms for solving many-objective optimization problems. In the proposed method, each individual in a population is assigned to a reference vector at first according to the angle between the objective vector of this individual and the reference vectors to divide the population into some subpopulations. Then for each subpopulation, an ideal point will be specified according to the minimum value of each objective among all solutions in the corresponding subpopulation. The individual in each subpopulation with the maximum ratio of the cosine of the angle between its objective vector and the corresponding reference vector to the distance from the individual to the corresponding ideal point will be selected to be passed to the next generation. The proposed algorithm is compared with five state-of-the-art algorithms on the DTLZ1-7 and WFG3-4 test problems with up to 15 objectives. The experimental results showed the competitiveness of the proposed method.

In evolutionary many-objective optimization, an effective environmental selection strategy is crucial to the performance of evolutionary algorithms, especially when the number of objectives is large. In this paper, we propose a new method for environmental selection, where three indicators are calculated and nondominated sorted for individual selection to be passed to the next generation. Of the three indicators, two aim to select individuals close to the center and edge part of the front in the objective space, and third one focuses on promoting the diversity of the population in the decision space to find promising solutions. The proposed algorithm is evaluated on the commonly used DTLZ test suite for many-objective optimization. Our comparative experimental results show that the proposed method is competitive compared to the state-of-the-art, especially on convergence performance in solving many-objective problems.

Model management plays an essential role in surrogate-assisted evolutionary optimization of expensive problems, since the strategy for selecting individuals for fitness evaluation using the real objective function has substantial influences on the final performance. Among many others, infill criterion driven Gaussian process (GP)-assisted evolutionary algorithms have been demonstrated competitive for optimization of problems with up to 50 decision variables. In this paper, a multiobjective infill criterion (MIC) that considers the approximated fitness and the approximation uncertainty as two objectives is proposed for a GP-assisted social learning particle swarm optimization algorithm. The MIC uses nondominated sorting for model management, thereby avoiding combining the approximated fitness and the approximation uncertainty into a scalar function, which is shown to be particularly important for high-dimensional problems, where the estimated uncertainty becomes less reliable. Empirical studies on 50-D and 100-D benchmark problems and a synthetic problem constructed from four real-world optimization problems demonstrate that the proposed MIC is more effective than existing scalar infill criteria for GP-assisted optimization given a limited computational budget.

In solving many real-world optimization problems, neither mathematical functions nor numerical simulations are available for evaluating the quality of candidate solutions. Instead, surrogate models must be built based on historical data to approximate the objective functions and no new data will be available during the optimization process. Such problems are known as offline data-driven optimization problems. Since the surrogate models solely depend on the given historical data, the optimization algorithm is able to search only in a very limited decision space during offline data-driven optimization. This paper proposes a new offline data-driven evolutionary algorithm to make the full use of the offline data to guide the search. To this end, a surrogate management strategy based on ensemble learning techniques developed in machine learning is adopted, which builds a large number of surrogate models before optimization and adaptively selects a small yet diverse subset of them during the optimization to achieve the best local approximation accuracy and reduce the computational complexity. Our experimental results on the benchmark problems and a transonic airfoil design example show that the proposed algorithm is able to handle offline data-driven optimization problems with up to 100 decision variables.

In many real-world optimization problems, it is very time-consuming to evaluate the performance of candidate solutions because the evaluations involve computationally intensive numerical simulations or costly physical experiments. Therefore, standard population based meta-heuristic search algorithms are not best suited for solving such expensive problems because they typically require a large number of performance evaluations. To address this issue, many surrogate-assisted meta-heuristic algorithms have been proposed and shown to be promising in achieving acceptable solutions with a small computation budget. While most research focuses on reducing the required number of expensive fitness evaluations, not much attention has been paid to take advantage of the large amount of unlabelled data, i.e., the solutions that have not been evaluated using the expensive fitness functions, generated during the optimization. This paper aims to make use of semi-supervised learning techniques that are able to enhance the training of surrogate models using the unlabelled data together with the labelled data in a surrogate-assisted particle swarm optimization algorithm. Empirical studies on five 30-dimensional benchmark problems show that the proposed algorithm is able to find high-quality solutions for computationally expensive problems on a limited computational budget.

When using a fixed number of neighbors for training a local surrogate model in surrogate assisted evolutionary optimization algorithms, it may suffer from the large uncertainty because the actual distribution of candidate's neighborhood may be neglected. In this paper, we propose to firstly analyze the distribution characteristics of candidate's neighborhood through a modified overlapping clustering method before training a local surrogate, and then use the clustering based evolution control strategy or model management strategy to facilitate the evolutionary algorithm to converge to the right optimum. Simulation results on four widely used benchmark functions demonstrate the efficacy of the proposed method.

Surrogate models have shown to be effective in assisting metaheuristic algorithms for solving computationally expensive complex optimization problems. The effectiveness of existing surrogate-assisted metaheuristic algorithms, however, has only been verified on low-dimensional optimization problems. In this paper, a surrogate-assisted cooperative swarm optimization algorithm is proposed, in which a surrogate-assisted particle swarm optimization (PSO) algorithm and a surrogate-assisted social learning-based PSO (SL-PSO) algorithm cooperatively search for the global optimum. The cooperation between the PSO and the SL-PSO consists of two aspects. First, they share promising solutions evaluated by the real fitness function. Second, the SL-PSO focuses on exploration while the PSO concentrates on local search. Empirical studies on six 50-D and six 100-D benchmark problems demonstrate that the proposed algorithm is able to find high-quality solutions for high-dimensional problems on a limited computational budget.

As a population-based meta-heuristic technique for global search, particle swarm optimization (PSO) performs quite well on a variety of problems. However, the requirement on a large number of fitness evaluations poses an obstacle for the PSO algorithm to be applied to solve complex optimization problems with computationally expensive objective functions. This paper extends a fitness estimation strategy for PSO (FESPSO) based on its search dynamics to reduce fitness evaluations using the real fitness function. In order to further save the fitness evaluations and improve the estimation accuracy, a similarity measure and a reliability measure are introduced into the FESPSO. The similarity measure is used to judge whether the fitness of a particle will be estimated or evaluated using the real fitness function, and the reliability measure is adopted to determine whether the approximated value will be trusted. Experimental results on six commonly used benchmark problems show the effectiveness and competitiveness of our proposed algorithm. Preliminary empirical analysis of the search behavior is also performed to illustrate the benefit of the proposed estimation mechanism.

Most evolutionary algorithms, including particle swarm optimization (PSO) algorithms, involve at least one population (swarm) to realize information exchange or information sharing among different individuals. To enhance the algorithms' global search ability, several multi-swarm PSO algorithms have been proposed. In this paper, a novel multi-swarm evolutionary framework based on a feedback mechanism is introduced. The framework consists of a search operator similar to those in PSO and a mutation strategy, on the top of the feedback mechanism. The framework is compared with a multi-swarm PSO and the canonical PSO on a few widely used benchmarks to demonstrate its performance.