In evolutionary multi-objective optimization (EMO), a set of well-distributed non-dominated solutions is typically obtained by an EMO algorithm. However, the choice of a final solution from the obtained solution set has not been discussed in many studies. Moreover, for standard EMO algorithms without preference information, it has rarely been discussed whether the obtained solution set includes a sufficient number of well-balanced trade-off solutions over all objectives for selection by a decision maker. Since the final goal of multi-objective optimization is to find the most preferred solution for the decision maker, it is necessary for EMO algorithms to find those promising solutions. From this viewpoint, one issue in many EMO algorithms is that the obtained solution sets usually include many solutions on the boundary of the Pareto front, especially in the case of many-objective optimization. Since boundary solutions are not balanced over all objectives, they are less likely to be selected as a final solution by the decision maker. To address this issue, we propose a modified MOEA/D algorithm with an interior weight vector generation strategy after discussing the negative effects of boundary weight vectors. Experimental results demonstrate the effectiveness of the proposed approach in finding balanced and promising solutions.

External archives have been demonstrated to be an effective technique for evolutionary multiobjective optimization (EMO) algorithms. Conducting subset selection from external archives offers advantages in finding solution sets that outperform the final population. While previous research has examined the impact of external archives on unconstrained multiobjective optimization problems, their effectiveness on constrained multi-objective optimization problems (CMOPs) remains unexplored. This paper investigates the effects of unbounded external archives (UEAs) and three different subset selection methods on the performance of eight constrained EMO algorithms through computational experiments on artificial and real-world CMOPs. Our experimental results demonstrate that UEAs are effective for constrained EMO algorithms in finding solution sets with improved convergence and diversity in comparison with the final population of each algorithm. It is also demonstrated that the effectiveness of UEAs depends on the constraint handling techniques (CHTs) in EMO algorithms and the choice of subset selection method.

As the initial phase of evolutionary multi-objective optimization (EMO), population initialization is always essential for subsequent evolutionary processes aimed at solving multi-objective optimization problems. While random initialization (i.e., random sampling) remains the most frequently used initialization method in EMO algorithms, many studies have indicated that the utilization of alternative initialization methods instead of random initialization can significantly improve the performance of the original EMO algorithms. Some studies have also investigated the effects of different initialization techniques or parameters (e.g., methods and population size). However, there is a scarcity of recent reviews focusing on population initialization for EMO algorithms. To bridge this research gap in the EMO research community, this paper provides a short review on this crucial topic. Specifically, the current choice of initialization methods is briefly summarized by using some representative EMO algorithms. The effects of population initialization and new initialization method design are then extensively reviewed. Insights into population initialization are also provided. This study aims to provide a comprehensive understanding of population initialization for newcomers to the EMO community.

Characteristics of an evolutionary multi-objective optimization (EMO) algorithm can be explained using its best solution set. For example, the best solution set for SMS-EMOA is the same as the optimal distribution of solutions for hypervolume maximization. For NSGA-III, if the Pareto front has intersection points with all reference lines, all of those intersection points are the best solution set. For MOEA/D, the best solution set is the set of the optimal solution of each sub-problem. Whereas these EMO algorithms can be analyzed in this manner, the best solution set for the most well-known and frequently-used EMO algorithm NSGA-II has not been discussed in the literature. This is because NSGA-II is not based on any clear criterion to be optimized (e.g., hypervolume maximization, distance minimization to the nearest reference line). As the first step toward the best solution set analysis for NSGA-II, we discuss the optimal distribution of solutions for the crowding distance under the simplest setting: the maximization of the minimum crowding distance on linear Pareto fronts of two-objective optimization problems. That is, we discuss the optimal distribution of solutions on a straight line. Our theoretical analysis shows that the uniformly distributed solutions are not the best solution set. However, it is also shown by computational experiments that the uniformly distributed solutions (except for the duplicated two extreme solutions at each edge of the Pareto front) are obtained from a modified NSGA-II with the (μ + 1) generation update scheme.

When we manually design an evolutionary optimization algorithm, we implicitly or explicitly assume a set of target optimization problems. In the case of automated algorithm design, target optimization problems are usually explicitly shown. Recently, the use of large language models (LLMs) for the design of evolutionary multi-objective optimization (EMO) algorithms have been examined in some studies. In those studies, target multi-objective problems are not always explicitly shown. It is well known in the EMO community that the performance evaluation results of EMO algorithms depend on not only test problems but also many other factors such as performance indicators, reference point, termination condition, and population size. Thus, it is likely that the designed EMO algorithms by LLMs depends on those factors. In this paper, we try to examine the implicit assumption (i.e., implicit knowledge) about the performance comparison of EMO algorithms in LLMs. For this purpose, we ask LLMs to design a benchmarking experiment of EMO algorithms. Our experiments show that LLMs often suggest classical benchmark settings: Performance examination of NSGA-II, MOEA/D and NSGA-III on ZDT, DTLZ and WFG by HV and IGD under the standard parameter specifications.

This study explores the characteristics of the extensive non-dominated archive generated by evolutionary multi-objective optimization (EMO) algorithms. Our findings reveal that such archives typically contain not only high-quality solutions positioned near the Pareto front, but also less desirable ones that deviate significantly from it. We further evaluate the effectiveness of various subset selection strategies in isolating the high-quality solutions within the archive. The insights gained from this work aim to assist EMO practitioners in gaining a deeper understanding of archive composition and in selecting appropriate subset selection methods to support decision-making.

In this paper, we discuss the optimal distributions of solutions for maximizing the minimum crowding distance on linear Pareto fronts of two- and three-objective problems. Our discussions are twofold. One is theoretical discussions to analytically derive the optimal distributions. The other is empirical discussions, which are based on a newly implemented indicator-based algorithm to maximize the minimum crowding distance. First, we show the upper bound on the minimum crowding distance, which is derived from the upper bound on the total crowding distance. Next, we theoretically derive the optimal distributions for maximizing the minimum crowding distance on two-objective linear Pareto fronts. Our analysis shows that two solutions always overlap in the optimal distributions when the population size is an even number. For an odd number population size, the optimal distribution cannot be uniquely specified except for some anchor solutions (i.e., many different distributions are optimal). The theoretically obtained optimal distributions are compared with experimental results by our indicator-based algorithm. Then, we show an example of optimal distributions for a three-objective linear Pareto front where the population size is a multiple of three. Our indicator-based algorithm is also compared with the standard NSGA-II algorithm and its steady-state variant.

In most multi-objective test problems, the dimensionality of the Pareto set is the same as the dimensionality of the Pareto front. That is, an m-objective test problem with n decision variables usually has an (m-1)-dimensional Pareto front and an (m-1)-dimensional Pareto set. Thanks to this property, the mapping between the Pareto front and the Pareto set is usually a one-to-one mapping. In this paper, we formulate two-objective distance minimization problems in an n-dimensional decision space using Manhattan distance. The two objectives are defined by two target points in the decision space. That is, each objective is to minimize the Manhattan distance from the solution to each target point. The main characteristic feature of our test problems is that the dimensionality of the Pareto set can be arbitrarily specified between 1 and n by the locations of the two target points. For example, the Pareto set dimensionality in a 20-dimensional decision space is an arbitrarily specified integer between 1 and 20. These test problems have multimodality since many points in the Pareto set are mapped to the same single point on the Pareto front. In this paper, we first explain the relation between the locations of the two target points and the Pareto set dimensionality. Then, through computational experiments, we show that our simple test problems pose difficult challenges for both multi-objective algorithms and multi-modal multi-objective algorithms. We also discuss the scalability of the proposed test problems with respect to the number of equivalent Pareto sets and the number of objectives.

Recently, performance comparison results of evolutionary multi-objective optimization (EMO) algorithms have been reported in many studies. However, EMO algorithms have not been compared with mathematical programming-based methods in those studies. To demonstrate the usefulness of EMO algorithms, it is needed to clearly show their advantages over mathematical programming-based methods in solving multi-objective optimization problems since those methods are usually highly efficient and effective. In this paper, a novel improved linear programming-based relaxation method, named ILP-R, is proposed for addressing multi-objective knapsack problems (MOKP), which are used as the test problems for performance comparison. Extensive experimental results show that ILP-R outperforms a basic linear programming-based relaxation method and EMO algorithms. Nevertheless, EMO algorithms exhibit the ability to further improve the solutions generated by the ILP-R method. Furthermore, a knowledge-based mutation method is explored to demonstrate its effectiveness in further improving the performance of EMO algorithms that use the heuristic ILP-R solutions as the initial population.

In the field of evolutionary multi-objective optimization (EMO), EMO algorithms are often categorized into three types: dominance-based, decomposition-based and indicator-based algorithms. In this categorization, NSGA-III is handled as a decomposition-based algorithm. This is because a single solution is assigned to each of uniformly generated reference vectors as in MOEA/D. However, in a recent survey paper, NSGA-III was categorized in the same group as NSGA-II based on their generation update mechanisms. Another recent study demonstrated that NSGA-III shows a similar search behavior to NSGA-II for combinatorial multi-objective problems. However, for DTLZ test problems, NSGA-III shows almost the same search behavior as MOEA/D. In this paper, we demonstrate that the shape of the Pareto front is the main factor about the search behavior of NSGA-III. If a test problem has a regular (i.e., triangular) Pareto front, NSGA-III shows the same search behavior as MOEA/D. However, if a test problem has an irregular Pareto front (e.g., inverted triangular), NSGA-III shows a similar search behavior to NSGA-II. We also demonstrate that the objective space normalization in NSGA-III is not stable for multi-objective problems with inverted triangular Pareto fronts.

In evolutionary multi-objective optimization (EMO), reproducing high-quality offspring solutions is a key factor in developing effective EMO algorithms (EMOAs). Many model-based methods have been proposed to generate high-quality offspring solutions. Almost all studies use the solutions in the current and previous generations as the training data. When using existing solutions as training data, the neural network is effective at identifying promising improvement directions for poor solutions. However, it does not perform well in determining effective improvement directions for solutions that are already good. In this paper, we propose a novel method that uses momentum moves to generate new solutions and includes the good new solutions in the training dataset. The momentum method offers a novel approach to enhancing the quality of the current solutions without incurring any additional computation costs. Our approach is demonstrated on both artificial and real-world problems, and compared with the algorithm using the traditional training dataset. Results show that including the momentum-based new solutions in the training dataset significantly improves the computational efficiency of the algorithm.

For a multi-objective optimization problem, an inverse model approximates a mapping from the objective space to the decision space. Recently, the inverse model has been used in some evolutionary multi-objective optimization algorithms (EMOAs) to generate offspring solutions. In those algorithms, the inverse model is usually built based on the current population and is used to generate offspring solutions around the current population. In this paper, we utilize both the current and previous populations to estimate the locations of future (i.e., improved) solutions in the objective space. The estimated objective vectors are presented to the inverse model to generate improved solutions in the decision space for future generations. Thus, our inverse model is used to generate better solutions than the current solutions instead of generating new solutions by interpolating the current solutions. Based on this idea, a solution generation method is proposed, which can be easily embedded into almost all EMOAs. We embed our method into two standard EMOAs (i.e., NSGA-II and NSGA-III) and two inverse model-based EMOAs (i.e., IM-MOEA and IM-MOEA/D). Our experimental results show that our method improves the performance of these EMOAs in most of the three-objective WFG and LSMOP problems.

Multi-objective neural architecture search (NAS) aims to find a set of model architectures to achieve different trade-offs between the model performance and the model complexity. The evaluation of the model performance usually requires a time-consuming training process, which is the main bottleneck in a NAS algorithm. To address this issue, many training-free metrics have been proposed in the literature to estimate the model performance without training. In this paper, we examine 13 training-free metrics on a two-objective NAS problem based on NAS-Bench-201. Our results show that the training-free metric which has a high correlation with the model performance does not always generate an estimated Pareto front with high quality. We embed these training-free metrics into four widely-used evolutionary multi-objective optimization algorithms (EMOAs) to solve the two-objective NAS problem. Our results show that the EMOAs with the Synaptic Flow metric obtain the best approximation of the Pareto front among the 13 training-free metrics.

Momentum is a technique that adds the momentum moves from the earlier iterations into the current update to accelerate convergence. While the momentum technique has been widely used in single-objective optimization, its application in evolutionary multi-objective optimization (EMO) has not gained much attention. Since EMO algorithms are population-based algorithms, how to choose solutions for applying momentum becomes an important issue. Inspired by Polyak’s momentum method and Nesterov’s momentum method in single-objective optimization, we propose four different momentum methods for EMO. Our findings demonstrate that the performance of EMOAs with momentum is strongly affected by the choice of solutions to which momentum moves are applied.

NSGA-II, the most well-known evolutionary multi-objective optimization (EMO) algorithm, is widely believed to be ineffective at handling many-objective optimization problems (i.e., problems with four or more conflicting objectives). Since NSGA-II is a Pareto dominance-based EMO algorithm, it loses strong selection pressure as the percentage of non-dominated solutions increases. Additionally, if dominance-resistant solutions (DRSs) exist, they further impair the convergence ability of NSGA-II. Two approaches have been proposed to improve convergence and eliminate DRSs. One is to increase the dominated region of each solution, while the other is to increase the correlation among objectives. These two approaches work well for many-objective optimization and are equivalent under certain conditions. However, they require appropriate parameter values for different problem types. To avoid the difficulty of setting parameter values appropriately, we propose a dual population-based NSGA-II (called DP-MNSGA-II) that utilizes two different sub-populations for objective modification. One subpopulation uses a small parameter value (i.e., small modification), while the other uses a relatively large value for objective modification in NSGA-II. Experimental results show that the proposed DP-MNSGA-II works well on different problems without the need for careful parameter tuning.

Multi-modal multi-objective optimization has become a hot research topic in the evolutionary multi-objective optimization (EMO) community. To support this line of study, many multi-modal multi-objective test problems have been developed to better understand the behavior of each algorithm in achieving a good balance between convergence and diversity in the decision space. However, the trade-off between decision space diversity and objective space diversity has not been well investigated. In this paper, first, we clearly explain that the widely used DTLZ1–4 test problems exhibit multi-modality (i.e., there exists a many-to-one mapping from the Pareto set to the Pareto front) despite their frequent use as standard benchmark problems for multi-objective optimization. Then, we demonstrate that the trade-off between decision space diversity and objective space diversity can be visually examined using the DTLZ1–4 problems with three or more objectives. Using these four test problems, we examine the search behavior of three standard EMO algorithms and three multi-modal multi-objective evolutionary algorithms (MMEAs). Experimental results show that uniformly distributed solutions obtained by the standard EMO algorithms in the objective space do not have good uniformity in the decision space. In contrast, solution sets obtained by different MMEAs show different trade-offs between decision space diversity and objective space diversity.

Subset selection, a fundamental problem in various domains, is to choose a subset of elements from a large candidate set under a given objective or multiple objectives. Pareto optimization for subset selection (POSS) has emerged as a powerful paradigm for addressing subset selection problems. Recently, some POSS variants have been proposed to further improve its performance. In this article, we propose a new POSS variant, named targeted POSS (TPOSS). TPOSS differs from POSS in four aspects: 1) problem formulation; 2) population initialization; 3) mutation; and 4) environmental selection. The main idea of TPOSS is to focus the search on the target region of subset selection with respect to the subset cardinality in order to improve the search efficiency. We conduct comprehensive experiments to compare TPOSS with six state-of-the-art algorithms on three subset selection tasks (i.e., sparse regression, unsupervised feature selection, and hypervolume subset selection) where the size of the candidate sets ranges from 20 to 400. Experimental results show that with respect to the objective value of the best feasible subset, TPOSS outperforms the other algorithms on all the three tasks, which suggests the potential of TPOSS to enhance subset selection in various domains.

Recently, multi-objective evolutionary algorithms (MOEAs) with an unbounded external archive (UEA) have received increasing attention in the evolutionary multi-objective optimization community. Its basic idea is to store all examined solutions during the optimization process and select representative solutions as the final output for the decision-maker (DM). Although many studies have investigated MOEAs with UEA, there is a lack of studies focusing on the final solution selection. Actually, selecting a good solution from UEA that meets the requirements of the DM is a challenging task due to the limited information processing capacity of the human decision-maker. Moreover, in many real-world scenarios, decision-makers often prefer not to evaluate a large number of solutions and may not have clear preferences over objectives. To fill this gap in post-processing for MOEAs with UEA, this paper proposes an interactive final solution selection (IFSS) method for multi-objective optimization. The proposed IFSS method aims to provide a good final solution through several interactions with the DM. In other words, the DM can obtain a satisfying solution after evaluating only a small number of solutions even without providing clearly specific preferences. Furthermore, a calibration strategy is introduced to significantly improve the performance of IFSS by slightly increasing the number of interactions. Extensive experiments are conducted on various test problems to demonstrate the effectiveness of the proposed IFSS method.

Recently, several real-world multi-objective optimization problem suites have been proposed to facilitate the evaluation of the performance of evolutionary multi-objective optimization (EMO) algorithms. In spite of the importance of using real-world problems to evaluate EMO algorithms, their characteristics are not well understood compared to artificial test problems. Thus, there is a need to examine and understand the challenges posed by these real-world problems. In this study, we attempt to explore the characteristics of the most recently proposed real-world application suite (i.e., RWA suite). Six EMO algorithms are evaluated on the RWA suite, including three classic algorithms and three recently-proposed algorithms. Based on the performance comparison results, we systematically analyze the RWA suite in terms of convergence and diversity difficulties.

To improve the convergence ability of evolutionary multi-objective optimization algorithms (EMOAs), various strategies have been proposed. One effective strategy is to use good momentum from the previous generations to create new solutions. However, the definition of good momentum has not been carefully studied. In this paper, we propose five different definitions of good momentum for EMOAs. Then, we explain their integration into popular EMOAs such as NSGA-II, MOEA/D, and SMS-EMOA. Through computational experiments, we demonstrate that the use of an appropriate definition of good momentum greatly accelerates the convergence of EMOAs on both artificial test problems and real-world problems, particularly on large-scale problems.

Recently, there has been a growing interest in large-scale multiobjective optimization problems within the evolutionary multiobjective optimization (EMO) community. These problems involve hundreds or thousands of decision variables and multiple conflicting objectives, which pose significant challenges for conventional EMO algorithms (EMOAs). It is generally believed that EMOAs have difficulty in efficiently finding good non-dominated solutions as the number of decision variables increases. To address this issue, in this paper, we propose a novel method that incorporates heuristic initialization and knowledge-based mutation into EMOAs for solving large-scale multi-objective 0-1 knapsack problems. Various large-scale multi-objective 0-1 knapsack problems with an arbitrary number of constraints are generated as test problems to evaluate the effectiveness of the proposed method. Experimental results show that the proposed novel initialization and mutation method significantly improves the performance of the original EMOAs in terms of both the convergence speed in early generations and the quality of the final population.

Newly proposed many-objective algorithms have been almost always compared with NSGA-III for performance evaluation. Since the authors of the NSGA-III paper have not provided any source code, researchers usually use an available implementation in popular optimization platforms. This can lead to unreliable comparison results if different performance of NSGA-III is obtained depending on the choice of a platform. In this paper, we show that the implementations of NSGA-III are slightly different between the two most frequently used EMO optimization platforms: PlatEMO and pymoo. Then, we examine the effect of the implementation difference on the performance of NSGA-III in each platform. Our experimental results show that almost the same results are obtained from the two implementations on the frequently-used DTLZ test problems. However, our experimental results also show that clearly different results are obtained from the two implementations on multi-objective combinatorial optimization problems. Finally, we demonstrate that the weaker performance of the PlatEMO implementation of NSGA-III can be improved by replacing its normalization mechanism with the corresponding mechanism in Pymoo. That is, our experimental results show that small differences in the normalization mechanisms of the two implementations lead to large differences in their performance on multi-objective combinatorial optimization problems.

An important issue in hypervolume-based evolutionary multi-objective optimization (EMO) algorithms is the specification of a reference point for hypervolume calculation. However, its appropriate specification has not been carefully studied in the literature. Some recent studies have pointed out the importance and difficulty of the reference point specification. Its appropriate specification depends on problem characteristics such as the Pareto front shape and the number of objectives. In this paper, the difficulty of the reference point specification in hypervolume-based EMO algorithms is circumvented by using two reference points. Instead of using only a single reference point, we propose a new hypervolume-based EMO algorithm that can effectively utilize two reference points cooperatively. Experimental results show that the proposed algorithm has good and robust performance on a wide range of test problems. In comparison to hypervolume-based EMO algorithms with only a single reference point, the proposed algorithm can find a wider and more uniformly distributed solution set. On a recently proposed real-world problem suite, the proposed algorithm shows competitive performance in comparison to state-of-the-art algorithms.

A long-standing question in the evolutionary multi-objective (EMO) community is how to generate a good initial population for EMO algorithms. Intuitively, as the starting point of optimization, a good initial population can have positive effects on the performance of EMO algorithms. However, in most existing EMO algorithms, one of the commonly-used initialization methods is to randomly generate a set of solutions as an initial population. One possible approach to improve random initialization is to include one or more Pareto optimal (near Pareto optimal) solution(s) in the initial population, which are expected to provide useful information and knowledge on the optimized problem. In this paper, to investigate the effectiveness of this initialization idea, we examine and quantify the effects of including one or more Pareto optimal solution(s) in the initial population on the performance of EMO algorithms. Experimental results demonstrate that it is worthwhile to first obtain and then include some Pareto optimal solutions in the initial population. Through a number of experiments and algorithm behavior analysis, this study provides supports and insights into EMO algorithm design and motivates further research on population initialization for EMO algorithms.

In the field of evolutionary multi-objective optimization, it is well known that dominance-based algorithms do not work well on many-objective problems. This is because almost all solutions in a population become non-dominated in early generations. Two approaches have been proposed to decrease the number of non-dominated solutions. One is to increase the dominated region by each solution: dominance modification. The other is to increase the correlation among objectives: objective modification. In this paper, first we show that these two approaches can be viewed as the same approach. We also explain that some regions of the Pareto front are dominated when the dominated region is increased. Next, we numerically examine the effects of dominance modification on the performance of NSGA-II on many-objective test problems. Through computational experiments, we demonstrate that its positive and negative effects are clearly shown by the hypervolume (HV) and inverted generational distance (IGD) indicators, respectively. Then, we discuss why these two indicators emphasize different effects of dominance modification using the optimal distribution of solutions for each indicator. Finally, we explain that objective space normalization is needed in dominance modification whereas it has no effects on the Pareto dominance relation.

Population initialization is always needed in evolutionary multi-objective optimization (EMO) algorithms. Intuitively, a well-designed initialization method can help facilitate the evolutionary process and improve the performance of EMO algorithms. However, very few studies have investigated the effects of initialization methods on the performance of EMO algorithms. Many existing EMO algorithms randomly generate an initial population to start the evolutionary process. To fill this research gap and attract more attention from EMO researchers to this important yet under-explored issue, in this paper, we examine the effects of various initialization methods that may become promising alternatives to the commonly-used random initialization method. Each initialization method is evaluated through computational experiments on test problems of various sizes with 5–1000 decision variables. Experimental results clearly demonstrate the advantage of well-designed initialization methods over the random initialization method. This study provides useful insights into EMO algorithm design and motivates further research on population initialization.

Recently, it has been pointed out in many studies that the performance of evolutionary multi-objective optimization (EMO) algorithms can be improved by selecting solutions from all examined solutions stored in an unbounded external archive. This is because in general the final population is not the best subset of the examined solutions. To obtain a good final solution set in such a solution selection framework, subset selection from a large candidate set (i.e., all examined solutions) has been studied. However, since good subsets cannot be obtained from poor candidate sets, a more important issue is how to find a good candidate set, which is the focus of this paper. In this paper, we first visually demonstrate that the entire Pareto front is not covered by the examined solutions through computational experiments using MOEA/D, NSGA-III and SMS-EMOA on DTLZ test problems. That is, the examined solution set stored in the unbounded archive has some large holes (i.e., some uncovered area of the Pareto front). Next, to evaluate the quality of the examined solution set (i.e., to measure the size of the largest hole), we propose the use of a variant of the inverted generational distance (IGD) indicator. Then, we propose a simple modification of EMO algorithms to improve the quality of the examined solution set. Finally, we demonstrate the effectiveness of the proposed modification through computational experiments.

The multiobjective evolutionary algorithm based on decomposition (MOEA/D) with the penalty-based boundary intersection (PBI) function (denoted as MOEA/D-PBI) has been frequently used in many studies in the literature. One essential issue in MOEA/D-PBI is its penalty parameter value specification. However, it is not easy to specify the penalty parameter value appropriately. This is because MOEA/D-PBI shows different search behavior when the penalty parameter values are different. The PBI function with a small penalty parameter value is good for convergence. However, the PBI function with a large value of penalty parameter is needed to preserve the diversity and uniformity of solutions. Although some methods for adapting the penalty parameter value for each weight vector have been proposed, they usually lead to slow convergence. In this article, we propose the idea of using two different values of penalty parameter simultaneously in MOEA/D-PBI. Although the idea is simple, the proposed algorithm is able to utilize both the convergence ability of a small penalty parameter value and the diversification ability of a large penalty parameter value of the PBI function. Experimental results demonstrate that the proposed algorithm works well on a wide range of test problems.

Multi-modal multi-objective optimization problems have multiple equivalent Pareto sets, each of which is mapped to the entire Pareto front. A number of multi-modal multi-objective algorithms have been proposed to find all equivalent Pareto sets. Their performance is evaluated by computational experiments on multi-modal multi-objective test problems. A common feature of those test problems is that a single point on the Pareto front in the objective space corresponds to multiple clearly separated Pareto optimal solutions in the decision space. In this paper, we propose a new type of multi-modal multi-objective test problems where a single point on the Pareto front corresponds to an infinite number of Pareto optimal solutions (i.e., a subset of the decision space). This means that the mapping from the Pareto set in the decision space to the Pareto front in the objective space is a set-to-point mapping. For example, all points on a line in the decision space are mapped to the same single point on the Pareto front. As a result, the dimensionality of the Pareto set is larger than that of the Pareto front. We examine the search behavior of multi-modal multi-objective algorithms using the proposed test problems. Some interesting observations are reported.

Recently, large-scale multiobjective optimization has received increasing attention from the evolutionary multiobjective optimization (EMO) community. This has led to the emergence of a specialized research area called evolutionary large-scale multiobjective optimization (ELMO). In general, it is believed that multiobjective optimization problems become more difficult as the number of decision variables increases. However, the following two counterintuitive observations are obtained from careful examinations of recent ELMO studies. One is that experimental results on some large-scale multiobjective test problems were improved by increasing the number of decision variables. The other is that better results were obtained for some other large-scale multiobjective test problems by conventional EMO algorithms (EMOAs) than state-of-the-art ELMO algorithms (ELMOAs). These observations suggest that ELMOAs have not always been evaluated on appropriate test problems. Moreover, their performance is not always better than the performance of conventional EMOAs. In this letter, we first re-examine the performance of ELMOAs and conventional EMOAs on a wide variety of scalable multiobjective test problems. Then, counterintuitive experimental results are analyzed using the anytime performance evaluation scheme and distributions of randomly generated initial solutions. Based on the analysis, suggestions on how to handle large-scale multiobjective test problems with counterintuitive results are proposed.

In the evolutionary multi-objective optimization (EMO) community, an unbounded external archive has been used in some studies for evaluating the performance of EMO algorithms. Those studies show that the unbounded external archive often includes better solutions than the final population. Thus, it is likely that the search ability of an EMO algorithm can be improved by periodically updating the current population using the unbounded external archive (i.e., by periodically choosing good solutions from all the examined solutions as the current population). However, the usefulness of such a global generation update scheme has not been studied in the literature. In this paper, we examine the effect of the periodical global generation update on the performance of well-known and frequently-used EMO algorithms: NSGA-II, MOEA/D and NSGA-III. We use the PBI function with uniformly distributed weight vectors for the periodical global generation update. In our computational experiments, we obtain clearly improved results by the periodical global generation update. We also examine the effect of the frequency of the global generation update (e.g., every 20 generations) on the performance of each EMO algorithm and its run time.

The multi-objective evolutionary algorithm based on decomposition (MOEA/D) is one of the most popular algorithms in the field of evolutionary multi-objective optimization (EMO). Even though MOEA/D has been widely used in many studies, it is likely that the performance of MOEA/D is not always optimized since the same MOEA/D implementation is often used on various problems with different characteristics. However, obtaining an appropriate implementation of MOEA/D for a different problem is not always easy, since there exists a wide variety of choices for the components and parameters in MOEA/D. In this paper, we examine the use of a genetic algorithm-based hyper-heuristic procedure to offline tune MOEA/D on a single test problem, a set of similar test problems, and a set of various test problems. A total of 26 benchmark test problems are used in our study. Experimental results show that the MOEA/D tuned for a set of various test problems does not always perform well. It is also shown that the MOEA/D tuned for a single test problem and for a set of similar test problems always has high performance. Our experimental results strongly suggest the necessity of using a tuning procedure to obtain a different MOEA/D implementation for a different type of problems.

The quality of solutions in multiobjective evolutionary algorithms (MOEAs) is usually evaluated by objective functions. However, function evaluations (FEs) are usually time-consuming in real-world problems. A large number of FEs limit the application of MOEAs. In this paper, we propose a fuzzy classifier-based selection strategy to reduce the number of FEs of MOEAs. First, all evaluated solutions in previous generations are used to build a fuzzy classifier. Second, the built fuzzy classifier is used to predict each unevaluated solution's label and its membership degree. The reproduction procedure is repeated to generate enough offspring solutions (classified as positive by the classifier). Next, unevaluated solutions are sorted based on their membership degrees in descending order. The same number of solutions as the population size are selected from the top of the sorted unevaluated solutions. Then, the best half of the chosen solutions are selected and stored in the new population without evaluations. The other half solutions are evaluated. Finally, the evaluated solutions are used together with evaluated current solutions for environmental selection to form another half of the new population. The proposed strategy is integrated into two MOEAs. Our experimental results demonstrate the effectiveness of the proposed strategy on reducing FEs.

The research on large-scale multi- and many-objective optimization has received increasing attention in the evolutionary multi-objective optimization (EMO) community. A number of large-scale EMO algorithms based on different strategies (e.g., divide-and-conquer, coevolution, and dimensionality reduction) have been proposed over the last decade. The performance of the large-scale EMO algorithms was empirically evaluated using several benchmark test suites, including the ZDT, DTLZ, WFG, MaF, UF and LSMOP test suites. Even though these test suites are theoretically scalable to any number of decision variables, they are not necessarily appropriate for examining the performance of large-scale EMO algorithms. In fact, among these benchmark test suites, only the LSMOP test suite is specifically designed to test the performance of large-scale EMO algorithms. In this paper, we propose a new scalable multi- and many-objective test problem for examining large-scale EMO algorithms. The proposed test problem has the following features: 1) the number of objectives and decision variables can be arbitrarily specified; 2) the interaction strength among the objectives can be adjusted by a correlation parameter. The performance of six EMO algorithms is examined on the new test problem. Our experimental results show that the proposed new test problem poses difficulties to some state-of-the-art large-scale EMO algorithms.

Hypervolume subset selection (HSS) aims to select a subset with a fixed size from a candidate solution set so that the hypervolume of the subset is maximized. The greedy HSS (GHSS) is the most efficient way for solving the HSS problem. When we use GHSS, we implicitly assume that well-distributed solutions over the entire Pareto front will be selected. However, the distribution of selected solutions by GHSS has not been studied. In this paper, we investigate this issue by examining selected solution subsets for different reference point specifications in GHSS. First, we show that a sufficiently large reference point is a good choice for GHSS to select a well-distributed subset on a triangular Pareto front. However, it is not easy to properly specify a reference point for an inverted triangular Pareto front. Then, we propose a dynamic reference point specification method for GHSS to select a well-distributed subset for various types of Pareto fronts. Static and dynamic reference point specifications are compared through computational experiments using 3- and 5-objective candidate solution sets from various types of Pareto front shapes. The experimental results demonstrate the effect of different reference point specifications on the subsets selected by GHSS and the usefulness of the dynamic reference point specification for GHSS.

Hypervolume is widely used as a performance indicator in the field of evolutionary multiobjective optimization (EMO). It is used not only for performance evaluation of EMO algorithms (EMOAs) but also in indicator-based EMOAs to guide the search. Since its initial proposal in the late 1990s, a wide variety of studies have been done on various topics, including hypervolume calculation, optimal μ-distribution, subset selection, hypervolume-based EMOAs, and extensions of the hypervolume indicator. However, currently there is no work to systematically survey the hypervolume indicator for these topics whereas it has been frequently used in the EMO field. This article aims to fill this gap and provide a comprehensive survey on the hypervolume indicator. We expect that this survey will help EMO researchers to understand the hypervolume indicator more deeply and thoroughly, and promote further utilization of the hypervolume indicator in the EMO field.

As an important branch in the field of soft computing, TSK fuzzy systems have been diversely applied to supervised learning in recent years. However, real-world data may contain label noise, which has a negative impact on supervised learning. Label noise samples change the distribution of samples in each class, mislead learning algorithms and make classification problems more complicated. There are various sources of label noise, such as wrong assignment of labels during the data collection, contamination during the data storage, and so on. Thus, it is usually costly and time-consuming to obtain data with no label noise. When dealing with label noise data, existing TSK fuzzy system algorithms still have room for improvement. This article proposes a robust TSK fuzzy system based on semisupervised learning for label noise data (RTSK-FS-SS). By introducing an intuitionistic fuzzy set method, the proposed algorithm can detect label noise samples. An improved learning vector quantization is further adopted to overcome the challenge that traditional unsupervised learning-based antecedent part generation processes unable to make full use of the label information of training samples. Finally, we discard the label of suspicious samples and a consequent parameter learning method based on semisupervised learning is proposed. The proposed algorithm is validated using extensive experiments.

In general, performance comparison results of optimization algorithms depend on the parameter specifications in each algorithm. For fair comparison, it may be needed to use the best specifications for each algorithm instead of using the same specifications for all algorithms. This is because each algorithm has its best specifications. However, in the evolutionary multi-objective optimization (EMO) field, performance comparison has usually been performed under the same parameter specifications for all algorithms. Especially, the same population size has always been used. In this paper, we discuss this practice from a viewpoint of fair comparison of EMO algorithms. First, we demonstrate that performance comparison results depend on the population size. Next, we explain a new trend of performance comparison where each algorithm is evaluated by selecting a pre-specified number of solutions from the examined solutions (i.e., by selecting a solution subset with a pre-specified size). Then, we discuss the selected subset size specification. Through computational experiments, we show that performance comparison results do not strongly depend on the selected subset size while they depend on the population size.

In the evolutionary multi-objective optimization (EMO) community, hypervolume (HV) has been frequently used to evaluate the performance of EMO algorithms. The HV is a Pareto compliant indicator which can simultaneously evaluate both the convergence of solutions to the Pareto front and their diversity. No other Pareto compliant indicator is known. In the EMO community, it is implicitly assumed that a set of uniformly distributed solutions over the entire Pareto front including its boundary has the best HV value. This is true for a linear Pareto front of a two-objective problem when a reference point for HV calculation is not too close to the Pareto front. In this paper, we numerically examine this issue for three-objective problems. We perform computational experiments to search for the optimal distribution of a small number of solutions for HV maximization. This is to visually explain the characteristic features of the optimal distribution. Our experimental results clearly show that a set of uniformly distributed solutions is not always optimal for HV maximization. It is also shown that the optimal distribution for HV maximization is often inconsistent with our intuition. For example, a set of ten solutions systematically generated by Das and Dennis method is not optimal.

In machine learning field, feature selection can be treated as a bi-objective optimization problem. It is reported that a decomposition-based evolutionary multi-objective optimization algorithm (i.e., MOEA/D-STAT) has good diversity performance when coping with feature selection. However, feature selection is also a time-consuming problem considering a large dataset it involves. The computation time can be easily reduced by introducing the parallelization into MOEA/D-STAT, thanks to the decomposition idea of MOEA/D. To the best of our knowledge, this is the first attempt to implement the parallelization of MOEA/D-STAT for feature selection. In this paper, we consider both master-slave models and island models, which are two different approaches of parallelization. In the master-slave models, different offspring assignment mechanisms are considered. In the island models, different island size specification mechanisms are examined. Our experimental results show that the master-slave models can achieve higher speedup and better performance than the island models.

In the evolutionary multi-objective optimization (EMO) community, it is usually assumed that the final population is presented to the decision maker as the result of the execution of an EMO algorithm. Recently, an unbounded external archive was used to evaluate the performance of EMO algorithms in some studies where a pre-specified number of solutions are selected from all the examined non-dominated solutions. In this framework, which is referred to as the solution selection framework, the final population does not have to be a good solution set. Thus, the solution selection framework offers higher flexibility to the design of EMO algorithms than the final population framework. In this paper, we examine the design of multi-objective evolutionary algorithm based on decomposition (MOEA/D) under these two frameworks. First, we show that the performance of MOEA/D is improved by linearly changing the reference point specification during its execution through computational experiments with various combinations of initial and final specifications. Robust and high performance of the solution selection framework is observed. Then, we examine the use of a genetic algorithm-based offline hyper-heuristic method to find the best configuration of MOEA/D in each framework. Finally, we further discuss solution selection after the execution of an EMO algorithm in the solution selection framework.

The use of Fuzzy Inference System (FIS) in decision making problems has received little attention so far. This may be due to the difficulty in gathering a complete set of fuzzy rules, which is free from noise, and the complexity in constructing an FIS model that is able to satisfy a number of important properties, including the monotonicity property. Previously, we have proposed a single-input Monotone-Interval FIS (MI-FIS) model, which can handle incomplete and non-monotone fuzzy rules. Besides that, we have proposed the idea of a monotone test (MT) for a set of fuzzy rules, which give an indication pertaining to the degree of monotonicity of a fuzzy rules set. In this paper, a multi-input MI-FIS model is firstly presented. The focus of this paper is on the use of MI-FIS and MT for undertaking multi expert decision-making (MEDM) problems. A three-phase MEDM framework consists of modelling, aggregation, and exploitation phases is proposed. In the modelling phase, an MT index for each fuzzy rule base from each expert, which is potentially non-monotone and incomplete, is obtained. The provided fuzzy rule bases are also modelled as MI-FISs. In the aggregation phase, an overall collective rating score of an alternative from a number of experts is obtained through the fuzzy weighted averaging operator. We suggest including MT as part of the aggregation phase. In exploitation phase, a rank ordering procedure among the alternatives is established using a possibility method. The developed framework is evaluated with simulated information. The results show that including the MT index in the aggregation phase is able to increase the robustness of the proposed FIS-MEDM model in the presence of noisy fuzzy rule sets.

To maintain the monotonicity property of a fuzzy inference system, a monotonically ordered and complete set of fuzzy rules is necessary. However, monotonically ordered fuzzy rules are not always available, e.g., errors in human judgments lead to nonmonotone fuzzy rules. The focus of this paper is on a new monotone fuzzy rule relabeling (MFRR) method that is able to relabel a set of nonmonotone fuzzy rules to meet the monotonicity property with reduced computation. Unlike the brute-force approach, which is susceptible to the combinatorial explosion problem, the proposed MFRR method explores within a reduced search space to find the solutions, therefore decreasing the computational requirements. The usefulness of the proposed method in undertaking failure mode and effect analysis problems is demonstrated using publicly available information. The results indicate that the MFRR method can produce optimal solutions with reduced computational time.

A search in the literature reveals that mathematical conditions (usually sufficient conditions) for the Fuzzy Inference System (FIS) models to satisfy the monotonicity property have been developed. A monotonically-ordered fuzzy rule base is important to maintain the monotonicity property of an FIS. However, it may difficult to obtain a monotonically-ordered fuzzy rule base in practice. We have previously introduced the idea of fuzzy rule relabeling to tackle this problem. In this paper, we further propose a monotonicity index for the FIS system, which serves as a metric to indicate the degree of a fuzzy rule base fulfilling the monotonicity property. The index is useful to provide an indication whether a fuzzy rule base should (or should not) be used in practice, even with fuzzy rule relabeling. To illustrate the idea, the zero-order Sugeno FIS model is exemplified. We add noise as errors into the fuzzy rule base to formulate a set of non-monotone fuzzy rules. As such, the metric also acts as a measure of noise in the fuzzy rule base. The results show that the proposed metric is useful to indicate the degree of a fuzzy rule base fulfilling the monotonicity property.

In this paper, a new online updating framework for constructing monotonicity-preserving Fuzzy Inference Systems (FISs) is proposed. The framework encompasses an optimization-based Similarity Reasoning (SR) scheme and a new monotone fuzzy rule relabeling technique. A complete and monotonically-ordered fuzzy rule base is necessary to maintain the monotonicity property of an FIS model. The proposed framework attempts to allow a monotonicity-preserving FIS model to be constructed when the fuzzy rules are incomplete and not monotonically-ordered. An online feature is introduced to allow the FIS model to be updated from time to time. We further investigate three useful measures, i.e., the belief, plausibility, and evidential mass measures, which are inspired from the Dempster-Shafer theory of evidence, to analyze the proposed framework and to give an insight for the inferred outcomes from the FIS model.

A complete and monotonically-ordered fuzzy rule base is necessary to maintain the monotonicity property of a Fuzzy Inference System (FIS). In this paper, a new monotone fuzzy rule relabeling technique to relabel a non-monotone fuzzy rule base provided by domain experts is proposed. Even though the Genetic Algorithm (GA)-based monotone fuzzy rule relabeling technique has been investigated in our previous work [7], the optimality of the approach could not be guaranteed. The new fuzzy rule relabeling technique adopts a simple brute force search, and it can produce an optimal result. We also formulate a new two-stage framework that encompasses a GA-based rule selection scheme, the optimization based-Similarity Reasoning (SR) scheme, and the proposed monotone fuzzy rule relabeling technique for preserving the monotonicity property of the FIS model. Applicability of the two-stage framework to a real world problem, i.e., failure mode and effect analysis, is further demonstrated. The results clearly demonstrate the usefulness of the proposed framework.