Local search is a fundamental method in operations research and combinatorial optimisation. It has been widely applied to a variety of challenging problems, including multi-objective optimisation where multiple, often conflicting, objectives need to be simultaneously considered. In multi-objective local search algorithms, a common practice is to maintain an archive of all non-dominated solutions found so far, from which the algorithm iteratively samples a solution to explore its neighbourhood. A central issue in this process is how to explore the neighbourhood of a selected solution. In general, there are two main approaches: 1) systematic exploration and 2) random sampling. The former systematically explores the solution's neighbours until a stopping condition is met -- for example, when the neighbourhood is exhausted (i.e., the best improvement strategy) or once a better solution is found (i.e., first improvement). In contrast, the latter randomly selects and evaluates only one neighbour of the solution. One may think systematic exploration may be more efficient, as it prevents from revisiting the same neighbours multiple times. In this paper, however, we show that this may not be the case. We first empirically demonstrate that the random sampling method is consistently faster than the systematic exploration method across a range of multi-objective problems. We then give an intuitive explanation for this phenomenon using toy examples, showing that the superior performance of the random sampling method relies on the distribution of ``good neighbours''. Next, we show that the number of such neighbours follows a certain probability distribution during the search. Lastly, building on this distribution, we provide a theoretical insight for why random sampling is more efficient than systematic exploration, regardless of whether the best improvement or first improvement strategy is used.

Evolutionary Algorithms (EAs) have become the most popular tool for solving widely-existed multi-objective optimization problems. In Multi-Objective EAs (MOEAs), there is increasing interest in using an archive to store non-dominated solutions generated during the search. This approach can 1) mitigate the effects of population oscillation, a common issue in many MOEAs, and 2) allow for the use of smaller, more practical population sizes. In this paper, we analytically show that the archive can even further help MOEAs through reusing its solutions during the process of new solution generation. We first prove that using a small population size alongside an archive (without incorporating archived solutions in the generation process) may fail on certain problems, as the population may remove previously discovered but promising solutions. We then prove that reusing archive solutions can overcome this limitation, resulting in at least a polynomial speedup on the expected running time. Our analysis focuses on the well-established SMS-EMOA algorithm applied to the commonly studied OneJumpZeroJump problem as well as one of its variants. We also show that reusing archive solutions can be better than using a large population size directly. Finally, we validate our theoretical findings through experiments on well-known practical optimization problems.

Ore-flow blending in open pit mining is a complex real-time scheduling decision problem constrained by high dimensional spatiotemporal dynamics and stochastic uncertainties. Traditional methods often struggle to reconcile the need for long term global stability with the requirement for sub second response times to dynamic events. To address this, we propose a heterogeneous ant colony optimization framework with semantic manifold learning. The framework bridges the gap between static offline learning and dynamic online adaptation. In the offline phase, we introduce a semantic manifold learning mechanism combining genetic programming with quality-diversity optimization. By constructing a behaviorally diverse archive of elite heuristics, this method provides the online system with a "warm start" and diverse initial perspectives, significantly enhancing real-time responsiveness. In the online phase, a heterogeneous ant colony system is deployed where different ant sub populations are coupled with distinct elite rules from the offline archive. This architecture enables parallel exploration of the solution space from multiple semantic perspectives, granting the system intrinsic robustness and high adaptability. Extensive experiments on real-world mine demonstrate that this method significantly outperforms state-of-the-art techniques in ore flow stability control and exhibits superior robustness and scalability across varying road network topologies, equipment configurations, and dynamic scenarios.

Batch Bayesian Optimisation (BBO) has emerged as a potent approach for optimising expensive black-box functions. Central to BBO is the issue of selecting a number of solutions at the same time through a batch method, in the hope for them to represent good, yet different, trade-offs between exploitation and exploration. To address this issue, one of the recent advancements has leveraged multi-objective optimisation to simultaneously consider several acquisition functions (e.g., PI, EI, and LCB), allowing them to complement each other. However, acquisition functions may behave similarly (since they all aim for a good balance between exploitation and exploration), restricting the search on different promising areas. In this paper, we attempt to address the above issue. We directly treat exploitation (reflected by quality, i.e., the posterior mean) and exploration (reflected by uncertainty) as two objectives. When selecting trade-off solutions between the two objectives, we consider a dynamically updated Pareto front where the uncertainty changes once a solution is selected, thereby allowing exploration on different promising areas. Through an extensive experiment study, we show the effectiveness of the proposed method in comparison with state-of-the-arts in the area.

Recently, there has been growing interest within the theoretical community in analytically studying multi-objective evolutionary algorithms. This runtime analysis-focused research can help formally understand algorithm behaviour, explain empirical observations, and provide theoretical insights to support algorithm development and exploration. However, the test problems commonly used in the theoretical analysis are predominantly limited to problems with heavy "artificial" characteristics (e.g., symmetric, homogeneous objectives and linear Pareto fronts), which may not be able to well represent realistic scenarios. In this paper, we first discuss commonly used multi-objective functions in the theory domain and systematically review their features, limitations and implications to practical use. Then, we present several new functions with more realistic features, such as heterogeneous objectives, local optimality and nonlinearity of the Pareto front, through simply mixing and matching classical single-objective functions in the area (e.g., LeadingOnes, Jump and RoyalRoad). We hope these functions can enrich the existing test problem suites, and strengthen the connection between theoretic and practical research.

Modern configurable systems provide tremendous opportunities for engineering future intelligent software systems. A key difficulty thereof is how to effectively self-adapt the configuration of a running system such that its performance (e.g., runtime and throughput) can be optimized under time-varying workloads. This unfortunately remains unaddressed in existing approaches as they either overlook the available past knowledge or rely on static exploitation of past knowledge without reasoning the usefulness of information when planning for self-adaptation. In this paper, we tackle this challenging problem by proposing DLiSA, a framework that self-adapts configurable systems. DLiSA comes with two properties: firstly, it supports lifelong planning, and thereby the planning process runs continuously throughout the lifetime of the system, allowing dynamic exploitation of the accumulated knowledge for rapid adaptation. Secondly, the planning for a newly emerged workload is boosted via distilled knowledge seeding, in which the knowledge is dynamically purified such that only useful past configurations are seeded when necessary, mitigating misleading information. Extensive experiments suggest that the proposed DLiSA significantly outperforms state-of-the-art approaches, demonstrating a performance improvement of up to 229 % and a resource acceleration of up to$2.22 \times$on generating promising adaptation configurations. All data and sources can be found at our repository: https://github.com/ideas-labo/dlisa.

Evolutionary algorithms (EAs) have been widely applied to multi-objective optimization due to their population-based nature. Population update, a key component in multi-objective EAs (MOEAs), is usually performed in a greedy, deterministic manner. However, recent studies have questioned this practice and shown that stochastic population update (SPU), which allows inferior solutions have a chance to be preserved, can help MOEAs jump out of local optima more easily. Nevertheless, SPU risks losing high-quality solutions, potentially requiring a large population. Intuitively, a possible solution to this issue is to introduce an archive that stores the best solutions ever found. In this paper, we theoretically show that using an archive allows a small population and may enhance the search performance of SPU-based MOEAs. We examine two classic algorithms, SMS-EMOA and NSGA-II, on the bi-objective problem OneJumpZeroJump, and prove that using an archive can reduce the expected running time upper bound (even exponentially). The comparison between SMS-EMOA and NSGA-II also suggests that the (μ+μ) update mode may be more suitable for SPU than the (μ+1) update mode. We also validate our findings empirically. We hope this work may provide theoretical support to explore different ideas of designing algorithms in evolutionary multi-objective optimization.

The overlapping coalition structure generation problem (OCSGP) is a challenging computational problem in multi-agent systems. It focuses on selecting possibly overlapping coalitions from a set of agents to maximize the social welfare of all coalitions while containing all agents. However, in practical applications, coalitions may be formed to selectively respond to tasks from a pool of potential tasks assigned to agents. Consequently, this study considers OCSGP in a task-based setting, where each agent has finite resources and can only respond to tasks of interest, and each coalition can only take on mutually disjoint subsets of tasks. Specifically, we first present a model of the task-based OCSGP and investigate its computational complexity. Our theoretical results demonstrate that this specific OCSGP remains intractable even under restrictive assumptions. Subsequently, we develop a generic evolutionary algorithm framework (EAF) to find an approximately optimal overlapping coalition structure (OCS) in time quartic polynomial in the size of the instance. Particularly, we devise a specific solution-repair based heuristic of cubic time complexity to generate a feasible OCS. Finally, we compare the proposed EAF with a task-oriented heuristic and a hybrid algorithm for OCSGP, and examine its applicability in the pursuit-evasion problem. The experimental results reveal that the proposed EAF exhibits superior performance in finding feasible OCSs and demonstrates flexible adaptability to problem size and resource status.

Traveling salesman problem (TSP) is a combinatorial optimization problem, serving as basis for many real-world applications (e.g., transportation planning, circuit board design, and DNA sequencing). In TSP, it is common to encounter some dynamic events, for example, traffic jam, and roadworks in transportation planning. To deal with such dynamic TSP (DTSP) scenarios, numerous techniques have been designed over decades. In this article, we take a different perspective to study DTSP. Instead of focusing only on algorithm design for DTSP, we investigate how dynamic events in DTSP affect its fitness landscape (e.g., the location of local optimal solutions and the ruggedness level of search space). We consider three dynamic events, including node addition, node deletion and weight changes, and analyze how they affect the TSP with respect to the overall landscape structure and solution optimality. Experimental results show that the weight change event has great effect on the problem’s fitness landscape, introducing more local optima and reducing the basin of attraction for the global optimum. This may suggest that search algorithms need to have stronger exploration capability when handling weight change dynamic events. Furthermore, our experimental studies also demonstrate that the dynamic solution adaptation strategy on the original global optimum is effective for tracking the new optimum after dynamic changes.

Bayesian optimisation (BO) is an efficient approach for solving expensive optimisation problems, where acquisition functions play a major role in achieving the tradeoff between exploitation and exploration. The exploitation–exploration tradeoff is challenging; excessive focus on exploitation can stagnate the search, while too much exploration can slow convergence. Multi-objectivisation has been explored as an effective approach to mitigate the exploitation–exploration tradeoff problem. Along this line, in this article, we propose a Multi-Objectivisation-Based Adaptive Exploitation–Exploration Tradeoff Framework (MOEE) to balance exploitation and exploration in BO. MOEE considers the nondominated front formed by the exploitation and exploration objectives and adaptively switches the focus on exploration and exploitation on the basis of the search status. We verify our method on the 19 synthetic and practical problem instances with 1–20 dimensions, and the results show that our proposed multi-objectivisation framework can achieve a good balance between exploitation and exploration.

Multi-objective multi-point shortest path planning problems are commonly encountered in real-world applications. Numerous path planning algorithms have been proposed to accommodate different model assumptions. However, most existing algorithms can only identify a subset of the Pareto optimal paths and overlook equivalent Pareto optimal paths. Relying solely on a subset of Pareto optimal solutions is insufficient to effectively respond to unforeseeable road eventualities in the real-world traffic environment. In this paper, multi-objective multi-point shortest path planning problem is modeled as a multimodal multi-objective optimization problem with necessary points constrains. A multimodal multi-objective evolutionary algorithm using constraint dominance principle-based path comparison strategy and path similarity-based multimodal solutions selection strategy is proposed to address this problem. The proposed constraint dominance principle-based path comparison strategy can effectively navigate through large infeasible regions by relaxing necessary point constraints, thereby obtaining a true constrained Pareto front. The proposed path similarity-based multimodal solutions selection strategy can effectively balance the distribution of solutions in the decision space, thereby preserving multiple equivalent optimal solutions. The proposed algorithm is compared with five state-of-the-art path planning algorithms from the benchmark test suite derived from the 2021 IEEE CEC path planning competition, where city maps are adapted from real transportation networks in Chinese cities, in our experiments. The exceptional performance is demonstrated through thirty independent runs, yielding experimental results that showcase the superiority of the proposed algorithm on the test problem set. This superior performance highlights the potential for designing more resilient path planners suitable for scenarios affected by unpredictable road eventualities.

Local search has gained its popularity in addressing multi-objective combinatorial optimisation problems (MOCOPs) within the communities of evolutionary computation and operational research. The ease of defining the neighbourhood in discrete spaces of MOCOPs makes local search well-suited to conducting step-wise moves. On the other side, evolutionary algorithms are amongst very first choices in solving various multi-objective optimisation problems. Although most multi-objective evolutionary algorithms (MOEAs) are developed, tested and studied on the basis of continuous problems, when encountering a practical MOCOP, practitioners tend to resort to popular MOEAs. Therefore, a relevant question is that between MOEAs and local search heuristics, which one may be more suitable for MOCOPs. In this paper, we attempt to answer this question. We choose seven well-known "baseline" MOEAs and local search heuristics in the area and systematically study their behaviours on four MOCOPs. We find that, unsurprisingly, different search paradigms have their own sweet spots; it depends on problem types, problem settings and search budgets. However, surprisingly, there exists one search heuristic, SEMO, which can be seen as a "transition" between the two search paradigms (i.e., a simple MOEA or randomised local search), that performs consistently better than the other heuristics across all the settings.

Subset selection, which selects a subset of solutions according to certain criterion/indicator, is a topic closely related to evolutionary multiobjective optimization (EMO). The critical component of a multiobjective evolutionary algorithm (MOEA), environmental selection, is essentially a subset selection problem, i.e., selecting$N$solutions as the next-generation population from usually$2N$candidates (where$N$denotes the size of the population). Another use of subset selection is the solution preprocessing procedure for decision-making, in which typically a few representatives are selected from the final population or a large-size archive, in order not to overwhelm the decision maker. Existing work for subset selection in EMO is focused on developing greedy algorithms, but may suffer from being trapped in local optima. In this article, we approach the problem by providing a multiobjective evolutionary algorithmic framework. We consider several popular quality indicators for subset evaluation and present accelerated variants of a well-studied MOEA in the theoretical study area, global simple evolutionary multiobjective optimizer (GSEMO), for each indicator. We conduct rigorous theoretical analyses of the acceleration procedure. Moreover, we prove that our algorithms can achieve the best-so-far approximation guarantee by the submodularity of the indicators. We finally empirically show the effectiveness and scalability of the proposed algorithms, in addition to the potentials to be further improved by introducing popular MOEAs (e.g., using NSGA-II and MOEA/D to replace GSEMO).

Most multi-objective optimisation algorithms maintain an archive explicitly or implicitly during their search. Such an archive can be solely used to store high-quality solutions presented to the decision maker, but in many cases may participate in the search process (e.g., as the population in evolutionary computation). Over the last two decades, archiving, the process of comparing new solutions with previous ones and deciding how to update the archive/population, stands as an important issue in evolutionary multi-objective optimisation (EMO). This is evidenced by constant efforts from the community on developing various effective archiving methods, ranging from conventional Pareto-based methods to more recent indicator-based and decomposition-based ones. However, the focus of these efforts is on empirical performance comparison in terms of specific quality indicators; there is lack of systematic study of archiving methods from a general theoretical perspective. In this paper, we attempt to conduct a systematic overview of multi-objective archiving, in the hope of paving the way to understand archiving algorithms from a holistic perspective of theory and practice, and more importantly providing a guidance on how to design theoretically desirable and practically useful archiving algorithms. In doing so, we also present that archiving algorithms based on weakly Pareto compliant indicators (e.g., -indicator), as long as designed properly, can achieve the same theoretical desirables as archivers based on Pareto compliant indicators (e.g., hypervolume indicator). Such desirables include the property limit-optimal, the limit form of the possible optimal property that a bounded archiving algorithm can have with respect to the most general form of superiority between solution sets.

Deep neural networks have demonstrated impressive results in medical image analysis, but designing suitable architectures for each specific task is expertise-dependent and time-consuming. Neural architecture search (NAS) offers an effective means of discovering architectures. It has been highly successful in numerous applications, particularly in natural image classification. Yet, medical images possess unique characteristics, such as small regions and a wide variety of lesion sizes, that differentiate them from natural images. Furthermore, most current NAS methods struggle with high computational costs, especially when dealing with high-resolution image datasets. In this paper, we present a novel evolutionary neural architecture search method called Multi-Scale Training-Free Neural Architecture Search to address these challenges. Specifically, to accommodate the broad range of lesion region sizes in disease diagnosis, we develop a new reduction cell search space that enables the search algorithm to explicitly identify the optimal scale combination for multi-scale feature extraction. To overcome the issue of high computational costs, we utilize training-free indicators as performance measures for candidate architectures, which allows us to search for the optimal architecture more efficiently. More specifically, by considering the capability and simplicity of various networks, we formulate a multi-objective optimization problem that involves two training-free indicators and model complexity for candidate architectures. Extensive experiments on a large medical image benchmark and a publicly available breast cancer detection dataset are conducted. The empirical results demonstrate that our MSTF-NAS outperforms both human-designed architectures and current state-of-the-art NAS algorithms on both datasets, indicating the effectiveness of our proposed method.

A Software Product Line (SPL) is a set of software products that are built from a variability model. Real-world SPLs typically involve a vast number of valid products, making it impossible to individually test each of them. This arises the need for automated test suite generation, which was previously modeled as either a single-objective or a multi-objective optimization problem considering only objective functions. This article provides a completely different mathematical model by exploiting the benefits of Quality-Diversity (QD) optimization that is composed of not only an objective function (e.g., t -wise coverage or test suite diversity) but also a user-defined behavior space (e.g., the space with test suite size as its dimension). We argue that the new model is more suitable and generic than the two alternatives because it provides at a time a large set of diverse (measured in the behavior space) and high-performing solutions that can ease the decision-making process. We apply MAP-Elites, one of the most popular QD algorithms, to solve the model. The results of the evaluation, on both realistic and artificial SPLs, are promising, with MAP-Elites significantly and substantially outperforming both single- and multi-objective approaches, and also several state-of-the-art SPL testing tools. In summary, this article provides a new and promising perspective on the test suite generation for SPLs.

The multi-objective testing resource allocation problem (MOTRAP) is concerned on how to reasonably plan the testing time of software testers to save the cost and improve the reliability as much as possible. The feasible solution space of a MOTRAP is determined by its variables (i.e., the time invested in each component) and constraints (e.g., the pre-specified reliability, cost, or time). Although a variety of state-of-the-art constrained multi-objective optimisers can be used to find individual solutions in this space, their search remains inefficient and expensive due to the fact that this space is very tiny compared to the large search space. The decision maker may often suffer a prolonged but unsuccessful search that fails to return a feasible solution. In this work, we first formulate a heavily constrained MOTRAP on the basis of an architecture-based model, in which reliability, cost, and time are optimised under the pre-specified multiple constraints on reliability, cost, and time. Then, to estimate the feasible solution space of this specific MOTRAP, we develop theoretical and algorithmic approaches to deduce new tighter lower and upper bounds on variables from constraints. Importantly, our approach can help the decision maker identify whether their constraint settings are practicable, and meanwhile, the derived bounds can just enclose the tiny feasible solution space and help off-the-shelf constrained multi-objective optimisers make the search within the feasible solution space as much as possible. Additionally, to further make good use of these bounds, we propose a generalised bound constraint handling method that can be readily employed by constrained multi-objective optimisers to pull infeasible solutions back into the estimated space with theoretical guarantee. Finally, we evaluate our approach on application and empirical cases. Experimental results reveal that our approach significantly enhances the efficiency, effectiveness, and robustness of off-the-shelf constrained multi-objective optimisers and state-of-the-art bound constraint handling methods at finding high-quality solutions for the decision maker. These improvements may help the decision maker take the stress out of setting constraints and selecting constrained multi-objective optimisers and facilitate the testing planning more efficiently and effectively.

Software configuration tuning is essential for optimizing a given performance objective (e.g., minimizing latency). Yet, due to the software’s intrinsically complex configuration landscape and expensive measurement, there has been a rather mild success, particularly in preventing the search from being trapped in local optima. To address this issue, in this paper we take a different perspective. Instead of focusing on improving the optimizer, we work on the level of optimization model and propose a meta multi-objectivization (MMO) model that considers an auxiliary performance objective (e.g., throughput in addition to latency). What makes this model distinct is that we do not optimize the auxiliary performance objective, but rather use it to make similarly-performing while different configurations less comparable (i.e. Pareto nondominated to each other), thus preventing the search from being trapped in local optima. Importantly, by designing a new normalization method, we show how to effectively use the MMO model without worrying about its weight–the only yet highly sensitive parameter that can affect its effectiveness. Experiments on 22 cases from 11 real-world software systems/environments confirm that our MMO model with the new normalization performs better than its state-of-the-art single-objective counterparts on 82% cases while achieving up to 2:09× speedup. For 68% of the cases, the new normalization also enables the MMO model to outperform the instance when using it with the normalization from our prior FSE work under pre-tuned best weights, saving a great amount of resources which would be otherwise necessary to find a good weight. We also demonstrate that the MMO model with the new normalization can consolidate recent model-based tuning tools on 68% of the cases with up to 1:22× speedup in general.

In this paper, we show that when dealing with multi-objective combinatorial optimisation problems, the search, in different executions of a multi-objective evolutionary algorithm (MOEA), e.g., NSGA-II, tends to stagnate in different areas in the search space. In other words, the final populations obtained by an MOEA under multiple executions, which can be very close in the objective space, are located far away from each other in the search space. This phenomenon becomes more apparent with the increase of some type of problem complexity (e.g., the ruggedness level of problem landscape). Interestingly, the phenomenon only happens to combinatorial optimisation problems, but not to continuous ones. In this study, we consider three well-established MOEAs (NSGA-II, SMS-EMOA and MOEA/D) on two representative combinatorial optimisation problems (NK-landscape and TSP) and on two commonly used continuous problem suites (DTLZ and WFG). Experimental results show a clear difference between multi-objective combinatorial and continuous problems and suggest a need of more efforts to be put on developing effective MOEAs for combinatorial problems.

Evolutionary algorithms (EAs) have been widely and successfully applied to solve multi-objective optimization problems, due to their nature of population-based search. Population update is a key component in multi-objective EAs (MOEAs), and it is performed in a greedy, deterministic manner. That is, the next-generation population is formed by selecting the first population-size ranked solutions (based on some selection criteria, e.g., non-dominated sorting, crowdedness and indicators) from the collections of the current population and newly-generated solutions. In this paper, we question this practice. We analytically present that introducing randomness into the population update procedure in MOEAs can be beneficial for the search. More specifically, we prove that the expected running time of a well-established MOEA (SMS-EMOA) for solving a commonly studied bi-objective problem, OneJumpZeroJump, can be exponentially decreased if replacing its deterministic population update mechanism by a stochastic one. Empirical studies also verify the effectiveness of the proposed stochastic population update method. This work is an attempt to challenge a common practice for the population update in MOEAs. Its positive results, which might hold more generally, should encourage the exploration of developing new MOEAs in the area.

Audio recordings used as evidence have become increasingly important to litigation. Before their admissibility as evidence, an audio forensic expert is often required to help determine whether the submitted audio recordings are altered or authentic. Within this field, the copy-move forgery detection (CMFD), which focuses on finding possible forgeries that are derived from the same audio recording, has been an urgent problem in blind audio forensics. However, most of the existing methods require idealistic pre-segmentation and artificial threshold selection to calculate the similarity between segments, which may result in serious misleading and misjudgment especially on high frequency words. In this work, we present a robust method for detecting and locating an audio copy-move forgery on the basis of constant Q spectral sketches (CQSS) and the integration of a customised genetic algorithm (GA) and support vector machine (SVM). Specifically, the CQSS features are first extracted by averaging the logarithm of the squared-magnitude constant Q transform. Then, the CQSS feature set is automatically optimised by a customised GA combined with SVM to obtain the best feature subset and classification model at the same time. Finally, the integrated method, named CQSS-GA-SVM, is evaluated against the state-of-the-art approaches to blind detection of copy-move forgeries on real-world copy-move datasets with read English and Chinese corpus, respectively. The experimental results demonstrate that the proposed CQSS-GA-SVM exhibits significantly high robustness against post-processing based anti-forensics attacks and adaptability to the changes of the duplicated segment duration, the training set size, the recording length, and the forgery type, which may be beneficial to improving the work efficiency of audio forensic experts.

The objective of disassembly lines is to disassemble end-of-life products in a remanufacturing field. The disassembly line balancing problem (DLBP) considers how to allocate disassembly operations to operators on the disassembly line to optimize predetermined goals, such as cycle time. In practice, various environmental uncertainties (e.g., uncertain product quality) exist in the disassembly line. These uncertainties entail DLBP essentially a dynamic multiobjective optimization problem (DMOP). This study presents a dynamic DLBP (D-DLB) to model the effect of environmental uncertainties on the assignment of disassembly operations. Furthermore, a prediction-based dynamic optimization algorithm, termed domain generalization-based dynamic multiobjective evolutionary algorithm (DG-DMOEA), combining meta-learning with multiobjective optimization, is proposed to solve D-DLB. In DG-DMOEA, a meta-learning algorithm is employed to learn the parameters of a solution-generative model from the Pareto-optimal sets (POSs) in all historical environments. Subsequently, the solution-generative model is applied to generate a high-quality initial population that can assist multiobjective optimization algorithms in finding the POS in the new environment faster. Since no information in the new environment is required, learning can begin before the new environment arrives, significantly reducing computational time. Moreover, different solution-generative models can be designed for different DMOPs. Therefore, DG-DMOEA can thoroughly combine real-world problem properties to represent knowledge. The experimental results show that, compared with state-of-the-art methods, DG-DMOEA can considerably improve the quality of solutions and significantly enhance the ability to react quickly to environmental changes.

Balancing constraints and objective functions in constrained evolutionary multiobjective optimization is not an easy task. Overemphasis on constraints satisfaction may easily lead to the search to get stuck in local optimal regions, and overemphasis on objectives optimization may lead to substantial search resources wasted on infeasible regions. This article proposes a constrained multiobjective optimization algorithm, called CMOEA-SDE, aiming to achieve a good balance between the above two issues. To do so, CMOEA-SDE presents a strictly constrained dominance relation and a constrained shift-based density estimation strategy. Specifically, the former defines a new dominance relation that considers both constraint satisfaction and the objective functions. It favors good feasible solutions but still leaves room for infeasible solutions to be selected. Unlike most density estimation methods, which only consider the diversity of solutions, our shift-based density estimator covers both the feasibility and the diversity of solutions. That is, our estimator shifts the solutions’ positions based on the extent of the constraints they violate so that solutions violating constraints more severely are shifted to crowded areas, thus being eliminated early. Systematic experiments were conducted on four benchmark test suites and six real-world constrained multiobjective optimization problems. The experimental results suggest that the proposed algorithm can achieve very competitive performance against state-of-the-art constrained multiobjective evolutionary algorithms.

Evolving intelligent systems (EISs), particularly, the zero-order ones have demonstrated strong performance on many real-world problems concerning data stream classification while offering high model transparency and interpretability thanks to their prototype-based nature. Zero-order EISs typically learn prototypes by clustering streaming data online in a “one pass” manner for greater computation efficiency. However, such identified prototypes often lack optimality, resulting in less precise classification boundaries, thereby hindering the potential classification performance of the systems. To address this issue, a commonly adopted strategy is to minimize the training error of the models on historical training data or alternatively to iteratively minimize the intracluster variance of the clusters obtained via online data partitioning. This recognizes the fact that the ultimate classification performance of zero-order EISs is driven by the positions of prototypes in the data space. Yet, simply minimizing the training error may potentially lead to overfitting while minimizing the intracluster variance does not necessarily ensure the optimized prototype-based models to attain improved classification outcomes. To achieve better classification performance while avoiding overfitting for zero-order EISs, this article presents a novel multiobjective optimization approach, enabling EISs to obtain optimal prototypes via involving these two disparate but complementary strategies simultaneously. Five decision-making schemes are introduced for selecting a suitable solution to deploy from the final nondominated set of the resulting optimized models. Systematic experimental studies are carried out to demonstrate the effectiveness of the proposed optimization approach in improving the classification performance of zero-order EISs.

In presence of multiple objectives to be optimized in Search-Based Software Engineering (SBSE), Pareto search has been commonly adopted. It searches for a good approximation of the problem’s Pareto-optimal solutions, from which the stakeholders choose the most preferred solution according to their preferences. However, when clear preferences of the stakeholders (e.g., a set of weights that reflect relative importance between objectives) are available prior to the search, weighted search is believed to be the first choice, since it simplifies the search via converting the original multi-objective problem into a single-objective one and enables the search to focus on what only the stakeholders are interested in. This article questions such a “ weighted search first ” belief. We show that the weights can, in fact, be harmful to the search process even in the presence of clear preferences. Specifically, we conduct a large-scale empirical study that consists of 38 systems/projects from three representative SBSE problems, together with two types of search budget and nine sets of weights, leading to 604 cases of comparisons. Our key finding is that weighted search reaches a certain level of solution quality by consuming relatively less resources at the early stage of the search; however, Pareto search is significantly better than its weighted counterpart the majority of the time (up to 77% of the cases), as long as we allow a sufficient, but not unrealistic search budget. This is a beneficial result, as it discovers a potentially new “rule-of-thumb” for the SBSE community: Even when clear preferences are available, it is recommended to always consider Pareto search by default for multi-objective SBSE problems, provided that solution quality is more important. Weighted search, in contrast, should only be preferred when the resource/search budget is limited, especially for expensive SBSE problems. This, together with other findings and actionable suggestions in the article, allows us to codify pragmatic and comprehensive guidance on choosing weighted and Pareto search for SBSE under the circumstance that clear preferences are available. All code and data can be accessed at https://github.com/ideas-labo/pareto-vs-weight-for-sbse .

Configurable software systems can be tuned for better performance. Leveraging on some Pareto optimizers, recent work has shifted from tuning for a single, time-related performance objective to two intrinsically different objectives that assess distinct performance aspects of the system, each with varying aspirations to be satisfied, e.g., “ the latency is less than 10s ” while “ the memory usage is no more than 1GB ”. Before we design better optimizers, a crucial engineering decision to make therein is how to handle the performance requirements with clear aspirations in the tuning process. For this, the community takes two alternative optimization models: either quantifying and incorporating the aspirations into the search objectives that guide the tuning, or not considering the aspirations during the search but purely using them in the later decision-making process only. However, despite being a crucial decision that determines how an optimizer can be designed and tailored, there is a rather limited understanding of which optimization model should be chosen under what particular circumstance, and why. In this article, we seek to close this gap. Firstly, we do that through a review of over 426 articles in the literature and 14 real-world requirements datasets, from which we summarize four performance requirement patterns that quantify the aspirations in the configuration tuning. Drawing on these, we then conduct a comprehensive empirical study that covers 15 combinations of the state-of-the-art performance requirement patterns, four types of aspiration space, three Pareto optimizers, and eight real-world systems/environments, leading to 1,296 cases of investigation. Our findings reveal that (1) the realism of aspirations is the key factor that determines whether they should be used to guide the tuning; (2) the given patterns and the position of the realistic aspirations in the objective landscape are less important for the choice, but they do matter to the extents of improvement; (3) the available tuning budget can also influence the choice for unrealistic aspirations but it is insignificant under realistic ones. To promote open science practice, we make our code and dataset publicly available at: https://github.com/ideas-labo/aspiration-study .

The multi-objective testing resource allocation problem (MOTRAP) aims at seeking a good trade-off between system reliability, testing cost, and testing time, which is of significant importance to facilitate the testing planning. Yet most studies focus on the time constraint but rarely consider the practical reliability requirement. In this work, we address MOTRAP on an architecture-based model (ABM) with the personalized preference over reliability. More specifically, we first present a reliability-constrained MOTRAP model on the basis of ABM and illustrate how to use this model for real-world systems. Then, to leverage the problem's knowledge, we develop new lower and upper bounds on testing time invested in different components from both theoretical and algorithmic perspectives on the basis of the Lagrange multiplier and half-interval search. Importantly, these new derived bounds have strong implications due to the fact that they can be easily employed by optimizers as the limits of variables to prune the search space to the region of interests of the decision maker and locate feasible solutions with the expected reliability. Finally, we evaluate the proposed bounds in popular multi-objective optimizers for MOTRAP on application and empirical cases. Experimental results demonstrate that our new bounds practically improve the search performance of optimizers, and decision makers can easily combine these new bounds with off-the-shelf optimizers to find higher-quality solutions that they are interested in, which greatly soothes away stress on optimizer and solution selections of decision makers.

Real-world software product lines (SPLs) often encompass enormous valid configurations that are impossible to enumerate. To understand properties of the space formed by all valid configurations, a feasible way is to select a small and valid sample set. Even though a number of sampling strategies have been proposed, they either fail to produce diverse samples with respect to the number of selected features (an important property to characterize behaviors of configurations), or achieve diverse sampling but with limited scalability (the handleable configuration space size is limited to 1013). To resolve this dilemma, we propose a scalable diverse sampling strategy, which uses a distance metric in combination with the novelty search algorithm to produce diverse samples in an incremental way. The distance metric is carefully designed to measure similarities between configurations, and further diversity of a sample set. The novelty search incrementally improves diversity of samples through the search for novel configurations. We evaluate our sampling algorithm on 39 real-world SPLs. It is able to generate the required number of samples for all the SPLs, including those which cannot be counted by sharpSAT, a state-of-the-art model counting solver. Moreover, it performs better than or at least competitively to state-of-the-art samplers regarding diversity of the sample set. Experimental results suggest that only the proposed sampler (among all the tested ones) achieves scalable diverse sampling.

Automated algorithm configuration relieves users from tedious, trial-and-error tuning tasks. A popular algorithm configuration tuning paradigm is dynamic algorithm configuration (DAC), in which an agent learns dynamic configuration policies across instances by reinforcement learning (RL). However, in many complex algorithms, there may exist different types of configuration hyperparameters, and such heterogeneity may bring difficulties for classic DAC which uses a single-agent RL policy. In this paper, we aim to address this issue and propose multi-agent DAC (MA-DAC), with one agent working for one type of configuration hyperparameter. MA-DAC formulates the dynamic configuration of a complex algorithm with multiple types of hyperparameters as a contextual multi-agent Markov decision process and solves it by a cooperative multi-agent RL (MARL) algorithm. To instantiate, we apply MA-DAC to a well-known optimization algorithm for multi-objective optimization problems. Experimental results show the effectiveness of MA-DAC in not only achieving superior performance compared with other configuration tuning approaches based on heuristic rules, multi-armed bandits, and single-agent RL, but also being capable of generalizing to different problem classes. Furthermore, we release the environments in this paper as a benchmark for testing MARL algorithms, with the hope of facilitating the application of MARL.

Due to the ever-increasing amount of various information provided by the internet, recommendation systems are now used in a large number of fields as efficient tools to get rid of information overload. The content-based, collaborative-based and hybrid methods are the three classical recommendation techniques, whereas not all real-world problems (e.g. the food recommendation problem) can be best addressed by such classical recommendation techniques. This paper is devoted to solving the food recommendation problem based on many-objective optimization (MaOO). A novel recommendation approach is proposed by transforming the original recommendation problem into an MaOO one that contains four different objectives, i.e., the user preferences, nutritional values, dietary diversity, and user diet patterns. The experimental results demonstrate that the designed recommendation approach provides a more balanced way of recommending food than the classical recommendation methods that only consider individuals’ food preferences.

How to evaluate Pareto front approximations generated by multi/many-objective optimizers is a critical issue in the field of multiobjective optimization. Currently, there exist two types of comprehensive quality indicators (i.e., volume-based and distance-based indicators). Distance-based indicators, such as inverted generational distance (IGD), are usually computed by summing up the distance of each reference point to its nearest solution. Their high computational efficiency leads to their prevalence in many-objective optimization. However, in the existing distance-based indicators, the distributions of the solution sets are usually neglected, leading to their lack of ability to well distinguish between different solution sets. This phenomenon may become even more severe in high-dimensional space. To address such an issue, a kernel-based indicator (KBI) is proposed as a comprehensive indicator. Different from other distance-based indicators, a kernel-based maximum mean discrepancy is adopted in KBI for directly measuring the difference that can characterize the convergence, spread, and uniformity of two sets, i.e., the solution set and reference set, by embedding them in reproducing kernel Hilbert space (RKHS). As a result, KBI not only reflects the distance between the solution set and the reference set but also can reflect the distribution of the solution set itself. In addition, to maintain the desirable weak Pareto compliance property of KBI, a nondominated set reconstruction approach is also proposed to shift the original solution set. The detailed theoretical and experimental analysis of KBI is provided in this article. The properties of KBI have also been analyzed by the optimal$\mu $-distribution.

With modern requirements, there is an increasing tendency of considering multiple objectives/criteria simultaneously in many Software Engineering (SE) scenarios. Such a multi-objective optimization scenario comes with an important issue — how to evaluate the outcome of optimization algorithms, which typically is a set of incomparable solutions (i.e., being Pareto nondominated to each other). This issue can be challenging for the SE community, particularly for practitioners of Search-Based SE (SBSE). On one hand, multi-objective optimization could still be relatively new to SE/SBSE researchers, who may not be able to identify the right evaluation methods for their problems. On the other hand, simply following the evaluation methods for general multi-objective optimization problems may not be appropriate for specific SBSE problems, especially when the problem nature or decision maker’s preferences are explicitly/implicitly known. This has been well echoed in the literature by various inappropriate/inadequate selection and inaccurate/misleading use of evaluation methods. In this paper, we first carry out a systematic and critical review of quality evaluation for multi-objective optimization in SBSE. We survey 717 papers published between 2009 and 2019 from 36 venues in seven repositories, and select 95 prominent studies, through which we identify five important but overlooked issues in the area. We then conduct an in-depth analysis of quality evaluation indicators/methods and general situations in SBSE, which, together with the identified issues, enables us to codify a methodological guidance for selecting and using evaluation methods in different SBSE scenarios.

Testing software product lines (SPLs) is difficult due to a huge number of possible products to be tested. Recently, there has been a growing interest in similarity-based testing of SPLs, where similarity is used as a surrogate metric for the$t$-wise coverage. In this context, one of the primary goals is to sample, by optimizing similarity metrics using search-based algorithms, a small subset of test cases (i.e., products) as dissimilar as possible, thus potentially making more$t$-wise combinations covered. Prior work has shown, by means of empirical studies, the great potential of current similarity-based testing approaches. However, the rationale of this testing technique deserves a more rigorous exploration. To this end, we perform correlation analyses to investigate how similarity metrics are correlated with the$t$-wise coverage. We find that similarity metrics generally have significantly positive correlations with the$t$-wise coverage. This well explains why similarity-based testing works, as the improvement on similarity metrics will potentially increase the$t$-wise coverage. Moreover, we explore, for the first time, the use of the novelty search (NS) algorithm for similarity-based SPL testing. The algorithm rewards “novel” individuals, i.e., those being different from individuals discovered previously, and this well matches the goal of similarity-based SPL testing. We find that the novelty score used in NS has (much) stronger positive correlations with the$t$-wise coverage than previous approaches relying on a genetic algorithm (GA) with a similarity-based fitness function. Experimental results on 31 software product lines validate the superiority of NS over GA, as well as other state-of-the-art approaches, concerning both$t$-wise coverage and fault detection capacity. Finally, we investigate whether it is useful to combine two satisfiability solvers when generating new individuals in NS, and how the performance of NS is affected by its key parameters. In summary, looking for novelty provides a promising way of sampling diverse test cases for SPLs.

Automatically tuning software configuration for optimizing a single performance attribute (e.g., minimizing latency) is not trivial, due to the nature of the configuration systems (e.g., complex landscape and expensive measurement). To deal with the problem, existing work has been focusing on developing various effective optimizers. However, a prominent issue that all these optimizers need to take care of is how to avoid the search being trapped in local optima — a hard nut to crack for software configuration tuning due to its rugged and sparse landscape, and neighboring configurations tending to behave very differently. Overcoming such in an expensive measurement setting is even more challenging. In this paper, we take a different perspective to tackle this issue. Instead of focusing on improving the optimizer, we work on the level of optimization model. We do this by proposing a meta multi-objectivization model (MMO) that considers an auxiliary performance objective (e.g., throughput in addition to latency). What makes this model unique is that we do not optimize the auxiliary performance objective, but rather use it to make similarly-performing while different configurations less comparable (i.e. Pareto nondominated to each other), thus preventing the search from being trapped in local optima.

It is known that the Pareto-based approach is not well suited for optimization problems with a large number of objectives, even though it is a class of mainstream methods in multiobjective optimization. Typically, a Pareto-based algorithm comprises two parts: 1) a Pareto dominance-based criterion and 2) a diversity estimator. The former guides the selection toward the optimal front, while the latter promotes the diversity of the population. However, the Pareto dominance-based criterion becomes ineffective in solving optimization problems with many objectives (e.g., more than 3) and, thus, the diversity estimator will determine the performance of the algorithm. Unfortunately, the diversity estimator usually has a strong bias toward dominance resistance solutions (DRSs), thereby failing to push the population forward. DRSs are solutions that are far away from the Pareto-optimal front but cannot be easily dominated. In this article, we propose a new Pareto-based algorithm to resolve the above issue. First, to eliminate the DRSs, we design an interquartile range method to preprocess the solution set. Second, to balance convergence and diversity, we present a penalty mechanism of alternating operations between selection and penalty. The proposed algorithm is compared with five state-of-the-art algorithms on a number of well-known benchmarks with 3-15 objectives. The experimental results show that the proposed algorithm can perform well on most of the test functions and generally outperforms its competitors.

Assessing the performance of Pareto front (PF) approximations is a key issue in the field of evolutionary multi/many-objective optimization. Inverted generational distance (IGD) has been widely accepted as a performance indicator for evaluating the comprehensive quality for a PF approximation. However, IGD usually becomes infeasible when facing a real-world optimization problem as it needs to know the true PF a priori. In addition, the time complexity of IGD grows quadratically with the size of the solution/reference set. To address the aforementioned issues, a grid-based IGD (Grid-IGD) is proposed to estimate both convergence and diversity of PF approximations for multi/many-objective optimization. In Grid-IGD, a set of reference points is generated by estimating PFs of the problem in question, based on the representative nondominated solutions of all the approximations in a grid environment. To reduce the time complexity, Grid-IGD only considers the closest solution within the grid neighborhood in the approximation for every reference point. Grid-IGD also possesses other desirable properties, such as Pareto compliance, immunity to dominated/duplicate solutions, and no need of normalization. In the experimental studies, Grid-IGD is verified on both the artificial and real PF approximations obtained by five many-objective optimizers. Effects of the grid specification on the behavior of Grid-IGD are also discussed in detail theoretically and experimentally.

The multiobjective testing resource allocation problem (MOTRAP) is how to efficiently allocate the finite testing time to various modules, with the aim of optimizing system reliability, testing cost, and testing time simultaneously. To deal with this problem, a common approach is to use multiobjective evolutionary algorithms (MOEAs) to seek a set of tradeoff solutions between the three objectives. However, such a tradeoff set may contain a substantial proportion of solutions with very low reliability level, which consume lots of computational resources but may be valueless to the software project manager. In this article, a MOTRAP model with a prespecified reliability is first proposed. Then, new lower bounds on the testing time invested in different modules are theoretically deduced from the necessary condition for the achievement of the given reliability, based on which an exact algorithm for determining the new lower bounds is presented. Moreover, several enhanced constraint-handling techniques (ECHTs) derived from the new bounds are successively developed to be combined with MOEAs to correct and reduce the constraint violation. Finally, the proposed ECHTs are evaluated in comparison with various state-of-the-art constraint-solving approaches. The comparative results demonstrate that the proposed ECHTs can work well with MOEAs, make the search focus on the feasible region of the prespecified reliability, and provide the software project manager with better and more diverse, satisfactory choices in test planning.

In evolutionary multiobjective optimisation ( EMO ), archiving is a common component that maintains an (external or internal) set during the search process, typically with a fixed size, in order to provide a good representation of high-quality solutions produced. Such an archive set can be used solely to store the final results shown to the decision maker, but in many cases may participate in the process of producing solutions (e.g., as a solution pool where the parental solutions are selected). Over the last three decades, archiving stands as an important issue in EMO, leading to the emergence of various methods such as those based on Pareto, indicator, or decomposition criteria. Such methods have demonstrated their effectiveness in literature and have been believed to be good options to many problems, particularly those having a regular Pareto front shape, e.g., a simplex shape. In this article, we challenge this belief. We do this through artificially constructing several sequences with extremely simple shapes, i.e., 1D/2D simplex Pareto front. We show the struggle of predominantly used archiving methods which have been deemed to well handle such shapes. This reveals that the order of solutions entering the archive matters, and that current EMO algorithms may not be fully capable of maintaining a representative population on problems with linear Pareto fronts even in the case that all of their optimal solutions can be found.

Some real-world optimization problems involve multiple decision makers holding different positions, each of whom has multiple conflicting objectives. These problems are defined as multiparty multiobjective optimization problems (MPMOPs). Although evolutionary multiobjective optimization has been widely studied for many years, little attention has been paid to multiparty multiobjective optimization in the field of evolutionary computation. In this paper, a class of MPMOPs, that is, MPMOPs having common Pareto optimal solutions, is addressed. A benchmark for MPMOPs, obtained by modifying an existing dynamic multiobjective optimization benchmark, is provided, and a multiparty multiobjective evolutionary algorithm to find the common Pareto optimal set is proposed. The results of experiments conducted using the benchmark show that the proposed multiparty multiobjective evolutionary algorithm is effective.

Many-objective optimization, which deals with an optimization problem with more than three objectives, poses a big challenge to various search techniques, including evolutionary algorithms. Recently, a meta-objective optimization approach (called bi-goal evolution, BiGE) which maps solutions from the original high-dimensional objective space into a bi-goal space of proximity and crowding degree has received increasing attention in the area. However, it has been found that BiGE tends to struggle on a class of many-objective problems where the search process involves dominance resistant solutions , namely, those solutions with an extremely poor value in at least one of the objectives but with (near) optimal values in some of the others. It is difficult for BiGE to get rid of dominance resistant solutions as they are Pareto nondominated and far away from the main population, thus always having a good crowding degree. In this paper, we propose an angle-based crowding degree estimation method for BiGE (denoted as aBiGE) to replace distance-based crowding degree estimation in BiGE. Experimental studies show the effectiveness of this replacement.

The quality of solution sets generated by decomposition-based evolutionary multi-objective optimisation (EMO) algorithms depends heavily on the consistency between a given problem's Pareto front shape and the specified weights' distribution. A set of weights distributed uniformly in a simplex often leads to a set of well-distributed solutions on a Pareto front with a simplex-like shape, but may fail on other Pareto front shapes. It is an open problem on how to specify a set of appropriate weights without the information of the problem's Pareto front beforehand. In this article, we propose an approach to adapt weights during the evolutionary process (called AdaW). AdaW progressively seeks a suitable distribution of weights for the given problem by elaborating several key parts in weight adaptation-weight generation, weight addition, weight deletion, and weight update frequency. Experimental results have shown the effectiveness of the proposed approach. AdaW works well for Pareto fronts with very different shapes: 1) the simplex-like, 2) the inverted simplex-like, 3) the highly nonlinear, 4) the disconnect, 5) the degenerate, 6) the scaled, and 7) the high-dimensional.

Coalition formation has been a fundamental form of resource cooperation for achieving joint goals in multiagent systems. Most existing studies still focus on the traditional assumption that an agent has to contribute its resources to all the goals, even if the agent is not interested in the goal at all. In this article, a natural extension of the traditional coalitional resource games (CRGs) is studied from both theoretical and empirical perspectives, in which each agent has uncompromising, personalized preferences over goals. Specifically, a new CRGs model with agents’ strict preferences for goals is presented, in which an agent is willing to contribute its resources only to the goals that are in its own interest set. The computational complexity of the basic decision problems surrounding the successful coalition is reinvestigated. The results suggest that these problems in such a strict preference way are complex and intractable. To find the largest successful coalition for possible computation reduction or potential parallel processing, a flow-network–based exhaust algorithm, called FNetEA, is proposed to achieve the optimal solution. Then, to solve the problem more efficiently, a hybrid algorithm, named 2D-HA, is developed to find the approximately optimal solution on the basis of genetic algorithm, two-dimensional (2D) solution representation, and a heuristic for solution repairs. Through extensive experiments, the 2D-HA algorithm exhibits the prominent ability to provide reassurances that the optimal solution could be found within a reasonable period of time, even in a super-large-scale space.

A Software Product Line (SPL) is a set of products built from a number of features, the set of valid products being defined by a feature model. Typically, it does not make sense to test all products defined by an SPL and one instead chooses a set of products to test (test selection) and, ideally, derives a good order in which to test them (test prioritisation). Since one cannot know in advance which products will reveal faults, test selection and prioritisation are normally based on objective functions that are known to relate to likely effectiveness or cost. This article introduces a new technique, the grid-based evolution strategy (GrES), which considers several objective functions that assess a selection or prioritisation and aims to optimise on all of these. The problem is thus a many-objective optimisation problem. We use a new approach, in which all of the objective functions are considered but one (pairwise coverage) is seen as the most important. We also derive a novel evolution strategy based on domain knowledge. The results of the evaluation, on randomly generated and realistic feature models, were promising, with GrES outperforming previously proposed techniques and a range of many-objective optimisation algorithms.

Overlapping coalition formation (OCF), which provides a natural framework for modeling scenarios where each agent can join and allocate their resources to several completely different coalitions at the same time, has become a very active topic in multiagent systems. For OCF in resource-constrained and subadditive task oriented domains, an agent may not possess sufficient resources to meet the needs of multiple coalitions simultaneously. As a result, there may exist many potential resource conflicts among the rival overlapping coalitions. To tackle such situations, we first present a natural variation of the traditional OCF model and analyze the size of the solution space and the computational complexity of the overlapping coalition structure generation (OCSG) problem. Next, we develop a generic task-oriented heuristic (TOH) for individual repairs that can be used in binary meta-heuristic algorithms to generate overlapping coalitions in a parallel manner. Moreover, we show how the proposed TOH repairs a 2-D individual to resolve resource conflicts and discuss several basic properties. Finally, to evaluate the effectiveness of TOH, we compare it with the existing agent-oriented heuristic for the OCSG problem. The empirical results demonstrate that TOH is of high efficiency and effectiveness in harsh environments with fierce competition over scarce resources.

Service composition aims to search a composition plan of candidate services that produces the optimal results with respect to multiple and possibly conflicting Quality-of-Service (QoS) attributes, e.g., latency, throughput and cost. This leads to a multi-objective optimization problem for which evolutionary algorithm is a promising solution. In this paper, we investigate different ways of injecting knowledge about the problem into the Multi-Objective Evolutionary Algorithm (MOEA) by seeding. Specifically, we propose four alternative seeding strategies to strengthen the quality of the initial population for the MOEA to start working with. By using the real-world WS-DREAM dataset, we conduced experimental evaluations based on 9 different workflows of service composition problems and several metrics. The results confirm the effectiveness and efficiency of those seeding strategies. We also observed that, unlike the discoveries for other problem domains, the implication of the number of seeds on the service composition problems is minimal, for which we investigated and discussed the possible reasons.

Recently, by taking advantage of evolutionary multiobjective optimization techniques in diversity preservation, the means of multiobjectivization has attracted increasing interest in the studies of multimodal optimization (MMO). While most existing work of multiobjectivization aims to find all optimal solutions simultaneously, in this paper, we propose to approximate multimodal fitness landscapes via multiobjectivization, thus providing an estimation of potential optimal areas. To begin with, an MMO problem is transformed into a multiobjective optimization problem (MOP) by adding an adaptive diversity indicator as the second optimization objective, and an approximate fitness landscape is obtained via optimization of the transformed MOP using a multiobjective evolutionary algorithm. Then, on the basis of the approximate fitness landscape, an adaptive peak detection method is proposed to find peaks where optimal solutions may exist. Finally, local search is performed inside the detected peaks on the approximate fitness landscape. To assess the performance of the proposed algorithm, extensive experiments are conducted on 20 multimodal test functions, in comparison with three state-of-the-art algorithms for MMO. Experimental results demonstrate that the proposed algorithm not only shows promising performance in benchmark comparisons, but also has good potential in assisting preference-based decision-making in MMO.

Studying the search behavior of evolutionary many-objective optimization is an important, but challenging issue. Existing studies rely mainly on the use of performance indicators which, however, not only encounter increasing difficulties with the number of objectives, but also fail to provide the visual information of the evolutionary search. In this paper, we propose a class of scalable test problems, called multiline distance minimization problem (ML-DMP), which are used to visually examine the behavior of many-objective search. Two key characteristics of the ML-DMP problem are: 1) its Pareto optimal solutions lie in a regular polygon in the 2-D decision space and 2) these solutions are similar (in the sense of Euclidean geometry) to their images in the high-dimensional objective space. This allows a straightforward understanding of the distribution of the objective vector set (e.g., its uniformity and coverage over the Pareto front) via observing the solution set in the 2-D decision space. Fifteen well-established algorithms have been investigated on three types of ten ML-DMP problem instances. Weakness has been revealed across classic multiobjective algorithms (such as Pareto-based, decomposition-based, and indicator-based algorithms) and even state-of-the-art algorithms designed especially for many-objective optimization. This, together with some interesting observations from the experimental studies, suggests that the proposed ML-DMP may also be used as a benchmark function to challenge the search ability of optimization algorithms.

A feature model (FM) is a compact representation of the information of all possible products from software product lines. The optimal feature selection involves the simultaneous optimization of multiple (usually more than three) objectives in a large and highly constrained search space. By combining our previous work on many-objective evolutionary algorithm (i.e., VaEA) with two different satisfiability (SAT) solvers, this article proposes a new approach named SATVaEA for handling the optimal feature selection problem. In SATVaEA, an FM is simplified with the number of both features and constraints being reduced greatly. We enhance the search of VaEA by using two SAT solvers: one is a stochastic local search--based SAT solver that can quickly repair infeasible configurations, whereas the other is a conflict-driven clause-learning SAT solver that is introduced to generate diversified products. We evaluate SATVaEA on 21 FMs with up to 62,482 features, including two models with realistic values for feature attributes. The experimental results are promising, with SATVaEA returning 100% valid products on almost all FMs. For models with more than 10,000 features, the search in SATVaEA takes only a few minutes. Concerning both effectiveness and efficiency, SATVaEA significantly outperforms other state-of-the-art algorithms.

Multimodal optimization has attracted increasing interest recently. Despite the emergence of various multimodal optimization algorithms during the last decade, little work has been dedicated to the development of benchmark tools. In this paper, we propose a visualization method for benchmark studies of multimodal optimization, called parallel peaks. Inspired by parallel coordinates, the proposed parallel peaks method is capable of visualizing both distribution information and convergence information of a given candidate solution set inside a 2D coordinate plane. To the best of our knowledge, this is the first visualization method in the multimodal optimization area. Our empirical results demonstrate that the proposed parallel peaks method can be robustly used to visualize candidate solutions sets with a range of properties, including high-accuracy solutions sets, high-dimensional solution sets and solution sets with a large number of optima. Additionally, by visualizing the populations obtained during the optimization process, it can also be used to investigate search behaviors of multimodal optimization algorithms.

Taking both convergence and diversity into consideration, this paper suggests a vector angle-based evolutionary algorithm for unconstrained (with box constraints only) many-objective optimization problems. In the proposed algorithm, the maximum-vector-angle-first principle is used in the environmental selection to guarantee the wideness and uniformity of the solution set. With the help of the worse-elimination principle, worse solutions in terms of the convergence (measured by the sum of normalized objectives) are allowed to be conditionally replaced by other individuals. Therefore, the selection pressure toward the Pareto-optimal front is strengthened. The proposed method is compared with other four state-of-the-art many-objective evolutionary algorithms on a number of unconstrained test problems with up to 15 objectives. The experimental results have shown the competitiveness and effectiveness of our proposed algorithm in keeping a good balance between convergence and diversity. Furthermore, it was shown by the results on two problems from practice (with irregular Pareto fronts) that our method significantly outperforms its competitors in terms of both the convergence and diversity of the obtained solution sets. Notably, the new algorithm has the following good properties: 1) it is free from a set of supplied reference points or weight vectors; 2) it has less algorithmic parameters; and 3) the time complexity of the algorithm is low. Given both good performance and nice properties, the suggested algorithm could be an alternative tool when handling optimization problems with more than three objectives.

In software testing, optimal testing resource allocation problems (OTRAPs) are important when seeking a good tradeoff between reliability, cost, and time with limited resources. There have been intensive studies of OTRAPs using multiobjective evolutionary algorithms (MOEAs), but little attention has been paid to the constraint handling. This paper comprehensively investigates the effect of the constraint handling on the performance of nondominated sorting genetic algorithm II (NSGA-II) for solving OTRAPs, from both theoretical and empirical perspectives. The heuristics for individual repairs are first proposed to handle constraint violations in NSGA-II, based on which several properties are derived. Additionally, theZ-score based Euclidean distance is adopted to estimate the difference between solutions. Finally, the above methods are evaluated and the experiments show several results. 1) The developed heuristics for constraint handling are better than the Existing Strategy in terms of the capacity and coverage values. 2) TheZ-score operation obtains better diversity values and reduces repeated solutions. 3) The modified NSGA-II for OTRAPs (called NSGA-II-TRA) performs significantly better than the existing MOEAs in terms of capacity and coverage values, which suggests that NSGA-II-TRA could obtain more and higher quality testing-time-allocation schemes, especially for large, complex datasets. 4) NSGA-II-TRA is robust according to the sensitivity analysis results.

It is known that Pareto dominance has its own weaknesses as the selection criterion in evolutionary multiobjective optimization. Algorithms based on Pareto criterion (PC) can suffer from problems such as slow convergence to the optimal front and inferior performance on problems with many objectives. Non-Pareto criterion (NPC), such as decomposition-based criterion and indicator-based criterion, has already shown promising results in this regard, but its high selection pressure may lead to the algorithm to prefer some specific areas of the problem's Pareto front, especially when the front is highly irregular. In this paper, we propose a bi-criterion evolution (BCE) framework of the PC and NPC, which attempts to make use of their strengths and compensates for each other's weaknesses. The proposed framework consists of two parts: PC evolution and NPC evolution. The two parts work collaboratively, with an abundant exchange of information to facilitate each other's evolution. Specifically, the NPC evolution leads the PC evolution forward and the PC evolution compensates the possible diversity loss of the NPC evolution. The proposed framework keeps the freedom on the implementation of the NPC evolution part, thus making it applicable for any non-Pareto-based algorithm. In the PC evolution, two operations, population maintenance and individual exploration, are presented. The former is to maintain a set of representative nondominated individuals and the latter is to explore some promising areas that are undeveloped (or not well-developed) in the NPC evolution. Experimental results have shown the effectiveness of the proposed framework. The BCE works well on seven groups of 42 test problems with various characteristics, including those in which Pareto-based algorithms or non-Pareto-based algorithms struggle.

A feature model specifies the sets of features that define valid products in a software product line. Recent work has considered the problem of choosing optimal products from a feature model based on a set of user preferences, with this being represented as a many-objective optimization problem. This problem has been found to be difficult for a purely search-based approach, leading to classical many-objective optimization algorithms being enhanced either by adding in a valid product as a seed or by introducing additional mutation and replacement operators that use an SAT solver. In this article, we instead enhance the search in two ways: by providing a novel representation and by optimizing first on the number of constraints that hold and only then on the other objectives. In the evaluation, we also used feature models with realistic attributes, in contrast to previous work that used randomly generated attribute values. The results of experiments were promising, with the proposed (SIP) method returning valid products with six published feature models and a randomly generated feature model with 10,000 features. For the model with 10,000 features, the search took only a few minutes.

Cloud computing provides promising platforms for executing large applications with enormous computational resources to offer on demand. In a Cloud model, users are charged based on their usage of resources and the required quality of service (QoS) specifications. Although there are many existing workflow scheduling algorithms in traditional distributed or heterogeneous computing environments, they have difficulties in being directly applied to the Cloud environments since Cloud differs from traditional heterogeneous environments by its service-based resource managing method and pay-per-use pricing strategies. In this paper, we highlight such difficulties, and model the workflow scheduling problem which optimizes both makespan and cost as a Multi-objective Optimization Problem (MOP) for the Cloud environments. We propose an evolutionary multi-objective optimization (EMO)-based algorithm to solve this workflow scheduling problem on an infrastructure as a service (IaaS) platform. Novel schemes for problem-specific encoding and population initialization, fitness evaluation and genetic operators are proposed in this algorithm. Extensive experiments on real world workflows and randomly generated workflows show that the schedules produced by our evolutionary algorithm present more stability on most of the workflows with the instance-based IaaS computing and pricing models. The results also show that our algorithm can achieve significantly better solutions than existing state-of-the-art QoS optimization scheduling algorithms in most cases. The conducted experiments are based on the on-demand instance types of Amazon EC2; however, the proposed algorithm are easy to be extended to the resources and pricing models of other IaaS services.

This paper introduces an optimization template library (OTL), a cross-platform C++ template library for multiobjective optimization. OTL has an object-oriented architecture, which allows that different modules can be arbitrarily combined with each other. Moreover, the C++ template technique is used to increase the flexibility of OTL. Meanwhile, generic programming is widely used in OTL, and the generic algorithms can be used to process different data structures. However, compared with C++, the Python script is more suitable for building the experimental platform. To ensure that all attributes of the experimental results can be fully maintained, a database is used to store the experimental data. Moreover, batch experiments can be easily defined in a set of configuration files; thus, the experiments can be executed automatically without human intervention. In addition, serial and various parallel execution modes are supported, and the user can easily switch the running mode to distributed computing to increase the computing speed. Finally, a highly customizable data visualization tool is created to play back the data sample stored in the database. From a series of comparative studies, the accuracy and running performance of OTL are verified by the statistical results.

Increasing interest in simultaneously optimizing many objectives (typically more than three objectives) of problems leads to the emergence of various many-objective algorithms in the evolutionary multi-objective optimization field. However, in contrast to the development of algorithm design, how to assess many-objective algorithms has received scant concern. Many performance indicators are designed in principle for any number of objectives, but in practice are invalid or infeasible to be used in many-objective optimization. In this paper, we explain the difficulties that popular performance indicators face and propose a performance comparison indicator (PCI) to assess Pareto front approximations obtained by many-objective algorithms. PCI evaluates the quality of approximation sets with the aid of a reference set constructed by themselves. The points in the reference set are divided into many clusters, and the proposed indicator estimates the minimum moves of solutions in the approximation sets to weakly dominate these clusters. PCI has been verified both by an analytic comparison with several well-known indicators and by an empirical test on four groups of Pareto front approximations with different numbers of objectives and problem characteristics.

This paper presents a meta-objective optimization approach, called Bi-Goal Evolution (BiGE), to deal with multi-objective optimization problems with many objectives. In multi-objective optimization, it is generally observed that 1) the conflict between the proximity and diversity requirements is aggravated with the increase of the number of objectives and 2) the Pareto dominance loses its effectiveness for a high-dimensional space but works well on a low-dimensional space. Inspired by these two observations, BiGE converts a given multi-objective optimization problem into a bi-goal (objective) optimization problem regarding proximity and diversity, and then handles it using the Pareto dominance relation in this bi-goal domain. Implemented with estimation methods of individuals' performance and the classic Pareto nondominated sorting procedure, BiGE divides individuals into different nondominated layers and attempts to put well-converged and well-distributed individuals into the first few layers. From a series of extensive experiments on four groups of well-defined continuous and combinatorial optimization problems with 5, 10 and 15 objectives, BiGE has been found to be very competitive against five state-of-the-art algorithms in balancing proximity and diversity. The proposed approach is the first step towards a new way of addressing many-objective problems as well as indicating several important issues for future development of this type of algorithms.

An inherent problem in multiobjective optimization is that the visual observation of solution vectors with four or more objectives is infeasible, which brings major difficulties for algorithmic design, examination, and development. This paper presents a test problem, called the Rectangle problem, to aid the visual investigation of high-dimensional multiobjective search. Key features of the Rectangle problem are that the Pareto optimal solutions 1) lie in a rectangle in the two-variable decision space and 2) are similar (in the sense of Euclidean geometry) to their images in the four-dimensional objective space. In this case, it is easy to examine the behavior of objective vectors in terms of both convergence and diversity, by observing their proximity to the optimal rectangle and their distribution in the rectangle, respectively, in the decision space. Fifteen algorithms are investigated. Underperformance of Pareto-based algorithms as well as most state-of-the-art many-objective algorithms indicates that the proposed problem not only is a good tool to help visually understand the behavior of multiobjective search in a high-dimensional objective space but also can be used as a challenging benchmark function to test algorithms' ability in balancing the convergence and diversity of solutions.

The Euclidean minimum spanning tree (EMST), widely used in a variety of domains, is a minimum spanning tree of a set of points in space where the edge weight between each pair of points is their Euclidean distance. Since the generation of an EMST is entirely determined by the Euclidean distance between solutions (points), the properties of EMSTs have a close relation with the distribution and position information of solutions. This paper explores the properties of EMSTs and proposes an EMST-based evolutionary algorithm (ETEA) to solve multi-objective optimization problems (MOPs). Unlike most EMO algorithms that focus on the Pareto dominance relation, the proposed algorithm mainly considers distance-based measures to evaluate and compare individuals during the evolutionary search. Specifically, in ETEA, four strategies are introduced: (1) An EMST-based crowding distance (ETCD) is presented to estimate the density of individuals in the population; (2) A distance comparison approach incorporating ETCD is used to assign the fitness value for individuals; (3) A fitness adjustment technique is designed to avoid the partial overcrowding in environmental selection; (4) Three diversity indicators-the minimum edge, degree, and ETCD-with regard to EMSTs are applied to determine the survival of individuals in archive truncation. From a series of extensive experiments on 32 test instances with different characteristics, ETEA is found to be competitive against five state-of-the-art algorithms and its predecessor in providing a good balance among convergence, uniformity, and spread.

Diversity assessment of Pareto front approximations is an important issue in the stochastic multiobjective optimization community. Most of the diversity indicators in the literature were designed to work for any number of objectives of Pareto front approximations in principle, but in practice many of these indicators are infeasible or not workable when the number of objectives is large. In this paper, we propose a diversity comparison indicator (DCI) to assess the diversity of Pareto front approximations in many-objective optimization. DCI evaluates relative quality of different Pareto front approximations rather than provides an absolute measure of distribution for a single approximation. In DCI, all the concerned approximations are put into a grid environment so that there are some hyperboxes containing one or more solutions. The proposed indicator only considers the contribution of different approximations to nonempty hyperboxes. Therefore, the computational cost does not increase exponentially with the number of objectives. In fact, the implementation of DCI is of quadratic time complexity, which is fully independent of the number of divisions used in grid. Systematic experiments are conducted using three groups of artificial Pareto front approximations and seven groups of real Pareto front approximations with different numbers of objectives to verify the effectiveness of DCI. Moreover, a comparison with two diversity indicators used widely in many-objective optimization is made analytically and empirically. Finally, a parametric investigation reveals interesting insights of the division number in grid and also offers some suggested settings to the users with different preferences.

This paper proposes a variation operator, called segment-based search (SBS), to improve the performance of evolutionary algorithms on continuous multiobjective optimization problems. SBS divides the search space into many small segments according to the evolutionary information feedback from the set of current optimal solutions. Two operations, micro-jumping and macro-jumping, are implemented upon these segments in order to guide an efficient information exchange among "good" individuals. Moreover, the running of SBS is adaptive according to the current evolutionary status. SBS is activated only when the population evolves slowly, depending on general genetic operators (e.g., mutation and crossover). A comprehensive set of 36 test problems is employed for experimental verification. The influence of two algorithm settings (i.e., the dimensionality and boundary relaxation strategy) and two probability parameters in SBS (i.e., the SBS rate and micro-jumping proportion) are investigated in detail. Moreover, an empirical comparative study with three representative variation operators is carried out. Experimental results show that the incorporation of SBS into the optimization process can improve the performance of evolutionary algorithms for multiobjective optimization problems.

It is commonly accepted that Pareto-based evolutionary multiobjective optimization (EMO) algorithms encounter difficulties in dealing with many-objective problems. In these algorithms, the ineffectiveness of the Pareto dominance relation for a high-dimensional space leads diversity maintenance mechanisms to play the leading role during the evolutionary process, while the preference of diversity maintenance mechanisms for individuals in sparse regions results in the final solutions distributed widely over the objective space but distant from the desired Pareto front. Intuitively, there are two ways to address this problem: 1) modifying the Pareto dominance relation and 2) modifying the diversity maintenance mechanism in the algorithm. In this paper, we focus on the latter and propose a shift-based density estimation (SDE) strategy. The aim of our study is to develop a general modification of density estimation in order to make Pareto-based algorithms suitable for many-objective optimization. In contrast to traditional density estimation that only involves the distribution of individuals in the population, SDE covers both the distribution and convergence information of individuals. The application of SDE in three popular Pareto-based algorithms demonstrates its usefulness in handling many-objective problems. Moreover, an extensive comparison with five state-of-the-art EMO algorithms reveals its competitiveness in balancing convergence and diversity of solutions. These findings not only show that SDE is a good alternative to tackle many-objective problems, but also present a general extension of Pareto-based algorithms in many-objective optimization.

Multiobjective evolutionary algorithm based on decomposition (MOEA/D) decomposes a multiobjective optimization problem into a set of scalar optimization subproblems and optimizes them in a collaborative manner. Subproblems and solutions are two sets of agents that naturally exist in MOEA/D. The selection of promising solutions for subproblems can be regarded as a matching between subproblems and solutions. Stable matching, proposed in economics, can effectively resolve conflicts of interests among selfish agents in the market. In this paper, we advocate the use of a simple and effective stable matching (STM) model to coordinate the selection process in MOEA/D. In this model, subproblem agents can express their preferences over the solution agents, and vice versa. The stable outcome produced by the STM model matches each subproblem with one single solution, and it tradeoffs convergence and diversity of the evolutionary search. Comprehensive experiments have shown the effectiveness and competitiveness of our MOEA/D algorithm with the STM model. We have also demonstrated that user-preference information can be readily used in our proposed algorithm to find a region that decision makers are interested in.

Balancing convergence and diversity plays a key role in evolutionary multiobjective optimization (EMO). Most current EMO algorithms perform well on problems with two or three objectives, but encounter difficulties in their scalability to many-objective optimization. This paper proposes a grid-based evolutionary algorithm (GrEA) to solve many-objective optimization problems. Our aim is to exploit the potential of the grid-based approach to strengthen the selection pressure toward the optimal direction while maintaining an extensive and uniform distribution among solutions. To this end, two concepts-grid dominance and grid difference-are introduced to determine the mutual relationship of individuals in a grid environment. Three grid-based criteria, i.e., grid ranking, grid crowding distance, and grid coordinate point distance, are incorporated into the fitness of individuals to distinguish them in both the mating and environmental selection processes. Moreover, a fitness adjustment strategy is developed by adaptively punishing individuals based on the neighborhood and grid dominance relations in order to avoid partial overcrowding as well as guide the search toward different directions in the archive. Six state-of-the-art EMO algorithms are selected as the peer algorithms to validate GrEA. A series of extensive experiments is conducted on 52 instances of nine test problems taken from three test suites. The experimental results show the effectiveness and competitiveness of the proposed GrEA in balancing convergence and diversity. The solution set obtained by GrEA can achieve a better coverage of the Pareto front than that obtained by other algorithms on most of the tested problems. Additionally, a parametric study reveals interesting insights of the division parameter in a grid and also indicates useful values for problems with different characteristics.

Grid has been widely used in the field of evolutionary multi-objective optimization (EMO) due to its property combining convergence and diversity naturally. Most EMO algorithms of grid-based fitness perform well on problems with two or three objectives, but encounter difficulties in their scalability to many-objective optimization. This paper develops the potential of using grid technique to balance convergence and diversity in fitness for many-objective optimization problems. To strengthen selection pressure and refine comparison level, three hierarchical grid-based criterions are incorporated into fitness to establish a completer order among individuals. Moreover, an adaptive fitness penalty mechanism in environmental selection is employed to guarantee the diversity of archive memory. Based on an extensive comparative study with three other EMO algorithms, the proposed algorithm is found to be remarkably successful in finding well-converged and well-distributed solution set.

Diversity maintenance of solutions is a crucial part in multi-objective optimization. However, most of existing studies show a good distribution with a large computational load or a comparative bad distribution quickly. In this paper, a method for pruning a set of non-dominated solutions using a Spanning Tree is proposed. This approach defines a density estimation metric — Spanning Tree Crowding Distance (STCD). Moreover, information of degree of solution combined with STCD is employed to truncate the population. From an extensive comparative study with three other methods on a number of 2, 3 and 4 objective test problems, the proposed method indicates a good balance among uniformity, spread and execution time.

Hypervolume indicator is a commonly accepted quality measure to assess the set of non-dominated solutions obtained by an evolutionary multiobjective optimization algorithm. Recently, an emerging trend in the design of evolutionary multiobjective optimization algorithms is to directly optimize a quality indicator. In this paper, we propose a hypervolume-based evolutionary algorithm for multiobjective optimization. There are two main contributions of our approach, on one hand, a unique fitness assignment strategy is proposed, on the other hand, we design a slicing based method to calculate the exclusive hypervolume of each individual for environmental selection. From an extensive comparative study with three other MOEAs on a number of two and three objective test problems, it is observed that the proposed algorithm has good performance in convergence and distribution.

Fitness assignment and external population maintenance are two important parts of mufti-objective evolutionary algorithms. In this paper, we propose a new MOEA which uses the information of minimum spanning tree to assign fitness and maintain the external population. Moreover, a Minimum Spanning Tree Crowding Distance (MSTCD) is defined to estimate the density of solutions. From an extensive comparative study with three other MOEAs on a number of two and three objective test problems, it is observed that the proposed algorithm has good performance in convergence and distribution.

Uniformity assessment of approximations of the Pareto-optimal set is an important issue in comparing the performance of multi-objective evolutionary algorithms. Although a number of performance metrics existed, many are applicable to low objective problems (2–3 objectives). In addition, most of the existed metrics are only applied to the final non-dominated set. In this paper, we suggest a running metric which evaluates the uniformity of solutions at every generation of a MOEA run. In particular, this metric can compare the uniformity of population with different size in any number of objectives. With an agglomeration of generation-wise populations, the metric reveals the change of uniformity in a MOEA run or helps provide a comparative evaluation of two or more MOEAs.