Small Language Models (SLMs) offer efficient and practical alternatives to large-scale models in resource-constrained environments. We present a principled framework for constructing SLM ensembles that jointly optimize three competing objectives: prediction accuracy, output diversity, and fairness. Our method combines an interpretable cost-function formulation with a multi-objective evolutionary algorithm to discover Pareto-optimal ensemble configurations. We further introduce a two-stage combiner that produces diverse candidate responses and selects final outputs via embedding-based semantic consensus. Experiments on the MentalChat16k mental-health dialogue dataset show that the best-performing SLM ensemble configurations can match or surpass a fine-tuned Llama 3.1 70B model, achieving improvements of 0.86% in ROUGE-1, 5.84% in ROUGE-2, 4.93% in ROUGE-L, and 7.01% in semantic similarity. These results indicate that strategically orchestrated ensembles of small models can offer competitive or superior performance to significantly larger LLMs, while providing greater flexibility, interpretability, and accessibility for researchers operating under limited resources.

Achieving balance in competitive games is extremely difficult: even small design asymmetries can create unfair advantages that undermine gameplay quality. We show that co-evolutionary algorithms can automatically detect and quantify game imbalance, providing multiple objective metrics that complement traditional manual design methods. Our approach employs competitive co-evolution between two populations representing opposing players' strategies, using alternating multi-objective evolution cycles, to prevent dominance and promote balanced competition. The key innovation is a novel hypervolume-based "movement" metric that measures how much each player sacrifices due to competitive pressure, directly quantifying the game balance. We illustrate our approach through three comprehensive case studies spanning unbalanced and balanced game configurations. Results demonstrate that our proposed multi-objective coevolutionary search approach quantifies imbalance in terms of unequal movement from cooperative to competitive Pareto fronts by both agents, and can successfully identifies when a balanced game has occurred. This work establishes co-evolutionary optimization as a practical tool for automated game balance assessment, with applications extending beyond games to generic competitive multi-agent system requiring fairness guarantees.

Interactive multi-criterion decision-making (iMCDM) procedures rely on a human decision-maker (DM) to iteratively provide preferences, so scalarized subproblems can move toward a preferred Pareto-optimal solution. This human-in-the-loop nature makes systematic benchmarking difficult, as preference information and interaction patterns vary across individuals and problems. To overcome this limitation and to enable computationally-oriented researchers to contribute more profoundly in the MCDM field, we introduce a Machine-based Decision Maker (Machine-DM) that replaces human DMs with pre-trained machine learning models capable of performing the key iMCDM tasks automatically. The Machine-DM predicts objective classifications, such as which objectives should be improved, relaxed, fixed, or allowed to vary and generates corresponding bounding parameters without requiring knowledge of the true target preferred solution. Using this Machine-DM, we develop machine-based versions of four well-known iMCDM procedures: STEM, GUESS, STOM, and NIMBUS. We further propose a set of performance metrics designed to evaluate performance of these iMCDM procedures. Using the Machine-DM concept we also propose a Bench-iMCDM framework for benchmarking iMCDM procedures. Applications to test and engineering problems demonstrate the usefulness of Machine-DM and highlight its potential to serve as a unified framework for comparing a broad class of iMCDM procedures and also to develop new ones.

Many practical problems involve multiple interdependent agents, each aiming to optimize its own objectives. Wargame strategy optimization, which requires optimizing strategies for at least two agents—attackers and defenders—presents unique challenges due to the interdependence of the agents’ strategies. This characteristic necessitates a co-evolutionary approach, where each agent’s strategy is continually adjusted in response to the other’s. The complexity increases when each agent pursues multiple conflicting objectives, resulting in Pareto-optimal strategy sets that require sequential decision-making (DM). To address these challenges, we introduce a novel multi-objective competitive co-evolutionary optimization (MoCCoEv) framework, specifically tailored for wargame strategy optimization. This framework integrates regularity-based search with an iterative and interactive DM approach, fostering a continuous interplay between co-evolving agents. Additionally, we introduce the concept of progressive shrinking, which interactively reduces the dimensions of the agents’ strategy parameters to mirror real-world decision-making by enforcing commitment to earlier moves and facilitating effective strategic choices. Our flexible and adaptable framework also supports alternative strategies, such as deception, and can be applied to other multi-agent optimization problems.

In practice, there is often a need to update the currently implemented (CI) solution to achieve better performance goals catering to new demands or adoption of new technologies. However, the new optimal solution, found by re-optimizing the problem, may be quite different from the CI solution implicating large costs, major changes, and laborious efforts, causing an apathy for its adoption. For such scenarios, we propose a concept of an “innovation path” (IP), containing a sequence of transitional solutions from the existing to the new target solution with gradual and controlled change from one to the next. To discover such intermediate solutions of the IP, we propose a bi-objective formulation with dynamic step-constraints as an IP Problem (IPP), such that a finite set of Pareto-optimal solutions of the resulting IPP become the desired intermediate IP solutions. Due to required gradual discovery of IP solutions, the IP-seeking task happens to be a non-trivial task. We demonstrate the working of the proposed approach on a number of single, two-objective, and many-objective test and engineering problems. The paper concludes with a number of extensions of this study, but the results of this study clearly indicate the usefulness of the proposed approach to other practical problems.

In many practical problem-solving tasks, instead of a single desired solution, often the goal is to find multiple alternate solutions (optimal or otherwise). In such tasks, most often, numerical methods are employed to find one solution at a time by making the desired solution the focus of the ensuing computational task. On the other hand, evolutionary multi-objective optimization (EMO) algorithms – population-based computational procedures – have demonstrated their ability to find multiple optimal solutions resulting from optimizing two or more conflicting objectives simultaneously. In this paper, we highlight the principles of an EMO procedure and discuss how it can be extended to find multiple alternate solutions for a number of different problem-solving tasks encountered in practice. This ‘multiobjectivization’ task requires researchers to choose at least two conflicting goals arising from the specific problem-solving task and apply a suitably modified version of an existing EMO algorithm to find multiple alternate solutions. These extensions broaden EMO research and its applications, and also enable a unified approach for solving various practical problem-solving tasks.

Wargame strategy optimization involves two competing agents—offense and defense—and requires extensive simulations of wargame tools, resulting in high computational costs for evaluating potential strategies. To reduce these costs, surrogate models are employed to approximate the objective functions, with their accuracy directly affecting the optimization process. This paper proposes a novel framework for progressive surrogate modeling to improve the quality of surrogate models while addressing the wargame strategy optimization problem. Our approach utilizes a Multi-objective Competitive Co-Evolutionary (MoCCoEv) algorithm, which iteratively refines the surrogate models. The process begins by using MoCCoEv algorithm to optimize wargame strategies generating new strategies. The new strategies are then replaced in the training data to update and improve the surrogate models. This cyclical process ensures that the models are continuously refined in the regions of interest, leading to more accurate predictions and enhanced optimization results.

Most practical optimization problems involve expensive evaluation procedures for computing objective and constraint functions. To obtain reasonable and accurate solutions close to true Pareto-optimal solutions, evolutionary multi-objective optimization (EMO) algorithms create surrogate models from already-evaluated high-fidelity solutions and use them during optimization to save computational time. However, most surrogate-assisted EMO algorithms are designed to evaluate all objectives and constraints of a solution, if found worthy of a high-fidelity evaluation. Such algorithms are inefficient if objectives and constraints involve heterogeneity with orders of magnitude of difference in evaluation times. Clearly, functions with relatively small evaluation time can be high-fidelity evaluated more often to obtain an overall idea of the potential importance of the solution before deciding to spend more time on evaluating expensive functions. In this paper, we propose an EMO approach that carefully determines which constraints and objectives should be high-fidelity evaluated for every population member and suggests a mixed-fidelity survival selection procedure capable of working with low- and high-fidelity evaluated population members. Results on a number of test and engineering problems indicate the viability of such a constrained multi- and many-objective optimization algorithm and encourage further attention.

Evolutionary Multi-objective Optimization (EMO) algorithms are widely used to solve real-world multi-objective optimization problems, aiming to obtain a set of non-dominated solutions close to the Pareto front. However, most EMO methods assume deterministic decision variables, ignoring inherent uncertainties in engineering applications, which can lead to design failures, especially in reliability-based designs. Reliability-based Multi-objective Optimization (ReMOO) addresses this issue by incorporating variable uncertainty and probabilistic constraints to generate a Reliable Front. ReMOO operates using a bi-level framework: the outer level optimizes objective functions, while the inner level estimates reliability through computationally intensive methods, like Monte Carlo Simulation (MCS) or the Performance Measure Approach (PMA). Additionally, decision-makers (DMs) often select only a subset of reliable solutions, limiting computational efficiency. To overcome these challenges, this paper proposes a Machine Learning-assisted reliability-based Multi-Criteria Decision-Making (ML-ReMCDM) technique. ML models are trained on reliability-based constraints within the decision space before an EMO execution. In the inner loop, ML models predict probabilistic constraints and reliability indices, significantly reducing computational costs. Moreover, the outer loop computes only the DM-preferred segment of the reliable front, further enhancing efficiency. The ML-ReMCDM approach, implemented on several benchmark and real-world examples, demonstrates substantial improvements in computational efficiency as well as practical applicability.

The goal of a generic evolutionary multi- or many-objective algorithm is to explore a search space and find the trade-off optimal solutions for two or more conflicting objectives. In platform-based practical design optimization problems, it is not sufficient to just find a set of trade-off optimal solutions, certain regularity properties are expected in the entire fleet of trade-off solutions. For this purpose, we propose a scalable regularity-based optimization framework - RegEM(a)O - which automatically extracts polynomial regularity principles from the resulting Pareto-optimal front of multi- or many-objective problems. Thereafter, it attempts to search for a regular front of trade-off solutions following a similar form of polynomial regularity principles. Despite being slightly worse than the true Pareto-optimal solutions, regular solutions possess simple properties among them, making them easily interpretable, their inventory easily maintainable, and easily scalable. In this paper, we apply RegEM(a)O to a number of small and large-scale real-world engineering design problems to demonstrate its practical advantage.

Surrogates are commonly used in single and multi-objective optimization studies for quickly evaluating objective functions which are otherwise expensive to evaluate. Starting with a set of high-fidelity evaluated solutions, optimization algorithms generate new in-fill solutions for further high-fidelity evaluations to improve the previous surrogate models towards the optimal regions of the search space. In terms of evaluating in-fill solutions, most current optimization algorithms must evaluate all objectives using high-fidelity means, irrespective of relative differences in their evaluation times. In this paper, we address the issue of handling objective functions having heterogeneous high-fidelity evaluation times and propose a framework which combines evaluation time and surrogate accuracy of each objective to decide which population members should undergo a high-fidelity evaluation and for which objectives. Our proposed approach deals with heterogeneously evaluated solutions - some objectives with high-fidelity and some with surrogates - in a generic way. Initial results on a number of two and three-objective problems, presented in this paper, show promise for approach and encourages to launch a deeper study on more complex and constrained problems.

A multi-criterion optimization task requires a decision-making activity during or after the Pareto solutions are found. A convenient and effective decision-making task can be achieved with well-represented Pareto solutions and by means of user-friendly and easily comprehensible visualization tools. To have a comprehensive idea of the spread of Pareto solutions obtained by evolutionary multi-criterion optimization (EMO) or other algorithms, each Pareto solution can be associated with a unique identifier - a vector of size of the dimension of the Pareto surface. While the Pareto solutions can lie on an arbitrary surface with non-domination properties but having its shape and size largely dependent on the problem being solved, these identifiers can have much simpler properties, such as lying on a unit simplex or a unit sphere, or directly relating to preference values. Once achieved, these unique identifiers can help EMO algorithms evaluate the extent of the spread of solutions during the optimization process and decision-makers to have a better understanding of trade-offs among solutions to make better decisions. Moreover, apparent gaps or other complexities of the obtained Pareto solutions can be more easily located in the identifier space. In this paper, we present and compare five identifiers - ideal point reference vectors (RV), nadir point RVs, projection RVs, pseudo-weights, and angle vectors - with respect to their advantages and disadvantages in decision-making and visualization purposes. We also demonstrate that, if desired by the decision maker, these alternative identifiers can be used during optimization to achieve a good distribution of solutions in this space.

In practice, there is often a need to modify an existing implemented solution in order to achieve a better performance or accommodate new demands or technologies. However, a new optimal solution for the updated problem may be quite different from the existing solution, thereby causing an apathy for its implementation involving large cost, changes, and efforts. For such scenarios, we propose a concept of an "innovation path" (IP) containing a sequence of intermediate solutions from the existing to the new target solution with gradual and controlled change from one to the next. To discover intermediate solutions of the IP simultaneously, we propose a bi-objective formulation of the original problem, so that Pareto-optimal solutions of the resulting bi-objective problem become the IP solutions. We demonstrate the working of the proposed approach on a number of single and two-objective test and engineering problems. Results are encouraging and suggest further research and application to make the proposed innovation path approach more efficient and practical.

Evolutionary multi-objective optimization (EMO) algorithms are widely used to solve problems involving multiple conflicting objectives. In general, these problems result in a well-distributed and diverse set of Pareto-optimal solutions, consisting of individual objective-optimal solutions at their extreme and various compromise objective solutions at their core. However, in practice, decision-makers (DMs) usually have certain pre-conceived preference information which may make a majority of the Pareto solution set uninteresting to the DMs. In such cases, DM's preference information can be utilized to update EMO algorithms to focus on the preferred part of the Pareto set, rather than the entire Pareto set. While EMO researchers have proposed preference-based EMO algorithms for this purpose, appropriate metrics to evaluate their performance have received lukewarm attention. In this paper, we critically analyze a recently proposed preference-based hypervolume (R-HV) metric for its sensitivity to handle various scenarios and propose an updated version to remedy the difficulties associated with it. The updated R-HV metric is then compared with the original R-HV metric on solutions obtained from a number of preference-based EMO algorithms. The suggestion of a more appropriate R-HV metric presented in this paper should encourage further research in preference-based multi-objective optimization.

Black-box artificial intelligence (AI) induction methods such as deep reinforcement learning (DRL) are increasingly being used to find optimal policies for a given control task. Although policies represented using a black-box AI are capable of efficiently executing the underlying control task and achieving optimal closed-loop performance-controlling the agent from the initial time step until the successful termination of an episode, the developed control rules are often complex and neither interpretable nor explainable. In this article, we use a recently proposed nonlinear decision-tree (NLDT) approach to find a hierarchical set of control rules in an attempt to maximize the open-loop performance for approximating and explaining the pretrained black-box DRL (oracle) agent using the labeled state-action dataset. Recent advances in nonlinear optimization approaches using evolutionary computation facilitate finding a hierarchical set of nonlinear control rules as a function of state variables using a computationally fast bilevel optimization procedure at each node of the proposed NLDT. In addition, we propose a reoptimization procedure for enhancing the closed-loop performance of an already derived NLDT. We evaluate our proposed methodologies (open- and closed-loop NLDTs) on different control problems having multiple discrete actions. In all these problems, our proposed approach is able to find relatively simple and interpretable rules involving one to four nonlinear terms per rule, while simultaneously achieving on par closed-loop performance when compared to a trained black-box DRL agent. A postprocessing approach for simplifying the NLDT is also suggested. The obtained results are inspiring as they suggest the replacement of complicated black-box DRL policies involving thousands of parameters (making them noninterpretable) with relatively simple interpretable policies. The results are encouraging and motivating to pursue further applications of proposed approach in solving more complex control tasks.

Evolutionary multiobjective and many-objective optimization (EMO and EMaO) algorithms are increasingly used to identify the true shape and location of the Pareto-optimal front using a few representative well-converged and well-distributed solutions. The reason for their popularity is due to their ability to provide a better understanding of objective relationships for optimal solutions, and also to facilitate the choice of a preferred solution using an interactive or post-optimal multicriterion decision analysis. However, since EMO and EMaO algorithms are stochastic, a single application may not provide a true representative set with a desired number of Pareto solutions reliably in repetitive runs and importantly with a well-distributed set of solutions. In this article, we propose a multistage framework involving reference-vector-based evolutionary multi- and many-objective algorithms (MuSt-EMO and MuSt-EMaO) that attempts to recursively rectify shortcomings of previous stages by careful executions of subsequent stages so that a prescribed number of well-distributed and well-converged solutions are achieved at the end. The proposed multistage approach is implemented to a number of popular reference vector-based EMO/EMaO algorithms and is applied on various multi- and many-objective test and real-world problems.

Experienced users often have useful knowledge and intuition in solving real-world optimization problems. User knowledge can be formulated as intervariable relationships to assist an optimization algorithm in finding good solutions faster. Such intervariable interactions can also be automatically learned from high-performing solutions discovered at intermediate iterations in an optimization run—a process called innovization. These relations, if vetted by the users, can be enforced among newly generated solutions to steer the optimization algorithm toward practically promising regions in the search space. Challenges arise for large-scale problems where the number of such variable relationships may be high. This article proposes an interactive knowledge-based evolutionary multiobjective optimization (IK-EMO) framework that extracts hidden variable-wise relationships as knowledge from evolving high-performing solutions, shares them with users to receive feedback, and applies them back to the optimization process to improve its effectiveness. The knowledge extraction process uses a systematic and elegant graph analysis method which scales well with the number of variables. The working of the proposed IK-EMO is demonstrated on three large-scale real-world engineering design problems. The simplicity and elegance of the proposed knowledge extraction process and the achievement of high-performing solutions quickly indicate the power of the proposed framework. The results presented should motivate further such interaction-based optimization studies for their routine use in practice.

Multi-objective optimization problems give rise to a set of Pareto-optimal solutions, each of which makes a trade-off among the objectives. When multiple Pareto-optimal solutions are to be implemented for different applications as platform-based solutions, a solution principle common to them is highly desired for easier understanding, implementation, and management purposes. In this paper, we propose a systematic search methodology that deviates from finding Pareto-optimal solutions, but finds a set of near Pareto-optimal solutions sharing common principles of a desired structure and still possessing a trade-off among objectives. After proposing the regular evolutionary multi-objective optimization (RegEMO) algorithm, we first demonstrate its working principle on a number of constrained and unconstrained multi-objective test problems. Thereafter, we demonstrate the practical significance of the proposed approach to a number of engineering design problems. Searching for a set of solutions with common principles of desire, rather than theoretical Pareto-optimal solutions without any common structure, is a practically meaningful task and this paper should encourage more such practice-oriented developments of EMO in the near future.

Many critical science, societal, and engineering fields contain large-scale multiobjective optimization problems (LSMOPs), comprised of many decision variables. However, as the number of decision variables increases, optimization algorithms face exponentially large search spaces, thereby exhibiting a degraded performance. Nonetheless, LSMOPs whose optimal solutions correspond to sparse variable vectors can be solved more efficiently by evolutionary multiobjective optimization (EMO) algorithms. Despite the great recent strides in developing generic EMO algorithms for sparse LSMOPs, there is still room for improvement. Specifically, algorithms still struggle to find convergent and diverse Pareto fronts in an acceptable amount of time when solving sparse LSMOPs with thousands of decision variables. To better solve sparse LSMOPs, we propose a novel set of evolutionary operators to adapt small-scale EMO algorithms for sparse LSMOPs. These simple, novel, and effective operators include varied striped sparse population sampling (VSSPS), sparse simulated binary crossover (S-SBX), and sparse polynomial mutation (S-PM). These operators, combined with nondominated sorting genetic algorithm II (NSGA-II), make the proposed S-NSGA-II algorithm. S-NSGA-II runs near-universally faster than existing methods for problems containing up to 6400 decision variables, while performing as well as or better than contemporary sparse LSMOP algorithms with respect to hypervolume, especially, with problems with larger than 5000 decision variables.

The ongoing advancements in network architecture design have led to remarkable achievements in deep learning across various challenging computer vision tasks. Meanwhile, the development of neural architecture search (NAS) has provided promising approaches to automating the design of network architectures for lower prediction error. Recently, the emerging application scenarios of deep learning (e.g., autonomous driving) have raised higher demands for network architectures considering multiple design criteria: number of parameters/weights, number of floating-point operations, inference latency, among others. From an optimization point of view, the NAS tasks involving multiple design criteria are intrinsically multiobjective optimization problems; hence, it is reasonable to adopt evolutionary multiobjective optimization (EMO) algorithms for tackling them. Nonetheless, there is still a clear gap confining the related research along this pathway: on the one hand, there is a lack of a general problem formulation of NAS tasks from an optimization point of view; on the other hand, there are challenges in conducting benchmark assessments of EMO algorithms on NAS tasks. To bridge the gap: 1) we formulate NAS tasks into general multiobjective optimization problems and analyze the complex characteristics from an optimization point of view; 2) we present an end-to-end pipeline, dubbedEvoXBench, to generate benchmark test problems for EMO algorithms to run efficiently—without the requirement of GPUs or Pytorch/Tensorflow; and 3) we instantiate two test suites comprehensively covering two datasets, seven search spaces, and three hardware devices, involving up to eight objectives. Based on the above, we validate the proposed test suites using six representative EMO algorithms and provide some empirical analyses. The code ofEvoXBenchis available athttps://github.com/EMI-Group/EvoXBench.

This paper proposes a machine learning (ML) based unified innovized progress (UIP) operator to simultaneously enhance the convergence and diversity capabilities of reference vector based evolutionary multi-and many-objective optimization algorithms, namely, RV-EMâOAs. Recent studies have demonstrated that ML intervention could help enhance convergence of RV-EMâOAs by capturing efficient search directions, through mapping of inter-generational solutions along the different reference vectors (RVs). This paper first demonstrates that ML intervention can also help enhance the diversity capability of RV-EMâOAs through mapping of intra-generational solutions across the RVs. Subsequently, the UIP operator integrates the convergence and diversity enhancement capabilities in a manner that is generic -applicable to different RV-EMâOAs, and practicable -not requiring any extra solution evaluations over the base RV-EMâOAs. Based on 24,056 experimental runs on multi-and many-objective problems, the UIP operator, when integrated with different RV-EMâOAs, has provided statistically better performance in about 36% instances, and better or equivalent in about 92% instances, compared to the respective base RV-EMâOAs.

Owing to the stochasticity of Evolutionary Multi-objective Optimization (EMO) Algorithms and an application with a limited budget of solution evaluations, a perfectly converged and uniformly distributed Pareto-optimal (PO) front cannot be always guaranteed. Thus, a subsequent decision-making step or a curiosity on the part of the optimization researcher may demand solutions at regions not well-represented by the obtained PO front. In this study, we propose to train Machine Learning (ML) models to capture the mapping between unique identifiers of PO solutions – pseudo-weight vectors, computed from the existing PO front data, and their corresponding decision variable vectors. These learned models can then be used to predict PO decision variables for any new desired pseudo-weight vector. We evaluate the proposed approach with two different ML methods on a variety of multi-and many-objective test and real-world problems. This procedure can also be incorporated into an EMO algorithm to find a better converged set of PO solutions, attempt to fill apparent gaps, and find more non-dominated solutions at preferred regions of the PO front, facilitating a number of key advances for multi-objective optimization and decision-making tasks.

Selecting the appropriate best management practices (BMPs) is crucial for reducing pollution levels and improving the watershed’s water quality. However, identifying cost-effective BMP combinations for various locations is challenging, especially when using computationally expensive evaluation procedures like the Chesapeake Assessment Scenario Tool (CAST). This study presents a customized and hybrid evolutionary multiobjective optimization (EMO) algorithm aimed at enhancing the water quality in the Chesapeake Bay Watershed for two conflicting objectives: 1) cost of BMP implementation and 2) the amount of resulting nitrogen loading to streams. First, we present a surrogate model-based optimization approach and evaluate its accuracy and execution time against the CAST evaluation system. Then, we present a hybrid two-stage EMO procedure, which is initialized with solutions obtained from a point-based$\epsilon$-constraint procedure and works with a repair operator to satisfy equality constraints. The hybrid EMO procedure yields a set of nondominated tradeoff solutions for problems with as few as 1012 variables (West Virginia’s Tucker County) to as large as 153 818 variables (the whole state of West Virginia). Alternate tradeoff solutions provide a knowledge of different possible options and also importantly provide a flexible method of arriving at a single preferred solution for deployment. The EMO procedure is then integrated with CAST using recent RESTful API approaches, and interesting accuracy versus computational tradeoffs are discussed. Finally, a number of interesting insights of the scale-up optimization study reveal promising strategies to scale the application to multiple counties and the watershed level.

Innovization is a task for analyzing multiple Pareto-optimal solutions obtained by an evolutionary multi-objective optimization (EMO) algorithm to extract common features in the decision variables, leading to design rules or solution principles. The principles derived from innovized principles can provide valuable insights to the users about “how to create an optimal solution?”. Manual or automated machine learning-based innovization methods were proposed in the literature to extract innovized principles in a problem. Although different problems may demand different structures of the rules, the innovized rules can also be utilized to improve the performance of the subsequent iterations of the optimization algorithm or help in executing an efficient re-optimization of the same problem. In this paper, we consider a large-scale and multi-objective complex optimization task of minimizing cost and nitrogen loading in certain counties within the Chesapeake Bay Watershed (CBW) and find multiple trade-off solutions using the NSGA-III approach applied to the CBW's real evaluator tool (The Chesapeake Assessment Scenario Tool–CAST). 205 Best Management Practices (BMPs) are considered to be implemented at each land-river segment within a county, leading to as many as 65,260 variables for the resulting multi-objective optimization procedure. First, hundreds of trade-off solutions found by the CAST-NSGA-III procedure are analyzed manually to find the top-most BMPs used in them. After that, a re-optimization of CAST-NSGA-III is run with a few critical BMPs (resulting in a decrease of the variable to a range between 3% and 33%) found to commonly appear in the trade-off solution set of the previous runs. Interestingly, the resulting trade-off front with reduced BMPs is similar to the original run achieved with tens of thousands of variables. The findings are intriguing and demonstrate the efficacy of innovation in addressing intricate, real-world issues at a significant scale.

The primary task of evolutionary multi-objective optimization (EMO) is to find the globally best Pareto-optimal front. However, often decision makers (DMs) are not interested in obtaining the global frontier. Instead, they prefer a set of solutions for which there is no significant change in objective values within a small neighborhood of each decision variable vector, resulting in a set of robust solutions. The corresponding objective vectors are said to lie on the robust front. In practical applications, such as in engineering designs, designers are interested in robust designs which are less sensitive to the perturbation in the design variables and parameters, caused by the manufacturing process tolerances, material non-uniformity, uncertainties in supply-chain process, and by many other practical matters. An earlier robust EMO study proposed two different robustness measures and used the elitist non-dominated sorting genetic algorithms (NSGA-II) to find respective robust fronts. However, the limitation of NSGA-II in generating well-distributed and diverse set solutions for many-objective optimization, the robust optimization concept must be extended with evolutionary many-objective optimization (EMaO) algorithms to investigate the efficacy in more than three-objective problems. This study proposes an extension of test problems for robust many-objective optimization tasks and demonstrates the performance of updated NSGA-III procedure to two to eight-objective test and real-world problems.

Efforts to improve the adversarial robustness of convolutional neural networks have primarily focused on developing more effective adversarial training methods. In contrast, little attention was devoted to analyzing the role of architectural elements (e.g., topology, depth, and width) on adversarial robustness. This paper seeks to bridge this gap and present a holistic study on the impact of architectural design on adversarial robustness. We focus on residual networks and consider architecture design at the block level as well as at the network scaling level. In both cases, we first derive insights through systematic experiments. Then we design a robust residual block, dubbed RobustResBlock, and a compound scaling rule, dubbed RobustScaling, to distribute depth and width at the desired FLOP count. Finally, we combine RobustResBlock and RobustScaling and present a portfolio of adversarially robust residual networks, RobustResNets, spanning a broad spectrum of model capacities. Experimental validation across multiple datasets and adversarial attacks demonstrate that RobustResNets consistently outperform both the standard WRNs and other existing robust architectures, achieving state-of-the-art AutoAttack robust accuracy 63.7% with 500K external data while being 2× more compact in terms of parameters. Code is available at this URL.

Automated machine learning (AutoML) greatly eases human efforts in architecture engineering. However, mainstream AutoML methods like neural architecture search (NAS) are customized for well-designed search spaces wherein promising architectures are densely distributed. In contrast, AutoML-Zero builds machine-learning algorithms using basic primitives and can explore novel architectures beyond human knowledge. AutoML-Zero shows the potential to deploy machine learning systems by not taking advantage of either feature engineering or architectural engineering. In its current form, it only optimizes a single objective like accuracy and has no mechanism to ensure that the constraints of real-world applications are satisfied. We propose a multi-objective variant of AutoML-Zero called MOAZ, that distributes solutions on a Pareto front by trading off accuracy against the computational complexity of the machine learning algorithm. In addition to generating different Pareto-optimal solutions, MOAZ can effectively explore the sparse search space to improve search efficiency. Experimental results on linear regression tasks show MOAZ reduces the median complexity by 87.4% compared to AutoML-Zero while accelerating the median target performance achievement speed by 82%. In addition, our preliminary results on non-linear regression tasks show the potential for further improvements in search accuracy and for reducing the need for human intervention in AutoML.

Evolutionary multi-objective optimization (EMO) algorithms are predominantly used for solving multi- and many-objective optimization problems to arrive at the respective Pareto front. From a practical point of view, it is desirable for a decision-maker (DM) to consider objective vectors that are less sensitive to the small perturbation in design variables and problem parameters. Such insensitive, yet closer to Pareto-optimal solutions, lie on the so-called robust front. In real-world applications, such as engineering design and process optimization problems, perturbations in variables come from manufacturing tolerances, uncertainties in material properties, variations in operating conditions, etc. The existing EMO literature on robustness studies emphasized on finding the entire robust front, but hardly considered robustness in both optimization and decision-making tasks. In this paper, we propose and evaluate different algorithmic implementations of three aspects - multi-objective optimization, robustness consideration, and multi-criterion decision-making - together. Based on experimental results on two to eight-objective problems, we discuss the outcomes and advantages of different integration approaches of these three aspects and present the most effective combined approach. The results are interesting and should pave the way to develop more efficient multi-objective robust optimization and decision-making (MORODM) procedures for handling practical problems with uncertainties.

Autonomous robots deployed in the real world will need control policies that rapidly adapt to environmental changes. To this end, we propose AutoRobotics-Zero (ARZ), a method based on AutoML-Zero that discovers zero-shot adaptable policies from scratch. In contrast to neural network adaption policies, where only model parameters are optimized, ARZ can build control algorithms with the full expressive power of a linear register machine. We evolve modular policies that tune their model parameters and alter their inference algorithm on-the-fly to adapt to sudden environmental changes. We demonstrate our method on a realistic simulated quadruped robot, for which we evolve safe control policies that avoid falling when individual limbs suddenly break. This is a challenging task in which two popular neural network baselines fail. Finally, we conduct a detailed analysis of our method on a novel and challenging non-stationary control task dubbed Cataclysmic Cartpole. Results confirm our findings that ARZ is significantly more robust to sudden environmental changes and can build simple, interpretable control policies.

Abstract This work aims at reviewing the state of the art of the field of lexicographic multi/many-objective optimization. The discussion starts with a review of the literature, emphasizing the numerous application in the real life and the recent burst received by the advent of new computational frameworks which work well in such contexts, e.g., Grossone Methodology. Then the focus shifts on a new class of problems proposed and studied for the first time only recently: the priority-levels mixed-pareto-lexicographic multi-objective-problems (PL-MPL-MOPs). This class of programs preserves the original preference ordering of pure many-objective lexicographic optimization, but instantiates it over multi-objective problems rather than scalar ones. Interestingly, PL-MPL-MOPs seem to be very well qualified for modeling real world tasks, such as the design of either secure or fast vehicles. The work also describes the implementation of an evolutionary algorithm able to solve PL-MPL-MOPs, and reports its performance when compared against other popular optimizers.

Many societal and industrial problem-solving tasks involving search, optimization, design, and management are conveniently decomposed into hierarchical subproblems. While this process allows a systematic procedure to have a multistakeholder solution, the independent decision-making process for the lower level problem causes a deviation in the expected outcome of the upper level problem. In this article, we provide a new and computationally efficient evolutionary approach allowing upper level decision makers to analyze the vagaries of lower level decision making when choosing a preferred solution with the minimum deviation from their expectations. This concept is novel and pragmatic. We demonstrate the concept through a search for optimistic–pessimistic tradeoff solutions found by an evolutionary multiobjective optimization approach first on two difficult test problems, then on a watershed management problem and a telecommunication management problem. The approach is generic and can be applied to similar hierarchical management problems to achieve minimum deviation with a more predictive and reliable outcome. The proposed solution procedure is found to choose an optimistic solution that has approximately 31%–65% reduced deviation compared to another optimistic solution chosen at random in the test problems and approximately 85%–95% reduced deviation in the two practical problems, making the method of this study applicable to practical hierarchical problems.

The practicality of Pareto-dominance in solving many-objective optimization problems becomes questionable due to its inability to factor the critical human decision-making (HDM) elements, including the number of better objectives, the degree of betterment in objectives, and objectives’ relative preference. Relevant dominance principles are recently proposed to incorporate the first two HDM elements, often with the need for new tunable parameters. This article proposes a high-fidelity-dominance principle that factors all the three HDM elements, explicitly and simultaneously, and without requiring tuning of any parameter. This principle has been implemented in a reference-vector-based framework, leading to a computationally efficient many-objective evolutionary algorithm (MaOEA), namely, localized high-fidelity-dominance-based EA (LHFiD). Critically, LHFiD also has an inbuilt mechanism for on-the-fly determination of the timing for: 1) intermittent Nadir point estimation that enables faster convergence and 2) its self-termination that bears practically utility. This article is based on an extensive study involving 41 912 experiments, in which the proposed LHFiD approach is compared with the existing competitive MaOEAs. This article reports statistically better performance in about 60% instances, making it practical and worthy of further investigation and application.

Metaheuristics are known to be effective for solving a broad category of optimization problems. However, most heuristics require different parameter settings appropriately for a problem class or even for a specific problem. Researchers address this commonly by performing a parameter tuning study (also known as hyper-parameter optimization) or developing a parameter control mechanism that changes parameters dynamically. Whereas parameter tuning is computationally expensive and limits the parameter configuration to stay constant throughout the run, parameter control is also a challenging task because all dynamics induced by various operators must be learned to make an appropriate adaptation of parameters on the fly. This paper investigates parameter tuning and control for a well-known optimization method - differential evolution (DE). In contrast to most existing DE practices, an additional individualistic evolutionary operator called polynomial mutation is incorporated into the offspring creation. Results on test problems with up to 50 variables indicate that mutation can be helpful for multi-modal problems to escape from local optima. On the one hand, the effectiveness of parameter tuning for a specific problem becomes apparent; on the other hand, its generalization capabilities seem to be limited. Moreover, a generic coevolutionary approach for parameter control outperforms a random choice of parameters. Recognizing the importance of choosing a suitable parameter configuration to solve any optimization problem, we have incorporated a standard implementation of both tuning and control approaches into a single framework, providing a direction for the evolutionary computation and optimization researchers to use and further investigate the effects of parameters on DE and other metaheuristics-based algorithms.

The careful selection of Best Management Practices (BMPs) to reduce loading, such as nitrogen, phosphorus, and sediments, can substantially improve the water quality of water-sheds. This paper introduces the first implementation of a hybrid and customized evolutionary multi-objective (EMO) algorithm to improve the Chesapeake Bay Watershed's (CBW) water quality. To make the algorithm scalable, we inject a few solutions obtained using an integer programming algorithm (IPOPT) in the initial population of EMO. Also, a repair operator is applied to satisfy every equality constraint. Combining these approaches can find a set of non-dominated trade-off solutions from 1,012 variables (Tucker county in West Virginia) to a staggering 153,818 variable problem (the whole state of West Virginia). Furthermore, a pre-liminary analysis of obtained trade-off solutions finds interesting properties of BMP allocations, providing an optimistic picture of applying the proposed customized optimization algorithm in addressing other bigger states leading to the whole Chesapeake Bay watershed.

Dominance move (DoM) is a binary quality indicator that can be used in multiobjective and many-objective optimization to compare two solution sets obtained from different simulations. The DoM indicator can differentiate the sets for certain important features, such asconvergence,spread,uniformity, andcardinality. DoM does not require any reference point or any representative Pareto solution set, and it has an intuitive and physical meaning, similar to the$\epsilon $-indicator. It calculates the minimum total move of members of one set so that all elements in another set are to be dominated or identical to at least one member of the first set. Despite the aforementioned desired properties, DoM is hard to calculate, particularly for higher dimensions. There is an efficient and exact method to calculate it in biobjective problems. This work proposes a novel approach to calculate DoM using a mixed-integer programming (MIP) approach, which can handle two sets with two or more objectives and is shown to overcome the issue of information loss associated with the$\epsilon $-indicator. Experiments in the biobjective space are done to verify the model’s correctness. Furthermore, other experiments, using 3-, 5-, 10-, 15-, 20-, 25-, and 30-objective problems, are performed to show how the model behaves in higher dimensional cases. Algorithms, such as IBEA, MOEA/D, NSGA-III, NSGA-II, and SPEA2, are used to generate the solution sets; however, any other algorithm can also be used with the proposed MIP-DoM indicator. Further extensions are discussed to handle certain idiosyncrasies with some solution sets and improve the quality indicator and its use for other scenarios.

Innovization is a task of learning common relationships among some or all of the Pareto-optimal (PO) solutions in multi- and many-objective optimization problems. A recent study has shown that a chronological sequence of nondominated solutions obtained along the successive generations of an optimizer possesses salient patterns that can be learnt using a Machine Learning (ML) model, and can help the offspring solutions progress in useful directions. This article enhances each constitutive module of the above approach, including novel interventions on management of the convergence-diversity tradeoff while mapping the solutions from the previous and current generation; use of a computationally more efficient ML method, namely, Random Forest (RF); and changing the manner and extent to which the learnt ML model is utilized toward advancement of the offspring. The proposed modules constitute what is called the enhanced innovized progress (IP2) operator. To investigate the search efficacy provided by the IP2 operator, it is integrated with multi-and many-objective optimization algorithms, such as NSGA-II, NSGA-III, MOEA/D, and MaOEA-IGD, and tested on a range of two- to ten-objective test problems, and five real-world problems. Since the IP2 operator utilizes the history of gradual and progressive improvements in solutions over generations, without requiring any additional solution evaluations, it opens up a new direction for ML-assisted evolutionary optimization.

Learning effective problem information from already explored search space in an optimization run, and utilizing it to improve the convergence of subsequent solutions, have represented important directions in Evolutionary Multi-objective Optimization (EMO) research. In this article, a machine learning (ML)-assisted approach is proposed that: (a) maps the solutions from earlier generations of an EMO run to the current non-dominated solutions in the decision space ; (b) learns the salient patterns in the mapping using an ML method, here an artificial neural network (ANN); and (c) uses the learned ML model to advance some of the subsequent offspring solutions in an adaptive manner. Such a multi-pronged approach, quite different from the popular surrogate-modeling methods, leads to what is here referred to as the Innovized Progress (IP) operator. On several test and engineering problems involving two and three objectives, with and without constraints, it is shown that an EMO algorithm assisted by the IP operator offers faster convergence behavior, compared to its base version independent of the IP operator. The results are encouraging, pave a new path for the performance improvement of EMO algorithms, and set the motivation for further exploration on more challenging problems.

This paper discusses a dimensionality reduction procedure to tackle a many-objective formulation of a Vehicle Routing Problem with a Demand Responsive Transport (VRPDRT). The problem formulation presents eight objective functions that aim to reduce the operating costs while meeting passenger needs and providing a high-quality service. Two different dimensionality reduction-based approaches, aggregation and feature selection are employed to transform the many-objective formulation into a bi-objective one. The reduction, applied during the search evolution, follows a hierarchical clustering technique in which the objective functions' similarity and conflict are explored. The proposed approaches are compared with a classic version of MOEA/D that solves the problem in its original formulation. Moreover, different dimensionality reduction frequencies are tested to assess the impact on the algorithms' performance. When comparing the outcomes in the original objective space, the results show that the aggregation approach outperforms the feature selection method, regardless of the dimensionality reduction frequency. Furthermore, while there is no statistical difference between the MOEA/D and the aggregation approach and the MOEA/D outperforms the feature selection approaches.

A unique way evolutionary algorithms (EAs) are different from other search and optimization methods is their recombination operator. For real-parameter problems, it takes two or more high-performing population members and blends them to create one or more new solutions. Many real-parameter recombination operators have been proposed in the literature. Each operator involves at least a parameter that controls the extent of exploration (diversity) of the generated offspring population. It has been observed that different recombination operators and specific parameters produce the best performance for different problems. This fact imposes the user to use different operator and parameter combinations for every new problem. While an automated algorithm configuration method can be applied to find the best combination, in this paper, we propose an Ensembled Crossover based Evolutionary Algorithm (EnXEA), which considers a number of recombination operators simultaneously. Their parameter values and applies them with a probability updated adaptively in proportion to their success in creating better offspring solutions. Results on single-objective and multi-objective, constrained, and unconstrained problems indicate that EnXEA's performance is close to the best individual recombination operation for each problem. This alleviates the use of expensive parameter tuning either adaptively or manually for solving a new problem.

Co-evolutionary algorithms involve two co-evolving populations, each having its own set of objectives and constraints, that interact with each other during function evaluation. Co-evolutionary algorithms are of great interest in cooperative and competing games and search tasks in which multiple agents having different interests are in play. Despite a number of single-objective co-evolutionary studies, there has been limited interest in multi-objective co-evolutionary algorithms. A recent study has revealed that in addition to the challenges associated with the development of an efficient algorithm, a proper understanding of the conflicting objectives within a single population and their interaction among objectives of the second population becomes extremely difficult to comprehend. In this paper, we extend the previous proof-of-principle multi-objective co-evolutionary (MOCoEv) study in three important directions. First, we enhance MOCoEv's ability to handle mixed cooperating and conflicting scenarios among different players. Second, we propose an iterative multi-criterion decision-making (MCDM) approach to demonstrate how, in an arms-race type scenario, the most appropriate solution can be selected from the obtained Pareto-optimal solution set iteratively. Third, we extend the previous MOCoEv algorithm with a many-objective evolutionary algorithm (NSGA-III) to make them applicable to three or more objectives for each player. These three developments reveal better insights about the intricate issues related to multiple objectives and decision-making for co-evolutionary optimization and take MOCoEv a step closer to solving more complex multi-player problems.

In the last two decades, significant effort has been made to solve computationally expensive optimization problems using surrogate models. Regardless of whether surrogates are the primary drivers of an algorithm or improve the convergence of an existing method, most proposed concepts are rather specific and not very generalizable. Some important considerations are selecting a baseline optimization algorithm, a suitable surrogate methodology, and the surrogate's involvement in the overall algorithm design. This paper proposes a probabilistic surrogate-assisted framework (PSAF), demonstrating its applicability to a broad category of single-objective optimization methods. The framework injects knowledge from a surrogate into an existing algorithm through a tournament-based procedure and continuing the optimization run on the surrogate's predictions. The surrogate's involvement is determined by updating a replacement probability based on the accuracy from past iterations. A study of four well-known population-based optimization algorithms with and without the proposed probabilistic surrogate assistance indicates its usefulness in achieving a better convergence. The proposed framework enables the incorporation of surrogates into an existing optimization algorithm and, thus, paves the way for new surrogate-assisted algorithms dealing with challenges in less frequently addressed computationally expensive functions, such as different variable types, large dimensional problems, multiple objectives, and constraints.

An objective normalization strategy is essential in any evolutionary multiobjective or many-objective optimization (EMO or EMaO) algorithm, due to the distance calculations between objective vectors required to compute diversity and convergence of population members. For the decomposition-based EMO/EMaO algorithms involving the Penalty Boundary Intersection (PBI) metric, normalization is an important matter due to the computation of two distance metrics. In this article, we make a theoretical analysis of the effect of instabilities in the normalization process on the performance of PBI-based MOEA/D and a proposed PBI-based NSGA-III procedure. Although the effect is well recognized in the literature, few theoretical studies have been done so far to understand its true nature and the choice of a suitable penalty parameter value for an arbitrary problem. The developed theoretical results have been corroborated with extensive experimental results on three to 15-objective convex and non-convex instances of DTLZ and WFG problems. The article, makes important theoretical conclusions on PBI-based decomposition algorithms derived from the study.

Neural architecture search (NAS) has emerged as a promising avenue for automatically designing task-specific neural networks. Existing NAS approaches require one complete search for each deployment specification of hardware or objective. This is a computationally impractical endeavor given the potentially large number of application scenarios. In this paper, we propose Neural Architecture Transfer (NAT) to overcome this limitation. NAT is designed to efficiently generate task-specific custom models that are competitive under multiple conflicting objectives. To realize this goal we learn task-specific supernets from which specialized subnets can be sampled without any additional training. The key to our approach is an integrated online transfer learning and many-objective evolutionary search procedure. A pre-trained supernet is iteratively adapted while simultaneously searching for task-specific subnets. We demonstrate the efficacy of NAT on 11 benchmark image classification tasks ranging from large-scale multi-class to small-scale fine-grained datasets. In all cases, including ImageNet, NATNets improve upon the state-of-the-art under mobile settings ( ≤ 600M Multiply-Adds). Surprisingly, small-scale fine-grained datasets benefit the most from NAT. At the same time, the architecture search and transfer is orders of magnitude more efficient than existing NAS methods. Overall, experimental evaluation indicates that, across diverse image classification tasks and computational objectives, NAT is an appreciably more effective alternative to conventional transfer learning of fine-tuning weights of an existing network architecture learned on standard datasets. Code is available at https://github.com/human-analysis/neural-architecture-transfer.

For supervised classification problems involving design, control, and other practical purposes, users are not only interested in finding a highly accurate classifier but they also demand that the obtained classifier be easily interpretable. While the definition of interpretability of a classifier can vary from case to case, here, by a humanly interpretable classifier, we restrict it to be expressed in simplistic mathematical terms. As a novel approach, we represent a classifier as an assembly of simple mathematical rules using a nonlinear decision tree (NLDT). Each conditional (nonterminal) node of the tree represents a nonlinear mathematical rule (split-rule) involving features in order to partition the dataset in the given conditional node into two nonoverlapping subsets. This partitioning is intended to minimize the impurity of the resulting child nodes. By restricting the structure of the split-rule at each conditional node and depth of the decision tree, the interpretability of the classifier is ensured. The nonlinear split-rule at a given conditional node is obtained using an evolutionary bilevel optimization algorithm, in which while the upper level focuses on arriving at an interpretable structure of the split-rule, the lower level achieves the most appropriate weights (coefficients) of individual constituents of the rule to minimize the net impurity of two resulting child nodes. The performance of the proposed algorithm is demonstrated on a number of controlled test problems, existing benchmark problems, and industrial problems. Results on 2-500 feature problems are encouraging and open up further scopes of applying the proposed approach to more challenging and complex classification tasks.

Most evolutionary many-objective optimization (EMaO) algorithms start with a description of a number of the predefined set of reference points on a unit simplex. So far, most studies have used the Das and Dennis's structured approach for generating well-spaced reference points. Due to the highly structured nature of the procedure, this method cannot produce an arbitrary number of points, which is desired in an EMaO application. Although a layer-wise implementation has been suggested, EMO researchers always felt the need for a more generic approach. Motivated by earlier studies, we introduce a metric for defining well-spaced points on a unit simplex and propose a number of viable methods for generating such a set. We compare the proposed methods on a variety of performance metrics such as hypervolume (HV), deviation in triangularized simplices, distance of the closest point pair, and variance of the geometric means to nearest neighbors in up to 15-D spaces. We show that an iterative improvement based on Riesz s-energy is able to effectively find an arbitrary number of well-spaced points even in higher-dimensional spaces. Reference points created using the proposed Riesz s-energy method for a number of standard combinations of objectives and reference points as well as a source code written in Python are available publicly at https://www.egr.msu.edu/coinlab/blankjul/uniform.

This article concerns the study of mixed Pareto-lexicographic multiobjective optimization problems where the objectives must be partitioned in multiple priority levels (PLs). A PL is a group of objectives having the same importance in terms of optimization and subsequent decision making, while between PLs a lexicographic ordering exists. A naive approach would be to define a multilevel dominance relationship and apply a standard EMO/EMaO algorithm, but the concept does not conform to a stable optimization process as the resulting dominance relationship violates the transitive property needed to achieve consistent comparisons. To overcome this, we present a novel approach that merges a custom nondominance relation with theGrossonemethodology, a mathematical framework to handle infinite and infinitesimal quantities. The proposed method is implemented on a popular multiobjective optimization algorithm (NSGA-II), deriving a generalization of it called by us PL-NSGA-II. We also demonstrate the usability of our strategy by quantitatively comparing the results obtained by PL-NSGA-II against other priority and nonpriority-based approaches. Among the test cases, we include two real-world applications: one 10-objective aircraft design problem and one 3-objective crash safety vehicle design task. The obtained results show that PL-NSGA-II is more suited to solve lexicographical many-objective problems than the general purpose EMaO algorithms.

Convolutional neural networks (CNNs) are the backbones of deep learning paradigms for numerous vision tasks. Early advancements in CNN architectures are primarily driven by human expertise and by elaborate design processes. Recently, neural architecture search was proposed with the aim of automating the network design process and generating task-dependent architectures. While existing approaches have achieved competitive performance in image classification, they are not well suited to problems where the computational budget is limited for two reasons: 1) the obtained architectures are either solely optimized for classification performance, or only for one deployment scenario and 2) the search process requires vast computational resources in most approaches. To overcome these limitations, we propose an evolutionary algorithm for searching neural architectures under multiple objectives, such as classification performance and floating point operations (FLOPs). The proposed method addresses the first shortcoming by populating a set of architectures to approximate the entire Pareto frontier through genetic operations that recombine and modify architectural components progressively. Our approach improves computational efficiency by carefully down-scaling the architectures during the search as well as reinforcing the patterns commonly shared among past successful architectures through Bayesian model learning. The integration of these two main contributions allows an efficient design of architectures that are competitive and in most cases outperform both manually and automatically designed architectures on benchmark image classification datasets: CIFAR, ImageNet, and human chest X-ray. The flexibility provided from simultaneously obtaining multiple architecture choices for different compute requirements further differentiates our approach from other methods in the literature.

Researchers have spent a considerable effort in evaluating the goodness of a solution set obtained by an evolutionary multi-objective algorithm. However, most performance metrics assume that the knowledge of the exact Pareto-optimal set is available. Also, most metrics evaluate an algorithm's performance based on the final solution set, which fails to capture their performance during intermediate generations. In this paper, we investigate a running performance metric which can be applied to measure the performance at any time during the algorithm execution and no true optimum needs to be known for computing the metric. In general, multi-objective algorithms either improve the convergence based on the dominance relation or the diversity in the solution set. Our proposed running metric makes use of this fact by keeping track of the indicators regarding the extreme points and the ND solution set each generation and derives measures of convergence and diversity. Moreover, by introducing a threshold and comparing the values of indicators a set of termination criteria is also suggested. Finally, we demonstrate how our running performance metric can be used to compare multiple evolutionary multi-objective algorithms with each other. An implementation of the proposed methodology is available at pymoo, a multi-objective optimization framework: https://pymoo.org.

Many design optimization problems from practice involve a large number of variables. In handling such problems, optimization algorithms, in general, suffer from the well-known ”curse of dimensionality” issue. One of the ways to alleviate the issue somewhat is to use problem information to update the optimization algorithm so that more meaningful solutions are evolved quickly. In this paper, we consider a solid rocket motor design problem involving hundreds of integer variables and two conflicting objectives - minimize the error in matching developed thrust with a desired time-dependent thrust profile and simultaneously minimize the unburnt residue of propellant at the end of the burning process. The evaluation of both objectives involve a detailed burn simulation from the core to the shell of the rocket. After finding a set of trade-off solutions using an evolutionary multi-objective optimization algorithm, we use two learning-based optimization methods (akin to the concept of innovization) to find similar set of solutions using a fraction of the overall solution evaluations. The proposed methods are applied to seven different thrust profiles. Besides solving the large-scale problem quicker, a by-product of our approach is that learnt innovized principles stay as new and innovative knowledge for solving the solid rocket design problem, a matter which is extremely useful to the practitioners.

Automated Innovization procedure aims to extract hidden, non-intuitive, closed-form relationships from a design task without human intervention. Existing procedures involve the application of an Evolutionary Multi-objective optimization (EMO) Algorithm in two phases. The first phase of EMO algorithm leads to a set of Pareto-optimal (PO) solutions, while the second phase helps identify the implicit relationships. The latter involves clustering which in turn enables the evaluation of innovization-driven objective function. The existing procedures for Automated Innovization differ in their clustering technique and objective formulation. Unlike any existing study, this paper proposes a Unified Automated Innovization (UAI) framework which can deal with both continuous and discrete variable problems, and identify the inherent single- or multiple-cluster rules, as the case may be. The scope and efficacy of the proposed UAI, demonstrated through some benchmark design problems, is rooted in the novel contributions made in the clustering technique, and innovization-driven objective function formulation(s).

Practical multi-criterion decision making not only involves the articulation of preferences in the objective space, but also a consideration of how the variables impact these preferences. Trend mining is a recently proposed visualization technique that offers the decision maker a quick overview of the variables' effect on the structure of the objective space and easily discover interesting variable trends. The original trend mining approach relies on a set of predefined reference directions along which an interestingness score is measured for each variable. In this paper, we relax this requirement by automating the approach to find optimal reference directions that maximize the interestingness for each variable. Additional extensions include the use of an Achievement Scalarizing Function (ASF) for ranking solutions along a given reference direction, and an updated interestingness score formulation for more appropriately handling discrete variables. We demonstrate the working of the extended approach on DTLZ2 and WFG2 benchmarks for up to five objectives and on a biobjective engineering design problem. The results show that the ability of the proposed approach to detect variable trends in high dimensional objective spaces is heavily dependent on the quality of the solutions used.

Convolutional neural networks have witnessed remarkable improvements in computational efficiency in recent years. A key driving force has been the idea of trading-off model expressivity and efficiency through a combination of 1x1 and depth-wise separable convolutions in lieu of a standard convolutional layer. The price of the efficiency, however, is the sub-optimal flow of information across space and channels in the network. To overcome this limitation, we present MUXConv, a layer that is designed to increase the flow of information by progressively multiplexing channel and spatial information in the network, while mitigating computational complexity. Furthermore, to demonstrate the effectiveness of MUXConv, we integrate it within an efficient multi-objective evolutionary algorithm to search for the optimal model hyper-parameters while simultaneously optimizing accuracy, compactness, and computational efficiency. On ImageNet, the resulting models, dubbed MUXNets, match the performance (75.3% top-1 accuracy) and multiply-add operations (218M) of MobileNetV3 while being 1.6x more compact, and outperform other mobile models in all the three criteria. MUXNet also performs well under transfer learning and when adapted to object detection. On the ChestX-Ray 14 benchmark, its accuracy is comparable to the state-of-the-art while being 3.3x more compact and 14x more efficient. Similarly, detection on PASCAL VOC 2007 is 1.2% more accurate, 28% faster and 6% more compact compared to MobileNetV2.

Sustainable forest management is a crucial element in combating climate change, plastic pollution, and other unsolved challenges of the 21st century. Forests not only produce wood - a renewable resource that is increasingly replacing fossil-based materials - but also preserve biodiversity and store massive amounts of carbon. Thus, a truly optimal forest policy has to balance profit-oriented logging with ecological and societal interests, and should thus be solved as a multi-objective optimization problem. Economic forest research, however, has largely focused on profit maximization. Recent publications still scalarize the problem a priori by assigning weights to objectives. In this paper, we formulate a multi-objective forest management problem where profit, carbon storage, and biodiversity are maximized. We obtain Pareto-efficient forest management strategies by utilizing three state-of-the-art Multi-Objective Evolutionary Algorithms (MOEAs), and by incorporating domain-specific knowledge through customized evolutionary operators. An analysis of Pareto-efficient strategies and their harvesting schedules in the design space clearly shows the benefits of the proposed approach. Unlike many EMO application studies, we demonstrate how a systematic post-optimality trade-off analysis can be applied to choose a single preferred solution. Our pioneering work on sustainable forest management explores an entirely new application area for MOEAs with great societal impact.

Over 30 years, evolutionary multi- and many-objective optimization (EMO/EMaO) algorithms have been extensively applied to find well-distributed Pareto-optimal (PO) solutions in a single run. However, in real-world problems, the PO front may not always be a single continuous hyper-surface, rather several irregularities may exist involving disjointed surfaces, holes within the surface, or patches of mixed-dimensional surfaces. When a set of trade-off solutions are obtained by EMO/EMaO algorithms, there may exist less dense or no solutions (we refer as 'gaps') in certain parts of the front. This can happen for at least two reasons: (i) gaps naturally exist in the PO front, or (ii) no natural gaps exists, but the chosen EMO/EMaO algorithm is not able to find any solution in the apparent gaps. To make a confident judgement, we propose a three-step procedure here. First, we suggest a computational procedure to identify gaps, if any, in the EMO/EMaO-obtained PO front. Second, we propose a computational method to identify well-distributed gap-points in the gap regions. Third, we apply a focused EMO/EMaO algorithm to search for possible representative trade-off points in the gaps. We then propose two metrics to qualitatively establish whether a gap truly exists in the obtained dataset, and if yes, whether the gap naturally exists on the true Pareto-set. Procedures are supported by results on two to five-objective test problems and on a five-objective scheduling problem from a steel-making industry.

Convolutional neural networks (CNNs) are the backbones of deep learning paradigms for numerous vision tasks. Early advancements in CNN architectures are primarily driven by human expertise and elaborate design. Recently, neural architecture search (NAS) was proposed with the aim of automating the network design process and generating task-dependent architectures. This paper introduces NSGA-Net -- an evolutionary search algorithm that explores a space of potential neural network architectures in three steps, namely, a population initialization step that is based on prior-knowledge from hand-crafted architectures, an exploration step comprising crossover and mutation of architectures, and finally an exploitation step that utilizes the hidden useful knowledge stored in the entire history of evaluated neural architectures in the form of a Bayesian Network. The integration of these components allows an efficient design of architectures that are competitive and in many cases outperform both manually and automatically designed architectures on CIFAR-10 classification task. The flexibility provided from simultaneously obtaining multiple architecture choices for different compute requirements further differentiates our approach from other methods in the literature.

Multiobjective evolutionary algorithms (MOEAs) have progressed significantly in recent decades, but most of them are designed to solve unconstrained multiobjective optimization problems. In fact, many real-world multiobjective problems contain a number of constraints. To promote research on constrained multiobjective optimization, we first propose a problem classification scheme with three primary types of difficulty, which reflect various types of challenges presented by real-world optimization problems, in order to characterize the constraint functions in constrained multiobjective optimization problems (CMOPs). These are feasibility-hardness, convergence-hardness, and diversity-hardness. We then develop a general toolkit to construct difficulty adjustable and scalable CMOPs (DAS-CMOPs, or DAS-CMaOPs when the number of objectives is greater than three) with three types of parameterized constraint functions developed to capture the three proposed types of difficulty. In fact, the combination of the three primary constraint functions with different parameters allows the construction of a large variety of CMOPs, with difficulty that can be defined by a triplet, with each of its parameters specifying the level of one of the types of primary difficulty. Furthermore, the number of objectives in this toolkit can be scaled beyond three. Based on this toolkit, we suggest nine difficulty adjustable and scalable CMOPs and nine CMaOPs, to be called DAS-CMOP1-9 and DAS-CMaOP1-9, respectively. To evaluate the proposed test problems, two popular CMOEAs-MOEA/D-CDP (MOEA/D with constraint dominance principle) and NSGA-II-CDP (NSGA-II with constraint dominance principle) and two popular constrained many-objective evolutionary algorithms (CMaOEAs)-C-MOEA/DD and C-NSGA-III-are used to compare performance on DAS-CMOP1-9 and DAS-CMaOP1-9 with a variety of difficulty triplets, respectively. The experimental results reveal that mechanisms in MOEA/D-CDP may be more effective in solving convergence-hard DAS-CMOPs, while mechanisms of NSGA-II-CDP may be more effective in solving DAS-CMOPs with simultaneous diversity-, feasibility-, and convergence-hardness. Mechanisms in C-NSGA-III may be more effective in solving feasibility-hard CMaOPs, while mechanisms of C-MOEA/DD may be more effective in solving CMaOPs with convergence-hardness. In addition, none of them can solve these problems efficiently, which stimulates us to continue to develop new CMOEAs and CMaOEAs to solve the suggested DAS-CMOPs and DAS-CMaOPs.

The ultimate goal of multiobjective optimization is to help a decision maker (DM) identify solution(s) of interest (SOI) achieving satisfactory tradeoffs among multiple conflicting criteria. This can be realized by leveraging DM's preference information in evolutionary multiobjective optimization (EMO). No consensus has been reached on the effectiveness brought by incorporating preference in EMO (either a priori or interactively) versus a posteriori decision making after a complete run of an EMO algorithm. Bearing this consideration in mind, this article: 1) provides a pragmatic overview of the existing developments of preference-based EMO (PBEMO) and 2) conducts a series of experiments to investigate the effectiveness brought by preference incorporation in EMO for approximating various SOI. In particular, the DM's preference information is elicited as a reference point, which represents her/his aspirations for different objectives. The experimental results demonstrate that preference incorporation in EMO does not always lead to a desirable approximation of SOI if the DM's preference information is not well utilized, nor does the DM elicit invalid preference information, which is not uncommon when encountering a black-box system. To a certain extent, this issue can be remedied through an interactive preference elicitation. Last but not the least, we find that a PBEMO algorithm is able to be generalized to approximate the whole PF given an appropriate setup of preference information.

Scientists' publication impacts ranking is an important topic in scientometrics which is performed based on various proposed criteria. One of the well-known indicators is h-index which evaluates researchers achievements based on number of citations. The h-index has utilized in many research data sources because of its appropriate properties, but similar to other assessment indicators, it has own disadvantages. hindex cannot give a fair comparison between junior and senior researches. There are two reasons for this unfair comparison: (1) h-index depends on the research period of scholars and (2) the number of received citations can be increased by time, even if researcher doesn't publish new papers, the h-index increases. Consequently, in addition to h-index, the number of the years of academic research (called the research period) is preferable to be considered as an independent indicator, which makes us able to have a more fair evaluation. So these two objectives, maximizing h-index and minimizing research period, can be considered as a multi-criteria comparison task to assess researchers. In this paper, we propose a strategy based on Pareto dominance ranking which uses dominance concept to obtain an order for researchers. In order to complete ranking between scientists in the same rank, a multi-criteria decision making measure called VIKOR is utilized. Therefore, a total ranking measure (P-V index) is obtained using Perto front concept and VIKOR measure. The proposed method is applied on 235 researchers who are conducting research on Evolutionary Computation (EC) topic. The h-index value and the research period of scholars are collected via Google Scholar service. P-V index obtains 26 Pareto ranks for all researchers and places six EC scientists on the first Pareto front.

In recent years, discovering relationships among entities and their features in a dataset has been received a great attention in data analytics. This study aims to reveal the relationships among entities in a dataset according to a specific sequence of features which are guided according to the accuracy of the hierarchical clustering made up by the features. In this paper, a new metric, called Discriminating Features based Cohesion (DFC) factor, is defined as pair-wise stickiness measure among entities which indicates their degree of attachment (i.e., cohesive force). In this direction, a new framework is proposed; which utilizes an evolutionary algorithm (i.e., DE) for the optimal discriminating feature selection and also a hierarchical clustering method for computing DFC factors. DE algorithm is employed to identify features which their clustering hierarchical tree has the maximum accuracy, then the intermediate and final DFC factors' matrices are computed by using a hierarchical clustering of the most discriminating features. The intermediate and final DFC factors' matrices have been utilized to discovery the knowledge among Dataset Entities including answering crucial data mining queries which cannot be answered by using a standalone clustering method. In order to conduct a case study, a real-world dataset is utilized; which contains 17 entities (i.e., countries) presented by corresponding 24 continuous features. The DE algorithm finds the most discriminating features in each step, which are eliminated for the next step to calculate a matrix of DFC factors. In the final step, the proposed method ranks the entities in terms of their DFC factor and features based on their elimination order (i.e., discrimination power).

This paper introduces NSGA-Net --- an evolutionary approach for neural architecture search (NAS). NSGA-Net is designed with three goals in mind: (1) a procedure considering multiple and conflicting objectives, (2) an efficient procedure balancing exploration and exploitation of the space of potential neural network architectures, and (3) a procedure finding a diverse set of trade-off network architectures achieved in a single run. NSGA-Net is a population-based search algorithm that explores a space of potential neural network architectures in three steps, namely, a population initialization step that is based on prior-knowledge from hand-crafted architectures, an exploration step comprising crossover and mutation of architectures, and finally an exploitation step that utilizes the hidden useful knowledge stored in the entire history of evaluated neural architectures in the form of a Bayesian Network. Experimental results suggest that combining the dual objectives of minimizing an error metric and computational complexity, as measured by FLOPs, allows NSGA-Net to find competitive neural architectures. Moreover, NSGA-Net achieves error rate on the CIFAR-10 dataset on par with other state-of-the-art NAS methods while using orders of magnitude less computational resources. These results are encouraging and shows the promise to further use of EC methods in various deep-learning paradigms.

Nondominated sorting is a key operation used in multiobjective evolutionary algorithms (MOEA). Worst case time complexity of this algorithm is O(MN2), where N is the number of solutions and M is the number of objectives. For stochastic algorithms like MOEAs, it is important to devise an algorithm which has better average case performance. In this paper, we propose a new algorithm that makes use of faster scalar sorting algorithm to perform nondominated sorting. It finds partial orders of each solution from all objectives and use these orders to skip unnecessary solution comparisons. We also propose a specific order of objectives that reduces objective comparisons. The proposed method introduces a weighted binary search over the fronts when the rank of a solution is determined. It further reduces total computational effort by a large factor when there is large number of fronts. We prove that the worst case complexity can be reduced to Θ(MNCmaxlog2(F + 1)), where the number of fronts is F and the maximum number of solutions per front is Cmax; however, in general cases, our worst case complexity is still O(MN2). Our best case time complexity is O(MNlogN). We also achieve the best case complexity O(MNlogN + N2), when all solutions are in a single front. This method is compared with other state-of-the-art algorithms-efficient nondomination level update, deductive sort, corner sort, efficient nondominated sort and divide-and-conquer sort-in four different datasets. Experimental results show that our method, namely, bounded best order sort, is computationally more efficient than all other competing algorithms.

One of the main difficulties in applying an optimization algorithm to a practical problem is that evaluation of objectives and constraints often involve computationally expensive procedures. To handle such problems, a metamodel is first formed from a few exact (high-fidelity) solution evaluations and then optimized by an algorithm in a progressive manner. However, in solving multiobjective or many-objective optimization problems involving multiple constraints, a simple extension of the idea to form one metamodel for each objective and constraint function may not constitute the most efficient approach. The cumulative effect of errors from each metamodel may turn out to be detrimental for the accuracy of the overall optimization procedure. In this paper, we propose a taxonomy of different plausible metamodeling frameworks for multiobjective and many-objective optimization and provide a comparative study by discussing advantages and disadvantages of each framework. The results presented in this paper are obtained using the well-known Kriging metamodeling approach. Based on our extensive simulation studies on proposed frameworks, we report intriguing observations about the behavior of each framework, which may provide salient guidelines for further studies in this emerging area within evolutionary multiobjective optimization.

Optimization problems with variable-length decision space are a class of challenging optimization problems derived from some real-world applications, such as the composite laminate stacking problem and the sensor coverage problem. Unlike other optimization problems, the solutions in these problems might be represented as the vectors with different variable size (i.e., dimensionality). So far, some research efforts have been done on the use of evolutionary algorithms (EAs) for solving single objective variable-length optimization problems. In fact, the variable-length problem difficulty can also exist in multiobjective optimization. However, such challenging problems have not yet gained much attention in the area of evolutionary multiobjective optimization. To facilitate the research on the variable-length Pareto optimization, we first suggest a systematic toolkit for constructing benchmark multiobjective test problems with variable-length feature in this paper. Then, we also propose a variable-length multiobjective EA based on a two-level decomposition strategy, which decomposes a multiobjective optimization problem in terms of the penalty boundary intersection search directions and the dimensionality of variables. The performance of our proposed algorithm and the other three state-of-the-art algorithms on these problems are compared. To further show the effectiveness of our proposed algorithm, some experimental results on a bi-objective laminate stacking optimization problem are also reported and analyzed.

In multiobjective optimization, defining a good solution is a multifactored process. Most existing evolutionary multi- or many-objective optimization (EMO) algorithms have utilized two factors: 1) domination and 2) crowding levels of each solution. Although these two coarse-grained factors are found to be adequate in many EMO algorithms, their relative importance in an algorithm has been a matter of great concern to many current studies. We argue that beside these issues, other more fine-grained factors are of importance. For example, since extreme objective-wise solutions are important in establishing a noise-free and stable normalization process, reaching extreme solutions is more crucial than finding other solutions. In this paper, we propose an integrated algorithm, B-NSGA-III, that produces much better convergence and diversity preservation. For this purpose, in addition to emphasizing extreme objective-wise solutions, B-NSGA-III tries to find solutions near intermediate undiscovered regions of the front. B-NSGA-III addresses critical algorithmic issues of convergence and diversity-preservation directly through recent progresses in literature and integrates all these critical fine-grained factors seamlessly in an alternating phases scheme. The proposed algorithm is shown to perform better than a number of commonly used existing methods.

A large number of application problems involve two levels of optimization, where one optimization task is nested inside the other. These problems are known as bilevel optimization problems and have been widely studied by researchers in the area of mathematical optimization. Bilevel optimization problems are known to be difficult and computationally demanding. Most of the solution procedures proposed until now are either computationally very expensive or applicable to only a narrow class of bilevel optimization problems involving small number of variables. In this paper, we propose a global optimization algorithm for bilevel optimization using Kriging approximation based model that tries to reduce the computational expense by iteratively approximating an important mapping in bilevel optimization; namely, the lower level optimal value function mapping. The lower level optimal value function is useful in reducing the two level optimization task to one; however, identifying this function is not straightforward. Our approach aims at meta-modeling this mapping and solving a number of auxiliary single level problems to arrive at the bilevel optimum. In our study, we test the methodology on a number of test problems. The preliminary results are quite promising which suggest the viability of the approach in solving more complicated bilevel test problem. To the best knowledge of the authors, such kind of a solution procedure based on iterative approximation of the optimal lower level value function using a stochastic process has not been widely used in bilevel optimization.

Real-world optimization problems often involve constraints that relate to viability of implementing a solution. To solve such problems efficiently, a good constraint handling method is indispensable for an optimization algorithm. Population-based optimization algorithms allow a flexible way to handle constraints by making a careful comparison between feasible and infeasible solutions present in the population. A previous approach, which emphasized feasible solutions infinitely more than the infeasible solutions, has been popularly applied for more than one-and-half decade, mostly with real-parameter genetic algorithms (RGAs). Despite its popular use, the idea was criticized for its extreme selection pressure against infeasible solutions. Since optimal solutions often lie on the constraint boundaries, survival of certain infeasible solutions close to critical constraint boundaries should help RGA's recombination and mutation operators to produce near-optimal solutions. In this paper, we extend the earlier parameter-less constraint handling approach so as to strike a balance between survival of feasible and infeasible solutions in a GA population. The balance is controlled through an additional parameter that could be pre-specified or adaptively updated as the algorithm progresses. A parametric study is conducted to determine an appropriate value which works the best on most problems of this study. A significant improvement in performance is observed for the commonly-used g-series test problem suite and a real-world application problem (welded beam design). The approach is generic and can be easily extended to other real-parameter evolutionary algorithms, multi-objective and other advanced optimization tasks.

Multi-objective optimization can be used to solve land-use allocation problems involving multiple conflicting objectives. In this paper, we show how genetic algorithms can be improved in order to effectively and efficiently solve multi-objective land-use allocation problems. Our focus lies on improving crossover and mutation operators of the genetic algorithms. We tested a range of different approaches either based on the literature or proposed for the first time. We applied them to a land-use allocation problem in Switzerland including two conflicting objectives: ensuring compact urban development and reducing the loss of agricultural productivity. We compared all approaches by calculating hypervolumes and by analysing the spread of the produced non-dominated fronts. Our results suggest that a combination of different mutation operators, of which at least one includes spatial heuristics, can help to find well-distributed fronts of non-dominated solutions. The tested modified crossover operators did not significantly improve the results. These findings provide a benchmark for multi-objective optimization of land-use allocation problems with promising prospectives for solving complex spatial planning problems.

Uncertainties in variables and parameters cause optimization problems to move away from globally-optimal and uncertain solutions. Practitioners resort to finding robust and reliable solutions in such situations. Bilevel optimization problems involving a hierarchy of two nested optimization problems have received a growing attention in the recent past due to their relevance in practice. While a number of studies on bilevel solution methodologies and applications are available for a deterministic setup, but studies on uncertainties in bilevel optimization are rare. In this paper, we suggest methodologies for handling uncertainty in both lower and upper level variables that may occur from different practicalities. For the first time, we perform a systematic study demonstrating the effect of uncertainties in each level along with the definition of robustness and reliability in the context of bilevel optimization. The issues and complexities introduced due to such uncertainties are then studied through a number of test cases, for brevity, we only show results on three test cases. Finally, two real-world bilevel problems involving uncertainties in their variables are solved. The study provides foundations and demon- strates viable directions for further research in uncertainty-based bilevel optimization problems.

This paper proposes a novel procedure for generating parametric scalable functions with diverse properties to strengthen numerical evaluation of niching methods. It combines three simple basic functions to form a composite multimodal function, in which the function parameter controls the number of global minima. The resultant composite function may show a variety of challenges in global optimization such as high condition, correlation, and presence of many undesirable local minima, as well as those peculiar to multimodal optimization, such as nonuniformly distributed global minima with dissimilar basin sizes. Moreover, the proposed procedure results in computationally inexpensive composite functions, when compared to those generated by available methods. This allows for benchmarking long-term success of niching methods on problems with many global minima. Six parametric benchmark functions are proposed in which the function parameter controls the number of global minima. Detailed analysis on distribution, basin shapes and sizes of the global minima is provided. One of the proposed test problems involve constraints, a matter which did not receive much attention in the past. These test functions are employed to compare some of the most successful niching methods in the literature. The numerical results disclose a drastic disparity between the performance of different niching methods on the composite functions and the existing test problems, demonstrating that the proposed composite functions simulate distinct challenges. Our findings highlight importance of the core search algorithm and challenge suitability of many niching strategies in more complicated landscapes and higher dimensions.

Solutions to most practical numerical optimization problems must be evaluated for their performance over a number of different loading or operating conditions, which we refer here as scenarios. Therefore, a meaningful and resilient optimal solution must be such that it remains feasible under all scenarios and performs close to an individual optimal solution corresponding to each scenario. Despite its practical importance, multiscenario consideration has received a lukewarm attention, particularly in the context of multiobjective optimization. The usual practice is to optimize for the worst-case scenario. In this paper, we review existing methodologies in this direction and set our goal to suggest a new and potential population-based method for handling multiple scenarios by defining scenario-wise domination principle and scenario-wise diversity-preserving operators. To evaluate, the proposed method is applied to a number of numerical test problems and engineering design problems with a detail explanation of the obtained results and compared with an existing method. This first systematic evolutionary-based multiscenario, multiobjective optimization study on numerical problems indicates that multiple scenarios can be handled in an integrated manner using an evolutionary multiobjective optimization framework to find a well-balanced compromise set of solutions to multiple scenarios and maintain a tradeoff among multiple objectives. In comparison to an existing serial multiple optimization approach, the proposed approach finds a set of compromised tradeoff solutions simultaneously. An achievement of multiobjective tradeoff and multiscenario tradeoff is algorithmically challenging, but due to its practical appeal, further research and application must be spent.

Measuring the performance of an algorithm for solving multiobjective optimization problem has always been challenging simply due to two conflicting goals, i.e., convergence and diversity of obtained tradeoff solutions. There are a number of metrics for evaluating the performance of a multiobjective optimizer that approximates the whole Pareto-optimal front. However, for evaluating the quality of a preferred subset of the whole front, the existing metrics are inadequate. In this paper, we suggest a systematic way to adapt the existing metrics to quantitatively evaluate the performance of a preference-based evolutionary multiobjective optimization algorithm using reference points. The basic idea is to preprocess the preferred solution set according to a multicriterion decision making approach before using a regular metric for performance assessment. Extensive experiments on several artificial scenarios, and benchmark problems fully demonstrate its effectiveness in evaluating the quality of different preferred solution sets with regard to various reference points supplied by a decision maker.

An effective allocation of search effort is important in multiobjective optimization, particularly in many-objective optimization problems (MaOPs). This paper presents a new adaptive search effort allocation strategy for multiobjective evolutionary algorithm based on decomposition MOEA/D-M2M, a recent MOEA/D algorithm for challenging MaOPs. This proposed method adaptively adjusts the subregions of its subproblems by detecting the importance of different objectives in an adaptive manner. More specifically, it periodically resets the subregion setting based on the distribution of the current solutions in the objective space such that the search effort is not wasted on unpromising regions. The basic idea is that the current population can be regarded as an approximation to the Pareto front (PF) and thus one can implicitly estimate the shape of the PF and such estimation can be used for adjusting the search focus. The performance of proposed algorithm has been verified by comparing it with eight representative and competitive algorithms on a set of degenerated MaOPs with disconnected and connected PFs. Performances of the proposed algorithm on a number of nondegenerated test instances with connected and disconnected PFs are also studied.

It is our pleasure to introduce this Special Issue on Search-Based Software Engineering (SBSE) focusing on the application of evolutionary computation to solve real-world software engineering problems. Evolutionary computation (EC) methods have now become integral part of software engineering. New advancements in EC, such as multi- and many-objective optimization, uncertainty handling for robust and reliable solutions, knowledge discovery and knowledge-augmented EC, dynamic EC, have a great deal of applications in software engineering. Many applications in software engineering have emerged based on the usage of EC for the automation of all phases of the software development process, including the analysis, design, implementation, testing, and maintenance of large software systems. A total of 26 papers were submitted to the Special Issue. Each was subjected to at least three reviews and finally six were accepted for publication as described in the following.

Bilevel optimization is defined as a mathematical program, where an optimization problem contains another optimization problem as a constraint. These problems have received significant attention from the mathematical programming community. Only limited work exists on bilevel problems using evolutionary computation techniques; however, recently there has been an increasing interest due to the proliferation of practical applications and the potential of evolutionary algorithms in tackling these problems. This paper provides a comprehensive review on bilevel optimization from the basic principles to solution strategies; both classical and evolutionary. A number of potential application problems are also discussed. To offer the readers insights on the prominent developments in the field of bilevel optimization, we have performed an automated text-analysis of an extended list of papers published on bilevel optimization to date. This paper should motivate evolutionary computation researchers to pay more attention to this practical yet challenging area.

Bilevel optimization problems are often reduced to single level using Karush-Kuhn-Tucker (KKT) conditions; however, there are some inherent difficulties when it comes to satisfying the KKT constraints strictly. In this paper, we discuss single level reduction of a bilevel problem using approximate KKT conditions which have been recently found to be more useful than the original and strict KKT conditions. We embed the recently proposed KKT proximity measure idea within an evolutionary algorithm to solve bilevel optimization problems. The idea is tested on a number of test problems and comparison results have been provided against a recently proposed evolutionary algorithm for bilevel optimization. The proposed idea leads to significant savings in lower level function evaluations and shows promise in further use of KKT proximity measures in bilevel optimization algorithm development.

The scanning strategy for selective laser melting - an additive manufacturing process - determines the temperature fields during the manufacturing process, which in turn affects residual stresses and distortions, two of the main sources of process-induced defects. The goal of this study is to develop a multi-objective approach to optimize the cellular scanning strategy such that the two aforementioned defects are minimized. The decision variable in the chosen problem is a combination of the sequence in which cells are processed and one of six scanning strategies applied to each cell. Thus, the problem is a combination of combinatorial and choice optimization, which makes the problem difficult to solve. On a process simulation domain consisting of 32 cells, our multi-objective evolutionary method is able to find a set of trade-off solutions for the defined conflicting objectives, which cannot be obtained by performing merely a local search. Possible similarities in Pareto-optimal solutions are explored.

Single objective evolutionary constrained optimization has been widely researched by plethora of researchers in the last two decades whereas multi-objective constraint handling using evolutionary algorithms has not been actively proposed. However, real-world multi-objective optimization problems consist of one or many non-linear and non-convex constraints. In the present work, we develop an evolutionary algorithm based on hybrid constraint handling methodology (HyCon) to deal with constraints in bi-objective optimization problems. HyCon is a combination of an Evolutionary Multi-objective Optimization (EMO) coupled with classical weighted sum approach and is an extended version of our previously developed constraint handling method for single objective optimization. A constrained bi-objective problem is converted into a tri-objective problem where the additional objective is formed using summation of constrained violation. The performance of HyCon is tested on four constrained bi-objective problems. The non-dominated solutions are compared with a standard evolutionary multi-objective optimization algorithm (NSGA-II) with respect to hypervolume and attainment surface. The simulation results illustrates the effectiveness of the HyCon method. The HyCon either outperformed or produced similar performance as compared to NSGA-II.

Innovization is a task of learning common principles that exist among some or all of the Pareto-optimal solutions of a multi-objective optimization problem. Except a few earlier studies, most innovization related studies were performed on the final non-dominated solutions found by an EMO algorithm. Recently, authors showed that these principles can be learned during an optimization run and simultaneously utilized in the same optimization run to repair variables to achieve a faster convergence to the Pareto-optimal set. Different principles learned during an optimization run can not only have different number of variables but, may also have variables that are common among a number of principles. Moreover, a preference order for repairing variables may play an important role for proper convergence. Thus, when multiple principles exist, it is important to use a strategy that is most beneficial for repairing evolving population of solutions. This paper makes a first attempt to assess and understand the effect of different strategies to make innovization-based repair of variables most useful. Based on results on test problems, the paper also makes useful suggestions, which require immediate further experimentation on more complex and real-world problems.

In the last three decades there have been a number of efficient multi-objective optimization algorithms capable of solving real-world problems. However, due to the complexity of most real-world problems (high-dimensionality of problems, computationally expensive, and unknown function properties) researchers and decision-makers are increasingly facing the challenge of selecting an optimization algorithm capable of solving their hard problems. In this paper, we propose a simple yet efficient hybridization of multi- and many-objective optimization algorithms framework called hybrid many-objective optimization algorithm using fusion of solutions obtained by several many-objective algorithms (fusion) to gain the combined benefits of several algorithms and reducing the challenge of choosing one optimization algorithm to solve complex problems. During the optimization process, the Fusion framework (1) executes all optimization algorithms in parallel, (2) it combines solutions of these algorithms and extracts well-distributed solutions using predefined structured reference points or user-defined reference points, and (3) adaptively selects best-performing algorithm to tackle the problem at different stages of the search process. A case study of the fusion framework by considering GDE3, SMPSO, and SPEA2 as multi-objective optimization algorithms is presented. Experimental results on five unconstrained and four constrained benchmark test problems with three to ten objectives show that the Fusion framework significantly outperforms all algorithms involved in the hybridization process as well as the NSGA-III algorithm in terms of diversity and convergence of obtained solutions. Furthermore, the proposed framework is consistently able to find accurate solutions for all test problems which can be interpreted as its high robustness characteristic.

The interest in multi-modal optimization methods is increasing in the recent years since many of real-world optimization problems have multiple/many optima and decision makers prefer to find all of them. Multiple global/local peaks create difficulties for optimization algorithms. In this context, niching is well-known and widely used technique for finding multiple solutions in multi-modal optimization. One commonly used niching technique in evolutionary algorithms is the Clearing method. However, canonical clearing scheme reduces the exploration capacity of the evolutionary algorithms. In this paper, Delaunay Triangulation based Clearing (DT-Clearing) procedure is proposed to handle multi-modal optimizations more efficiently while preserving simplicity of canonical clearing approach. In DT-Clearing, cleared individuals are reallocated in the biggest empty spaces formed within the search space which are determined through Delaunay Triangulation. The reallocation of cleared individuals discourages wasting of the resources and allows better exploration of the landscape. The algorithm also uses an external memory, an archive of the explored niches, thus preventing the redundant visiting of the individuals, henceforth finding more solutions in lesser number of generations. The method is tested using multi-modal benchmark problems proposed for the IEEE CEC 2013, Special Session on Niching Methods for Multimodal Optimization. Our method obtains promising results in comparison with the canonical clearing and demonstrates to be a competitive niching algorithm.

In recent years, research interests have been paid in solving real-world optimization problems with variable-length representation. For population-based optimization algorithms, the challenge lies in maintaining diversity in sizes of solutions and in designing a suitable recombination operator for achieving an adequate diversity. In dealing with multiple conflicting objectives associated with a variable-length problem, the resulting multiple trade-off Pareto-optimal solutions may inherently have different variable sizes. In such a scenario, the fixed recombination and mutation operators may not be able to maintain large-sized solutions, thereby not finding the entire Pareto-optimal set. In this paper, we first construct multiobjective test problems with variable-length structures, and then analyze the difficulties of the constructed test problems by comparing the performance of three state-of-the-art multiobjective evolutionary algorithms. Our preliminary experimental results show that MOEA/D-M2M shows good potential in solving the multiobjective test problems with variable-length structures due to its diversity strategy along different search directions. Our correlation analysis on the Pareto solutions with variable sizes in the Pareto front indicates that mating restriction is necessary in solving variable-length problem.

The importance of using heuristics in an optimization algorithm is well established, particularly in solving complex real-world problems. It is then expected that users know certain key problem information a priori and are able to implement the information in a suitable optimization algorithm. However, in many problems, such problem information may not be available before an optimization task is performed, thereby making the heuristics-based algorithms difficult to be implemented. In this paper, we suggest a `derived heuristics' based optimization methodology for this purpose. In such a method, past results from an optimization algorithm are utilized to derive problem heuristics and then used in a future applications to achieve a faster and more accurate optimization task. Heuristics can also be derived from the optimization run and used in subsequent iterations. In a particular gold processing plant optimization problem, we demonstrate the use of derived heuristics by developing a customized evolutionary optimization procedure which is capable of handling various complexities offered by the problem, in a way which is much better than a classical point-based method and a population-based generic approach. The results of this paper is motivating for evolutionary computation researchers to apply the methodology to other more complex real-world problems.

Real-world optimization problems are often subject to several constraints which are expensive to evaluate in terms of cost or time. Although a lot of effort is devoted to make use of surrogate models for expensive optimization tasks, not many strong surrogate-assisted algorithms can address the challenging constrained problems. Efficient Global Optimization (EGO) is a Kriging-based surrogate-assisted algorithm. It was originally proposed to address unconstrained problems and later was modified to solve constrained problems. However, these type of algorithms still suffer from several issues, mainly: (1) early stagnation, (2) problems with multiple active constraints and (3) frequent crashes. In this work, we introduce a new EGO-based algorithm which tries to overcome these common issues with Kriging optimization algorithms. We apply the proposed algorithm on problems with dimension d ≤ 4 from the G-function suite [16] and on an airfoil shape example.

Bilevel and multi-objective optimization methods are often useful to spatially target agri-environmental policy throughout a watershed. This type of problem is complex and is comprised of a number of practicalities: (i) a large number of decision variables, (ii) at least two inter-dependent levels of optimization between policy makers and policy followers, and (iii) uncertainty in decision variables and problem parameters. Given agricultural and economic data from the Raccoon watershed in central Iowa, we formulate a bilevel multi-objective optimization problem that accommodates objectives of both policy makers and farmers. The solution procedure then explicitly accounts for the nested nature of farm-level management decisions in response to agri-environmental policy incentives constructed by policy makers. We specifically examine the spatial targeting of a fertilizer-reduction incentive policy while seeking to maximize farm-level productivity while generating mandated water quality improvements using this framework. We test three different evolutionary optimization algorithms - m-BLEAQ, NSGA-II, and SPEA2 - and show that m-BLEAQ is well suited for handling the bilevel optimization problems and the considered practicalities.

Finding an optimal design for a truss structure involves optimizing its topology, size, and shape. A truss design problem is usually multimodal, meaning that the problem offers multiple optimal designs in terms of topology and/or size of the members, but they are evaluated to have similar or equally good objective function values. From a practical standpoint, it is desirable to find as many alternative designs as possible, rather than finding a single design, as often practiced. A few metaheuristics based methods with niching techniques have been used for finding multiple topologies for the truss design problem, but these studies have ignored any emphasis in finding multiple solutions in terms of size. To overcome this issue, this paper proposes to formulate the truss problem as a bilevel optimization problem, where stable topologies can be found in the upper level and the optimized sizes of the members of these topologies can be found in the lower level. As a result, a new bilevel niching method is proposed to find multiple optimal solutions for topology level as well as for the size level simultaneously. The proposed method is shown to be superior over the state-of-the-art methods on several benchmark truss-structure design problems.

Supply-chain management problems are common to most industries and they involve a hierarchy of subtasks, which must be coordinated well to arrive at an overall optimal solution. Such problems involve a hierarchy of decision-makers, each having its own objectives and constraints, but importantly requiring a coordination of their actions to make the overall supply chain process optimal from cost and quality considerations. In this paper, we consider a specific supply-chain management problem from a company, which involves two levels of coordination: (i) yearly strategic planning in which a decision on establishing an association of every destination point with a supply point must be made so as to minimize the yearly transportation cost, and (ii) weekly operational planning in which, given the association between a supply and a destination point, a decision on the preference of available transport carriers must be made for multiple objectives: minimization of transport cost and maximization of service quality and satisfaction of demand at each destination point. We propose a customized multi-objective bilevel evolutionary algorithm, which is computationally tractable. We then present results on state-level and ZIP-level accuracy (involving about 40,000 upper level variables) of destination points over the mainland USA. We compare our proposed method with current non-optimization based practices and report a considerable cost saving.

Most real-world optimization problems involve computationally expensive simulations for evaluating a solution. Despite significant progress in the use of metamodels for single-objective optimization, metamodeling methods have received a lukewarm attention for multi-objective optimization. A recent study classified various metamodeling approaches, of which one particular method is interesting, challenging, and novel. In this paper, we study this so-called M6 method in detail. In this approach, a selection operator's assignment function, as it is implemented in an evolutionary multi-objective optimization (EMO) algorithm, is directly metamodeled. Thus, this methodology requires only one selection function to be metamodeled irrespective of multitude of objective and constraint functions in a problem. However, the flip side of the methodology is that the resulting function is multimodal having a different optimum for every desired Pareto-optimal solution. We have used two different selection functions based on two recent ideas: (i) KKT proximity measure function and (ii) multimodal based evolutionary multi-objective (MEMO) selection function. The resulting meta-modeling methods are applied to a number of standard two and three-objective constraint and unconstrained test problems. Near Pareto-optimal solutions are found using only a fraction of high-fidelity solution evaluations compared to usual EMO applications.

During the recent decades, many niching methods have been proposed and empirically verified on some available test problems. They often rely on some particular assumptions associated with the distribution, shape, and size of the basins, which can seldom be made in practical optimization problems. This study utilizes several existing concepts and techniques, such as taboo points, normalized Mahalanobis distance, and the Ursem's hill-valley function in order to develop a new tool for multimodal optimization, which does not make any of these assumptions. In the proposed method, several subpopulations explore the search space in parallel. Offspring of a subpopulation are forced to maintain a sufficient distance to the center of fitter subpopulations and the previously identified basins, which are marked as taboo points. The taboo points repel the subpopulation to prevent convergence to the same basin. A strategy to update the repelling power of the taboo points is proposed to address the challenge of basins of dissimilar size. The local shape of a basin is also approximated by the distribution of the subpopulation members converging to that basin. The proposed niching strategy is incorporated into the covariance matrix self-adaptation evolution strategy (CMSA-ES), a potent global optimization method. The resultant method, called the covariance matrix self-adaptation with repelling subpopulations (RS-CMSA), is assessed and compared to several state-of-the-art niching methods on a standard test suite for multimodal optimization. An organized procedure for parameter setting is followed which assumes a rough estimation of the desired/expected number of minima available. Performance sensitivity to the accuracy of this estimation is also studied by introducing the concept of robust mean peak ratio. Based on the numerical results using the available and the introduced performance measures, RS-CMSA emerges as the most successful method when robustness and efficiency are considered at the same time.

Refactoring is widely recognized as a crucial technique applied when evolving object‐oriented software systems. If applied well, refactoring can improve different aspects of software quality including readability, maintainability, and extendibility. However, despite its importance and benefits, recent studies report that automated refactoring tools are underused much of the time by software developers. This paper introduces an automated approach for refactoring recommendation, called MORE, driven by 3 objectives: (1) to improve design quality (as defined by software quality metrics), (2) to fix code smells, and (3) to introduce design patterns. To this end, we adopt the recent nondominated sorting genetic algorithm, NSGA‐III, to find the best trade‐off between these 3 objectives. We evaluated the efficacy of our approach using a benchmark of 7 medium and large open‐source systems, 7 commonly occurring code smells (god class, feature envy, data class, spaghetti code, shotgun surgery, lazy class, and long parameter list), and 4 common design pattern types (visitor, factory method, singleton, and strategy). Our approach is empirically evaluated through a quantitative and qualitative study to compare it against 3 different state‐of‐the art approaches, 2 popular multiobjective search algorithms, and random search. The statistical analysis of the results confirms the efficacy of our approach in improving the quality of the studied systems while successfully fixing 84% of code smells and introducing an average of 6 design patterns. In addition, the qualitative evaluation shows that most of the suggested refactorings (an average of 69%) are considered by developers to be relevant and meaningful.

Nondominated sorting (NDS), which divides a population into several nondomination levels (NDLs), is a basic step in many evolutionary multiobjective optimization (EMO) algorithms. It has been widely studied in a generational evolution model, where the environmental selection is performed after generating a whole population of offspring. However, in a steady-state evolution model, where a population is updated right after the generation of a new candidate, the NDS can be extremely time consuming. This is especially severe when the number of objectives and population size become large. In this paper, we propose an efficient NDL update method to reduce the cost for maintaining the NDL structure in steady-state EMO. Instead of performing the NDS from scratch, our method only updates the NDLs of a limited number of solutions by extracting the knowledge from the current NDL structure. Notice that our NDL update method is performed twice at each iteration. One is after the reproduction, the other is after the environmental selection. Extensive experiments fully demonstrate that, comparing to the other five state-of-the-art NDS methods, our proposed method avoids a significant amount of unnecessary comparisons, not only in the synthetic data sets, but also in some real optimization scenarios. Last but not least, we find that our proposed method is also useful for the generational evolution model.

Multimodal optimization (MMO) aiming to locate multiple optimal (or near-optimal) solutions in a single simulation run has practical relevance to problem solving across many fields. Population-based meta-heuristics have been shown particularly effective in solving MMO problems, if equipped with specifically-designed diversity-preserving mechanisms, commonly known as niching methods. This paper provides an updated survey on niching methods. This paper first revisits the fundamental concepts about niching and its most representative schemes, then reviews the most recent development of niching methods, including novel and hybrid methods, performance measures, and benchmarks for their assessment. Furthermore, this paper surveys previous attempts at leveraging the capabilities of niching to facilitate various optimization tasks (e.g., multiobjective and dynamic optimization) and machine learning tasks (e.g., clustering, feature selection, and learning ensembles). A list of successful applications of niching methods to real-world problems is presented to demonstrate the capabilities of niching methods in providing solutions that are difficult for other optimization methods to offer. The significant practical value of niching methods is clearly exemplified through these applications. Finally, this paper poses challenges and research questions on niching that are yet to be appropriately addressed. Providing answers to these questions is crucial before we can bring more fruitful benefits of niching to real-world problem solving.

There are two main tasks involved in addressing a multiobjective optimization problem (MOP) by evolutionary multiobjective (EMO) algorithms: 1) make the population converge close to the Pareto-optimal front and 2) maintain adequate population diversity. However, most state-of-the-art EMO algorithms are designed based on the “convergence first and diversity second” principle. It has been observed that although these EMO algorithms have been successful in optimizing many real-world MOPs, they fail to solve certain problems that feature a severe imbalance between diversity preservation and achieving convergence. This paper characterizes an imbalanced MOP by clearly defining properties and indicating the reasons for the existing EMO algorithms' difficulties in solving them. We then present 14 imbalanced problems, with and without constraints. Computational results using four existing EMO algorithms-elitist non-dominated sorting genetic algorithm (NSGA-II), multiobjective evolutionary algorithm based on decomposition (MOEA/D), strength Pareto evolutionary algorithm 2 (SPEA2), and S metric selection EMO algorithm (SMS-EMOA) and a proposed generalized vector-evaluated genetic algorithm are then presented. It is seen that these EMO algorithms cannot solve these imbalanced problems, but they are able to solve the problems when augmented by multiobjective to multiobjective (M2M), an approach that decomposes the population into several interacting subpopulations. These results and the successful application of the EMO methods with the M2M approach even on standard so-called balanced problems indicate the usefulness of using the M2M approach.

Many real-world problems have more than one objective and are dynamic in nature, where either an objective function or constraint can vary over time. These problems are referred to as dynamic multi-objective optimisation problems (DMOOPs). A key challenge for dynamic multi-objective optimisation (DMOO) research is efficiently evaluating and analysing the performance of DMOO algorithms (DMOAs). This includes benchmarks, performance measures and the approach used to analyse the obtained results. Most research in recent years focussed on either dynamic single-objective or static multi-objective optimisation. In the field of DMOO, research focussed on unconstrained DMOOPs. A few papers have recently proposed constrained DMOOPs. Therefore, a key sub-challenge in DMOO is to have a standard benchmark suite that contains both unconstrained and constrained DMOOPs with various characteristics. In addition, the constraints used in the benchmarks should be guided by constraints that occur in real-world problems. Most approaches used to analyse the performance of DMOAs do not take into account how well a DMOA tracks the changing optimal solutions over time, i.e. how well it performs in each of the various environments. Furthermore, there are still certain DMOOPs that the proposed algorithms struggle to solve. Therefore, more research is required with regards to the development of algorithms that can solve DMOOPs efficiently. Another important aspect of DMOO is the decision making process that can either occur offline or interactively. This paper discusses these key challenges and progress that has been made to address these challenges. Furthermore, actions to deal with the outstanding issues are also proposed.

In many-objective optimization, visualization of true Pareto front or obtained non-dominated solutions is difficult. A proper visualization tool must be able to show the location, range, shape, and distribution of obtained non-dominated solutions. However, existing commonly used visualization tools in many-objective optimization (e.g., parallel coordinates) fail to show the shape of the Pareto front. In this paper, we propose a simple yet powerful visualization method, called 3-dimensional radial coordinate visualization (3D-RadVis). This method is capable of mapping M-dimensional objective space to a 3-dimensional radial coordinate plot while preserving the relative location of solutions, shape of the Pareto front, distribution of solutions, and convergence trend of an optimization process. Furthermore, 3D-RadVis can be used by decision-makers to visually navigate large many-objective solution sets, observe the evolution process, visualize the relative location of a solution, evaluate trade-off among objectives, and select preferred solutions. The visual effectiveness of the proposed method is demonstrated on widely used many-objective benchmark problems containing variety of Pareto fronts (linear, concave, convex, mixed, and disconnected). In addition, we demonstrated the capability of 3D-RadVis for visual progress tracking of the NSGA-III algorithm through generations. It is worthwhile to mention that a suitable visualization is a crucial prerequisite for an effective interactive optimization.

Evolutionary algorithms are the most studied and successful population-based algorithms for solving single- and multi-objective optimization problems. However, many studies have shown that these algorithms fail to perform well when handling many-objective (more than three objectives) problems due to the loss of selection pressure to pull the population towards the Pareto front. As a result, there has been a number of efforts towards developing evolutionary algorithms that can successfully handle many-objective optimization problems without deteriorating the effect of evolutionary operators. A reference-point based NSGA-II (NSGA-III) is one such algorithm designed to deal with many-objective problems, where the diversity of the solution is guided by a number of well-spread reference points. However, NSGA-III still has difficulty preserving elite population as new solutions are generated. In this paper, we propose an improved NSGA-III algorithm, called EliteNSGA-III to improve the diversity and accuracy of the NSGA-III algorithm. EliteNSGA-III algorithm maintains an elite population archive to preserve previously generated elite solutions that would probably be eliminated by NSGA-III's selection procedure. The proposed EliteNSGA-III algorithm is applied to II many-objective test problems with three to I5 objectives. Experimental results show that the proposed EliteNSGA-III algorithm outperforms the NSGA-III algorithm in terms of diversity and accuracy of the obtained solutions, especially for test problems with higher objectives.

When optimizing an multiobjective optimization problem, the evolution of population can be regarded as a approximation to the Pareto Front (PF). Motivated by this idea, we propose an adaptive region decomposition framework: MOEA/D-AM2M for the degenerated Many-Objective optimization problem (MaOP), where degenerated MaOP refers to the optimization problem with a degenerated PF in a subspace of the objective space. In this framework, a complex MaOP can be adaptively decomposed into a number of many-objective optimization subproblems, which is realized by the adaptively direction vectors design according to the present population's distribution. A new adaptive weight vectors design method based on this adaptive region decomposition is also proposed for selection in MOEA/D-AM2M. This strategy can timely adjust the regions and weights according to the population's tendency in the evolutionary process, which serves as a remedy for the inefficiency of fixed and evenly distributed weights when solving MaOP with a degenerated PF. Five degenerated MaOPs with disconnected PFs are generated to identify the effectiveness of proposed MOEA/D-AM2M. Contrast experiments are conducted by optimizing those MaOPs using MOEA/D-AM2M, MOEA/D-DE and MOEA/D-M2M. Simulation results have shown that the proposed MOEA/D-AM2M outperforms MOEA/D-DE and MOEA/D-M2M.

Cooperative Coevolution (CC) framework has become a powerful approach to solve large-scale global optimization problems effectively. Although a number of significant modifications of CC algorithms have been introduced in recent years, the theoretical studies of population initialization strategies in the CC framework are quite limited so far. The population initialization strategies can help a population-based algorithm to start with better candidate solutions for achieving better results. In this paper, we propose a CC algorithm with population initialization strategies based on the center region to improve its performance. Three population initialization strategies, namely, center-based normal distribution sampling, central golden region, and hybrid random-center normal distribution sampling are utilized in the CC framework. These population initialization strategies attempt to generate points around center-point with different schemes. The performance of the proposed algorithm is evaluated on CEC-2013 LSGO benchmark functions. Simulation results confirm that the proposed algorithm obtains a promising performance on the majority of the nonseparable high dimension benchmark functions.

Metamodel based evolutionary algorithms have been used for solving expensive single and multi-objective optimization problems where evaluation of functions consume major portion of the running time. The system can be complex, high dimensional, multi-objective and black box function. In this paper, we have proposed a framework for solving expensive multi-objective optimization problems that uses high dimensional model representation (HDMR) as a basic model. The proposed method first explores the region of interest and then exploits them by narrowing the search space. It uses Kriging to interpolate subcomponents of HDMR and NSGA-II to solve the model space. It is compared with basic NSGA-II and multi-objective Kriging method on ZDT, DTLZ and CEC09 test problem suits. The results show that this framework is able to find a good distribution of solutions which are sufficiently converged to Pareto optimal fronts with limited number of solution evaluations.

In simulation-based Evolutionary Multi-objective Optimization the number of simulation runs is very limited, since the complex simulation models require long execution times. With the help of preference information, the optimization result can be improved by guiding the optimization towards relevant areas in the objective space, for example with the R-NSGA-II algorithm [9], which uses a reference point specified by the decision maker. When stochastic systems are simulated, the uncertainty of the objective values might degrade the optimization performance. By sampling the solutions multiple times this uncertainty can be reduced. However, resampling methods reduce the overall number of evaluated solutions which potentially worsens the optimization result. In this article, a Dynamic Resampling strategy is proposed which identifies the solutions closest to the reference point which guides the population of the Evolutionary Algorithm. We apply a single-objective Ranking and Selection resampling algorithm in the selection step of R-NSGA-II, which considers the stochastic reference point distance and its variance to identify the best solutions. We propose and evaluate different ways to integrate the sampling allocation method into the Evolutionary Algorithm. On the one hand, the Dynamic Resampling algorithm is made adaptive to support the EA selection step, and it is customized to be used in the time-constrained optimization scenario. Furthermore, it is controlled by other resampling criteria, in the same way as other hybrid DR algorithms. On the other hand, R-NSGA-II is modified to rely more on the scalar reference point distance as fitness function. The results are evaluated on a benchmark problem with variable noise landscape.

Bilevel optimization is a nested optimization problem that contains one optimization task as a constraint to another optimization task. Owing to enormous applications that are bilevel in nature, these problems have received attention from mathematical programming as well as evolutionary optimization community. However, most of the available solution methods can either be applied to highly restrictive class of problems, or are highly computationally expensive that they do not scale for large scale bilevel problems. The difficulties in bilevel programming arise primarily from the nested structure of the problem. In this paper, we propose a metamodeling based solution strategy that attempts to iteratively approximate the optimal lower level value function. To the best knowledge of the authors, this kind of a strategy has not been used to solve bilevel optimization problems, particularly in the context of evolutionary computation. The proposed method has been evaluated on a number of test problems from the literature.

Optimal control is a task where it is desired to determine the inputs of a dynamical system that optimize (minimize or maximize) a specified cost functional, also known as performance index, while satisfying any constraints on behaviour of the system. As the name suggests, inverse optimal control is the opposite of the former one and thus is associated with mining of the cost functional, optimal behaviour of which fits the given results best. In this paper, we present the importance of evolutionary bilevel optimization techniques as a promising approach to solve inverse optimal control problems. Generally, inverse optimal control problems are found to be ill posed which makes them computationally expensive in addition to the associated redundancy with the solution. Inverse optimal control theory works as a stepping stone in figuring out the underlying optimality criteria in a given task. It has several other applications in areas like Markov's Decision Processes and Game Theory. In our work, we solve inverse optimal control problems to retrieve the original functional in optimal control task using metaheuristic based bilevel optimization techniques. The dataset comprising of state variables generated from an optimal control problem is utilized to mine the functional. In the later part of our paper, we formulate a problem of human motion transfer as a bilevel optimization task, and subsequently solve it using a bilevel algorithm.

Although surrogate models have been successfully adopted by evolutionary algorithms to solve time-consuming multiobjective problems, their use has been confined to solving problems with a low number of objectives. On the other hand, scalarizing functions have proved to work well with many-objective problems. This paper presents a novel study on many-objective optimization concerning the use of surrogate models to approximate both (1) the fitness landscape of traditional multiobjective approaches and (2) the ranking relation imposed by such approaches. Our methodology involves a thorough comparison of four popular surrogate modeling techniques in order to approximate the fitness landscape and the ranking relations of three different scalarizing functions. Additionally, we explored the interactions of these methods through four well-known scalable test problems with four, six, eight, and ten objectives. Besides finding that Tchebycheff scalarizing function and Gaussian processes for machine learning are accurate methods to handle many-objective problems, one of our most important findings involves the capabilities of metamodeling techniques to approximate the ranking procedure from the information gathered from the parameter space. Such a capability can be effectively used for pre-screening purposes on MOEAs.

A measure for estimating the convergence characteristics of a set of non-dominated points obtained by a multi-objective optimization algorithm was developed recently. The idea of the measure was developed based on the Karush-Kuhn-Tucker (KKT) optimality conditions which require the gradients of objective and constraint functions. In this paper, we extend the scope of the proposed KKT proximity measure by computing gradients numerically and evaluating the accuracy of the numerically computed KKT proximity measure with the same computed using the exact gradient computation. The results are encouraging and open up the possibility of using the proposed KKTPM to non-differentiable problems as well.

Despite three decades of intense studies of evolutionary computation (EC), researchers outside the EC community still have a general impression that EC methods are expensive and are not efficient in solving large-scale problems. In this paper, we consider a specific integer linear programming (ILP) problem which, although comes from a specific industry, is similar to many other practical resource allocation and assignment problems. Based on a population based evolutionary optimization framework, we develop a computationally fast method to arrive at a near-optimal solution repeatedly. Two popular softwares (glpk and CPLEX) are not able to handle around 300 and 2,000 integer variable version of the problem, respectively, even after running for several hours. Our proposed method is able to find a near-optimal solution in less than second on the same computer. Moreover, the main highlight of this study is that our method scales in a sub-quadratic computational complexity in handling 50,000 to one billion variables. We believe that this is the first time such a large-sized real-world constrained problem has ever been handled using any optimization algorithm. The study clearly demonstrates the reasons for such a fast and scale-up application of the proposed method. The work should remain as a successful case study of EC methods for years to come.

Surrogate models are effective in reducing the computational time required for solving optimization problems. However, there have been a lukewarm interest in finding multiple trade off solutions for multi-objective optimization problems using surrogate models. The literature on surrogate modeling for constrained optimization problems is also rare. The diffculty lies in the requirement of building and solving multiple surrogate models, one for each Pareto-optimal solution. In this paper, we first provide a brief introduction of the past studies and suggest a computationally fast, Kriging-based, and generative procedure for finding multiple near Pareto optimal solutions in a systematic manner. The expected improvement metric is maximized using a real-parameter genetic algorithm for finding new solutions for high-fidelity evaluations. The approach is computationally fast due to the interlinking of building multiple surrogate models and in its systematic sequencing methodology for assisting one model with another. In standard two and three-objective test problems with and without constraints, our proposed methodology takes only a few hundreds of high-fidelity solution evaluations to find a widely distributed near Pareto optimal solutions compared to the standard EMO methods requiring tens of thousands of high-fidelity solution evaluations. The framework is generic and can be extended to utilize other surrogate modeling methods easily.

Bilevel optimization problems are referred to as having a nested inner optimization problem as a constraint to a outer optimization problem in the domain of mathematical programming. It is also known as Stackelberg problems in game theory. In the recent past, bilevel optimization problems have received a growing attention because of its relevance in practice applications. However, the hierarchical structure makes these problems difficult to handle and they are commonly optimized with a deterministic setup. With presence of constrains, bilevel optimization problems are considered for finding reliable solutions which are subjected to a possess a minimum reliability requirement under decision variable uncertainties. Definition of reliable bilevel solution, the effect of lower and upper level uncertainties on reliable bilevel solution, development of efficient reliable bilevel evolutionary algorithm, and supporting simulation results on test and engineering design problems amply demonstrate their further use in other practical bilevel problems.

Set-based multiobjective optimization methods, such as evolutionary multiobjective optimization (EMO) methods, attempt to find a set of Pareto-optimal solutions, instead of a single optimal solution. To evaluate these algorithms for their convergence to the efficient set in multiobjective optimization problems, the current performance metrics require the knowledge of the true Pareto-optimal solutions. In this paper, we develop a theoretically motivated Karush-Kuhn-Tucker proximity measure (KKTPM) that can provide an estimate of the proximity of a set of tradeoff solutions from the true Pareto-optimal solutions without any prior knowledge. Besides theoretical development of the proposed metric, the proposed KKTPM is computed for iteration-wise tradeoff solutions obtained from specific EMO algorithms on two-, three-, five-, and ten-objective optimization problems. Results amply indicate the usefulness of the proposed KKTPM as a metric for evaluating different sets of tradeoff solutions and also as a possible termination criterion for an EMO algorithm. Other possible uses of the proposed metric are also highlighted.

Traditionally, evolutionary algorithms (EAs) have been systematically developed to solve mono-, multi-, and many-objective optimization problems, in this order. Despite some efforts in unifying different types of mono-objective evolutionary and non-EAs, researchers are not interested enough in unifying all three types of optimization problems together. Such a unified algorithm will allow users to work with a single software enabling one-time implementation of solution representation, operators, objectives, and constraints formulations across several objective dimensions. For the first time, we propose a unified evolutionary optimization algorithm for solving all three classes of problems specified above, based on the recently proposed elitist, guided nondominated sorting procedure, developed for solving many-objectives problems. Using a new niching-based selection procedure, our proposed unified algorithm automatically degenerates to an efficient equivalent population-based algorithm for each class. No extra parameters are needed. Extensive simulations are performed on unconstrained and constrained test problems having single-, two-, multi-, and many-objectives and on two engineering optimization design problems. Performance of the unified approach is compared to suitable population-based counterparts at each dimensional level. Results amply demonstrate the merit of our proposed unified approach and motivate similar studies for a richer understanding of the development of optimization algorithms.

Bilevel optimization problems are characterized by a hierarchical leader-follower structure, in which the leader desires to optimize her own strategy taking the response of the follower into account. These problems are referred to as Stackelberg problems in the domain of game theory, and as bilevel problems in the domain of mathematical programming. In a number of practical scenarios, a bilevel problem is solved by a leader who needs to take multiple objectives into account and simultaneously deal with the decision uncertainty involved in modeling the follower's behavior. Such problems are often encountered in strategic product design, homeland security applications, and taxation policy. However, the hierarchical nature makes the problem difficult to solve and they are commonly simplified by assuming a deterministic setup with smooth objective functions. In this paper, we focus our attention on the development of a flexible evolutionary algorithm for solving multicriterion bilevel problems with lower level (follower) decision uncertainty. The performance of the algorithm is evaluated in a comparative study on a number of test problems. In addition to the numerical experiments, we consider two real-world examples from the field of environmental economics and management to illustrate how the framework can be used to obtain optimal strategies.

One of the most widely used techniques to improve the quality of existing software systems is refactoring—the process of improving the design of existing code by changing its internal structure without altering its external behavior. While it is important to suggest refactorings that improve the quality and structure of the system, many other criteria are also important to consider, such as reducing the number of code changes, preserving the semantics of the software design and not only its behavior, and maintaining consistency with the previously applied refactorings. In this article, we propose a multi-objective search-based approach for automating the recommendation of refactorings. The process aims at finding the optimal sequence of refactorings that (i) improves the quality by minimizing the number of design defects, (ii) minimizes code changes required to fix those defects, (iii) preserves design semantics, and (iv) maximizes the consistency with the previously code changes. We evaluated the efficiency of our approach using a benchmark of six open-source systems, 11 different types of refactorings (move method, move field, pull up method, pull up field, push down method, push down field, inline class, move class, extract class, extract method, and extract interface) and six commonly occurring design defect types (blob, spaghetti code, functional decomposition, data class, shotgun surgery, and feature envy) through an empirical study conducted with experts. In addition, we performed an industrial validation of our technique, with 10 software engineers, on a large project provided by our industrial partner. We found that the proposed refactorings succeed in preserving the design coherence of the code, with an acceptable level of code change score while reusing knowledge from recorded refactorings applied in the past to similar contexts.

An artificial lateral line consists of a set of flow sensors arranged around a fish-like body which aims at localizing the surrounding moving objects, a common example of which is a vibrating sphere, called a dipole. The presence of diverse sources of uncertainty in the flow environment and flow sensors leads to an error in localization and thus challenges practicability of the underlying idea, especially considering that localization accuracy significantly declines when uncertainties intensify. Accuracy of localization depends on selection of the parameters of the artificial lateral line including the number and the location of the sensors as well as the shape and the size of the lateral line. In this study, different sources of uncertainties are identified and modeled in the problem formulation. A parametric fitness function is defined that addresses computational and practical goals and encompasses the effect of different sources of uncertainties. A bi-level optimization tool is formed to find the optimum artificial lateral. Comparison of the optimized designs in different cases reveals that, the optimized design highly depends on the amount of uncertainties in the problem as well as the number of available sensors. The proposed methodology for handling noisy and nested optimization task can be extended to solve other similar problems.

As the computational complexity of the problem and/or the number of objectives increases, a large population has to be evaluated at each generation of algorithm, and this process needs more computational resources, or requires more time for the same computational resource. However, distributing the tasks into different processors (or cores) is a good solution in speeding up the process overall. In this study, a novel and pragmatic distributed computing approach for multiobjective evolutionary optimization algorithm is proposed. Instead of dividing the objective space into pre-defined cone-domination principles, as proposed in an earlier study, a distribution of reference points initialized on a hyper-plane spanning the entire objective space is assigned to different processors and the R-NSGA-II procedure is invoked to find respective partial efficient fronts. Our results show that the proposed distributed computing approach reduces the overall computational effort compared to that needed with a single-processor method.

Ship design is a complex engineering activity which requires a multidisciplinary consideration in arriving at design objectives and constraints. An optimal design of such problems require a multi-objective optimization method that is capable of finding multiple trade-off solutions, not only to choose a preferred solution for implementation, but also to have a deeper understanding of the interactions among design variables. In this paper, we consider two ship design models involving uncertainties in design variables, and demonstrate the usefulness of an evolutionary multiobjective optimization (EMO) method and subsequent data analysis procedures in arriving at valuable design principles that enhance the knowledge of a designer. The study is pedagogical yet provide key insights of ship design issues and importantly outlines the systematic procedure for applying the technology to other more complex design problems.

Most designs in practice go through a number of different loading or operating conditions. Therefore, a meaningful and resilient design must be such that it performs well under all such scenarios. Despite its practical importance, multi-scenario consideration has not been paid much attention in multi-objective optimization literature. In this paper, we address this challenging issue by suggesting an aggregate based handling of multiple scenarios and contrasts the proposed approach against a recently suggested approach which involves running multi-objective optimization multiple times and a rigid decision-making method. The proposed method is applied to two numerical test problems and two engineering design problems. This first evolutionary based multi-scenario, multi-objective optimization study should spur further interests among EMO researchers.

Following manual observation of hidden relationships present in Pareto-optimal (PO) solutions of a multi-objective optimization problem, an automated Innovization procedure was suggested earlier for extracting innovative design principles.The goal was to obtain closed form and simple to understand relations that exist among PO solutions in a design or other problems.The proposed automated Innovization method was developed for handling continuous variable spaces.Since, most practical design problems have discrete variables in their descriptions, the aim of this study is to extend the earlier automated Innovization procedure to handle discrete variable spaces.We discuss the difficulties posed to an automated procedure due to the search space granularity and demonstrate the working of our proposed method on one numerical problem and two engineering design problems.Our study amply demonstrates that the extension of a real-parameter automated Innovization is not straightforward to discrete spaces, however such a procedure for discrete spaces raises new challenges which must be addressed for handling problems with mixed continuous-discrete search space problems.

Achieving balance between convergence and diversity is a basic issue in evolutionary multiobjective optimization (EMO). In this paper, we propose a hybrid EMO algorithm that assigns different selection principles to two separate and co-evolving archives. Particularly, one archive maintains a repository with a competitive selection pressure towards the Pareto-optimal front (PF), the other preserves a population with a satisfied distribution in the objective space. Furthermore, to exploit guidance information towards the Pareto-optimal set (PS), we develop a restricted mating selection mechanism to select mating parents from each archive for offspring generation. Empirical studies are conducted on a set of benchmark problems with complicated PSs. Experimental results demonstrate the effectiveness and competitiveness of our proposed algorithm in balancing convergence and diversity.

Bilevel optimization problems have received a growing attention in the recent past. In this paper, we suggest methodologies for handling uncertainty in both lower and upper level decision variables that may occur from different practicalities. For the first time, we discuss and demonstrate the effect of uncertainties in each level on the overall definition of a robust bilevel solution and present simulation results on a number of test problems. Finally, the robust solutions of a bilevel circuit design problem are found using a previously suggested fast bilevel evolutionary algorithm (BLEAQ). Definition of robust bilevel solutions, effect of lower and upper level uncertainties in robust bilevel solutions, development of a robust bilevel evolutionary algorithm and simulation results on test and engineering design problems are contributions of this study.

MOEA/D is an evolutionary multi-objective optimization algorithm, which relies on decomposition methods such as, weighted-sum, Tchebycheff and Penalty-based Boundary Intersection (PBI) to convert a multi-objective problem into a set of single-objective problems. It is known that PBI can generate a more uniform set of solutions than the other decomposition methods. The drawback of PBI is that it has a penalty parameter (θ) that has to be specified by the user. This penalty parameter can affect the convergence rate of MOEA/D as well as the uniformity of solutions. Unfortunately, there are very limited studies on sensitivity analysis of MOEA/D on the penalty parameter of PBI. This paper is dedicated to a comprehensive analysis of PBI's penalty parameter, and its effect on a user-preference algorithm (R-MEAD2) and a non-user-preference algorithm (MOEA/D). Unlike the previous studies that only rely on Hypervolume as their performance measure, we study the effect of θ on convergence, uniformity, and the combination of convergence and uniformity independently. The experimental results suggest that user-preference algorithms consistently perform better with a relatively larger θ value as compared to their non-user-preference counterparts. The results also suggest that on some problems, such as multi-modal functions, convergence is the dominant factor on the overall performance, where a smaller θ is preferable. Conversely, on some other problems, a larger θ is suggested where uniformity is the dominant factor. Finally, we briefly investigate the relationship between θ and the number of objectives.

Decomposition-based elitist non-dominated sorting genetic algorithm (NSGA-III) is a recently proposed many-objective optimization algorithm that uses multiple pre-defined yet adaptable reference directions to maintain diversity among its solutions. Designing to solve specifically many-objective problems having four or more objectives, the authors of NSGA-III restricted the population size to be equal to the number of chosen reference directions. This restriction hinders the usage of NSGA-III to single-objective optimization problems, where, by definition, there is only one reference direction. For this reason, a unified algorithm - U-NSGA-III - has been recently proposed to handle this issue. U-NSGA-III is capable of adapting automatically to the dimensionality of the problem in hand through its niching based selection operator. However, the authors of U-NSGA-III abided by this single-fold restriction in all NSGA-III simulations of their study. In this paper we test the possibility of ignoring this restriction of NSGA-III and use multiple population folds to solve single, multi and many-objective problems. Simulations are performed on a variety of constrained and unconstrained single, multi and many-objective problems for this purpose. The strengths and weaknesses of multi-fold NSGA-III compared to those of U-NSGA-III are thoroughly investigated here. The robustness of NSGA-III in each type of problems is also discussed. This study provides a more comprehensive evaluation of the original NSGA-III procedure, which seems to have a wider scope than the original study had foreseen.

In this paper, we consider multi-objective bilevel optimization problems in the context of transportation policy formulation. In such problems, an authority managing a network of roads is the leader that tries to solve the problem by taking into account the possible actions of the network users who are considered as the followers. In the presence of multiple objectives, the resulting solution set is a Pareto-optimal frontier that consists of optimal decisions of the leader and corresponding optimal responses from the follower. The authority's objectives are to maximize its revenues through tolls and minimize the pollution levels. The network users' objectives are to minimize travel cost and travel time. In addition to accommodating multiple objectives at both levels, the benefits of the proposed formulation is that it allows incorporating various real-world complexities, like admitting complex road network topologies and allowing the modelling of several road user classes with different preferences. A recently proposed algorithm for multi-objective bilevel optimization is used to solve the problem.

Most of the practical optimization problems involve variables and parameters that are not reliable and often vary around their nominal values. If the optimization problem is solved at the nominal values without taking the uncertainty into account, it can lead to severe operational implications. In order to avoid consequences that can be detrimental for the system, one resorts to the robust optimization paradigm that attempts to optimize the “worst case” solution arising as a result of perturbations. In this paper, we propose an evolutionary algorithm for robust optimization of unconstrained problems involving uncertainty. The algorithm utilizes a novel crossover operator that identifies a cone-based descent region to produce the offspring. This leads to a large saving in function evaluations, but still guarantees convergence on difficult multimodal problems. A number of test cases are constructed to evaluate the proposed algorithm and comparisons are drawn against two benchmark cases.

Retaining water at the root level of crops has been a major focus in precision irrigation system from technological, societal, and environmental points of view. Subsurface water retention technology (SWRT) through impermeable membranes placed at certain depths under has shown 1.4 to 3.4-fold increase in production in crops. However, the sizing, placement and surface water scheduling are important parameters for achieving an optimal yield. In this paper, for the first time, we consider a water flow simulation model through soil (HYDRUS-2D) which is integrated with an evolutionary multi-objective optimization (EMO) algorithm to find optimal membrane configurations and surface water supply under multiple conflicting objectives. The initial results presented in this paper clearly demonstrate the merit of such a collaboration and supports further studies. Not only the integrated approach finds optimal membrane configurations and water supply, but also reveals a number of useful insights and knowledge about optimal precision irrigation, a matter which has long term contributions to water conservation, ground-water preservation, and many natural resources directly affecting modern society.

The process of writing model transformations is a complex and error-prone one. Thus, efficient techniques and tools for validating model transformations are needed. One of them is model transformation testing. The generation of test cases for model transformations is mainly based on metamodel and rules coverage criteria. In this paper, we propose to treat model transformation testing as a bi-level optimization problem to combine the generation of test cases with mutation testing. In our adaptation, the upper-level problem generates a set of test cases that maximizes the coverage of metamodels and errors introduced by the lower level to the transformation rules. The lower level maximizes the number of generated errors in the rules that cannot be detected by the test cases produced by the upper level. The main advantage of our bi-level formulation is that the evaluation of test cases is not limited to the coverage of metamodels, but it allows evaluating their ability to detect errors. The statistical analysis of our experiments on different transformation mechanisms confirms the outperformance of our bi-level proposal compared with state-of-the-art model transformation testing techniques. Copyright © 2015 John Wiley & Sons, Ltd.

Multiobjective evolutionary algorithm based on decomposition (MOEA/D), which bridges the traditional optimization techniques and population-based methods, has become an increasingly popular framework for evolutionary multiobjective optimization. It decomposes a multiobjective optimization problem (MOP) into a number of optimization subproblems. Each subproblem is handled by an agent in a collaborative manner. The selection of MOEA/D is a process of choosing solutions by agents. In particular, each agent has two requirements on its selected solution: one is the convergence toward the efficient front, the other is the distinction with the other agents' choices. This paper suggests addressing these two requirements by defining mutual-preferences between subproblems and solutions. Afterwards, a simple yet effective method is proposed to build an interrelationship between subproblems and solutions, based on their mutual-preferences. At each generation, this interrelationship is used as a guideline to select the elite solutions to survive as the next parents. By considering the mutual-preferences between subproblems and solutions (i.e., the two requirements of each agent), the selection operator is able to balance the convergence and diversity of the search process. Comprehensive experiments are conducted on several MOP test instances with complicated Pareto sets. Empirical results demonstrate the effectiveness and competitiveness of our proposed algorithm.

Achieving balance between convergence and diversity is a key issue in evolutionary multiobjective optimization. Most existing methodologies, which have demonstrated their niche on various practical problems involving two and three objectives, face significant challenges in many-objective optimization. This paper suggests a unified paradigm, which combines dominance- and decomposition-based approaches, for many-objective optimization. Our major purpose is to exploit the merits of both dominance- and decomposition-based approaches to balance the convergence and diversity of the evolutionary process. The performance of our proposed method is validated and compared with four state-of-the-art algorithms on a number of unconstrained benchmark problems with up to 15 objectives. Empirical results fully demonstrate the superiority of our proposed method on all considered test instances. In addition, we extend this method to solve constrained problems having a large number of objectives. Compared to two other recently proposed constrained optimizers, our proposed method shows highly competitive performance on all the constrained optimization problems.

Software systems nowadays are complex and difficult to maintain due to continuous changes and bad design choices. To handle the complexity of systems, software products are, in general, decomposed in terms of packages/modules containing classes that are dependent. However, it is challenging to automatically remodularize systems to improve their maintainability. The majority of existing remodularization work mainly satisfy one objective which is improving the structure of packages by optimizing coupling and cohesion. In addition, most of existing studies are limited to only few operation types such as move class and split packages. Many other objectives, such as the design semantics, reducing the number of changes and maximizing the consistency with development change history, are important to improve the quality of the software by remodularizing it. In this article, we propose a novel many-objective search-based approach using NSGA-III. The process aims at finding the optimal remodularization solutions that improve the structure of packages, minimize the number of changes, preserve semantics coherence, and reuse the history of changes. We evaluate the efficiency of our approach using four different open-source systems and one automotive industry project, provided by our industrial partner, through a quantitative and qualitative study conducted with software engineers.

Multi-objective evolutionary algorithms (MOEAs) are often criticized for their high-computational costs. This becomes especially relevant in simulation-based optimization where the objectives lack a closed form and are expensive to evaluate. Over the years, meta-modeling or surrogate modeling techniques have been used to build inexpensive approximations of the objective functions which reduce the overall number of function evaluations (simulations). Some recent studies however, have pointed out that accurate models of the objective functions may not be required at all since evolutionary algorithms only rely on the relative ranking of candidate solutions. Extending this notion to MOEAs, algorithms which can ‘learn’ Pareto-dominance relations can be used to compare candidate solutions under multiple objectives. With this goal in mind, in this paper, we study the performance of ten different off-the-shelf classification algorithms for learning Pareto-dominance relations in the ZDT test suite of benchmark problems. We consider prediction accuracy and training time as performance measures with respect to dimensionality and skewness of the training data. Being a preliminary study, this paper does not include results of integrating the classifiers into the search process of MOEAs.

Genetic algorithms have been used as an optimization tool using evolutionary strategies. Genetic algorithms cover three basic steps for population refinement selection, cross-over and mutation. In normal Real-coded genetic algorithm(RGA), the population of real variables generated after population refinement operations, is used for the computation of the objective function. In this paper we have shown the effect made by mapping the refined population towards better solutions and thereby creating more biased search. The mapping used is non-uniform in nature and is the function of the position of the individual w.r.t. the best solution obtained so far in the algorithm, and hence the name Non-Uniform RGA or in short NRGA. Tests were performed on standard benchmark problems. The results were promising and should encourage further research in this dimension.

Most network optimization problems are studied under a static scenario in which connectivity of the network and weights associated with the links of the networks are assumed to be fixed. However, in practice, they are likely to change with time and if the network is to be used over time under dynamic conditions, they need to be re-optimized as soon as there is a change. Since optimization process requires some finite time, there is a need for a efficient dynamic optimization strategy for solving such problems. In this study, we extend a previously proposed “Frozen-time” algorithm to network optimization by which new and optimized networks can be obtained in a computationally fast manner. We propose three different variations of the optimization strategies and show proof-of-principle simulation results on a 20-node network having 190 different source-destination paths. The results are interesting and suggest a viable further research.

Evolutionary algorithms that rely on dominance ranking often suffer from a low selection pressure problem when dealing with many-objective problems. Decomposition and user-preference based methods can help to alleviate this problem to a great extent. In this paper, a user-preference based evolutionary multi-objective algorithm is proposed that uses decomposition methods for solving many-objective problems. Decomposition techniques that are widely used in multi-objective evolutionary optimization require a set of evenly distributed weight vectors to generate a diverse set of solutions on the Pareto-optimal front. The newly proposed algorithm, R-MEAD2, improves the scalability of its previous version, R-MEAD, which uses a simplexlattice design method for generating weight vectors. This makes the population size is dependent on the dimension size of the objective space. R-MEAD2 uses a uniform random number generator to remove the coupling between dimension and the population size. This paper shows that a uniform random number generator is simple and able to generate evenly distributed points in a high dimensional space. Our comparative study shows that R-MEAD2 outperforms the dominance-based method R-NSGA-II on many-objective problems.

For real-world problems, the task of decision-makers is to identify a solution that can satisfy a set of performance criteria, which are often in conflict with each other. Multi-objective evolutionary algorithms tend to focus on obtaining a family of solutions that represent the trade-offs between the criteria; however ultimately a single solution must be selected. This need has driven a requirement to incorporate decision-maker preference models into such algorithms - a technique that is very common in the wider field of multiple criteria decision making. This paper reviews techniques which have combined evolutionary multi-objective optimization and multiple criteria decision making. Three classes of hybrid techniques are presented: a posteriori, a priori, and interactive, including methods used to model the decision-makers preferences and example algorithms for each category. To encourage future research directions, a commentary on the remaining issues within this research area is also provided.

In this paper, we provide an improved evolutionary algorithm for bilevel optimization. It is an extension of a recently proposed Bilevel Evolutionary Algorithm based on Quadratic Approximations (BLEAQ). Bilevel optimization problems are known to be difficult and computationally demanding. The recently proposed BLEAQ approach has been able to bring down the computational expense significantly as compared to the contemporary approaches. The strategy proposed in this paper further improves the algorithm by incorporating archiving and local search. Archiving is used to store the feasible members produced during the course of the algorithm that provide a larger pool of members for better quadratic approximations of optimal lower level solutions. Frequent local searches at upper level supported by the quadratic approximations help in faster convergence of the algorithm. The improved results have been demonstrated on two different sets of test problems, and comparison results against the contemporary approaches are also provided.

In this paper, we address solution methodologies of an optimization problem under multiple scenarios. Often in practice, a problem needs to be considered for different scenarios, such as evaluating for different loading conditions, different blocks of data, multi-stage operations, etc. After reviewing various single-objective aggregate methods for handling objectives and constraints under multiple scenarios, we then suggest a multi-objective optimization approach for solving multi-scenario optimization problems. On a Byzantine agreement problem, we demonstrate the usefulness of the proposed multi-objective approach and explain the reasons for their superior behavior. The suggested procedure is generic and now awaits further applications to more challenging problems from engineering and computational fields.

There is a growing need for scalable search-based software engineering approaches that address software engineering problems where a large number of objectives are to be optimized. Software refactoring is one of these problems where a refactoring sequence is sought that optimizes several software metrics. Most of the existing refactoring work uses a large set of quality metrics to evaluate the software design after applying refactoring operations, but current search-based software engineering approaches are limited to using a maximum of five metrics. We propose for the first time a scalable search-based software engineering approach based on a newly proposed evolutionary optimization method NSGA-III where there are 15 different objectives to be optimized. In our approach, automated refactoring solutions are evaluated using a set of 15 distinct quality metrics. We evaluated this approach on seven large open source systems and found that, on average, more than 92% of code smells were corrected. Statistical analysis of our experiments over 31 runs shows that NSGA-III performed significantly better than two other many-objective techniques (IBEA and MOEA/D), a multi-objective algorithm (NSGA-II) and two mono-objective approaches, hence demonstrating that our NSGA-III approach represents the new state of the art in fully-automated refactoring.

Many of the modern optimization algorithms contain a number of parameters that require tuning before the algorithm can be applied to a particular class of optimization problems. A proper choice of parameters may have a substantial effect on the accuracy and efficiency of the algorithm. Until recently, parameter tuning has mostly been performed using brute force strategies, such as grid search and random search. Guesses and insights about the algorithm are also used to find suitable parameters or suggest strategies to adjust them. More recent trends include the use of meta-optimization techniques. Most of these approaches are computationally expensive and do not scale when the number of parameters increases. In this paper, we propose that the parameter tuning problem is inherently a bilevel programming problem. Based on this insight, we introduce an evolutionary bilevel algorithm for parameter tuning. A few commonly used optimization algorithms (Differential Evolution and Nelder-Mead) have been chosen as test cases, whose parameters are tuned on a number of standard test problems. The bilevel approach is found to quickly converge towards the region of efficient parameters. The code for the proposed algorithm can be accessed from the website http://bilevel.org.

We propose a novel recommendation tool for software refactoring that dynamically adapts and suggests refactorings to developers interactively based on their feedback and introduced code changes. Our approach starts by finding upfront a set of non-dominated refactoring solutions using NSGA-II to improve software quality, reduce the number of refactorings and increase semantic coherence. The generated non-dominated refactoring solutions are analyzed using our innovization component to extract some interesting common features between them. Based on this analysis, the suggested refactorings are ranked and suggested to the developer one by one. The developer can approve, modify or reject each suggested refactoring, and this feedback is used to update the ranking of the suggested refactorings. After a number of introduced code changes, a local search is performed to update and adapt the set of refactoring solutions suggested by NSGA-II. We evaluated this tool on four large open source systems and one industrial project provided by our partner. Statistical analysis of our experiments over 31 runs shows that the dynamic refactoring approach performed significantly better than three other search-based refactoring techniques, manual refactorings, and one refactoring tool not based on heuristic search.

In this paper, we propose a procedure for designing controlled test problems for single-objective bilevel optimization. The construction procedure is flexible and allows its user to control the different complexities that are to be included in the test problems independently of each other. In addition to properties that control the difficulty in convergence, the procedure also allows the user to introduce difficulties caused by interaction of the two levels. As a companion to the test problem construction framework, the paper presents a standard test suite of 12 problems, which includes eight unconstrained and four constrained problems. Most of the problems are scalable in terms of variables and constraints. To provide baseline results, we have solved the proposed test problems using a nested bilevel evolutionary algorithm. The results can be used for comparison, while evaluating the performance of any other bilevel optimization algorithm. The code related to the paper may be accessed from the website http://bilevel.org .

Having developed multiobjective optimization algorithms using evolutionary optimization methods and demonstrated their niche on various practical problems involving mostly two and three objectives, there is now a growing need for developing evolutionary multiobjective optimization (EMO) algorithms for handling many-objective (having four or more objectives) optimization problems. In this paper, we recognize a few recent efforts and discuss a number of viable directions for developing a potential EMO algorithm for solving many-objective optimization problems. Thereafter, we suggest a reference-point-based many-objective evolutionary algorithm following NSGA-II framework (we call it NSGA-III) that emphasizes population members that are nondominated, yet close to a set of supplied reference points. The proposed NSGA-III is applied to a number of many-objective test problems with three to 15 objectives and compared with two versions of a recently suggested EMO algorithm (MOEA/D). While each of the two MOEA/D methods works well on different classes of problems, the proposed NSGA-III is found to produce satisfactory results on all problems considered in this paper. This paper presents results on unconstrained problems, and the sequel paper considers constrained and other specialties in handling many-objective optimization problems.

In the precursor paper, a many-objective optimization method (NSGA-III), based on the NSGA-II framework, was suggested and applied to a number of unconstrained test and practical problems with box constraints alone. In this paper, we extend NSGA-III to solve generic constrained many-objective optimization problems. In the process, we also suggest three types of constrained test problems that are scalable to any number of objectives and provide different types of challenges to a many-objective optimizer. A previously suggested MOEA/D algorithm is also extended to solve constrained problems. Results using constrained NSGA-III and constrained MOEA/D show an edge of the former, particularly in solving problems with a large number of objectives. Furthermore, the NSGA-III algorithm is made adaptive in updating and including new reference points on the fly. The resulting adaptive NSGA-III is shown to provide a denser representation of the Pareto-optimal front, compared to the original NSGA-III with an identical computational effort. This, and the original NSGA-III paper, together suggest and amply test a viable evolutionary many-objective optimization algorithm for handling constrained and unconstrained problems. These studies should encourage researchers to use and pay further attention in evolutionary many-objective optimization.

Code smells represent design situations that can affect the maintenance and evolution of software. They make the system difficult to evolve. Code smells are detected, in general, using quality metrics that represent some symptoms. However, the selection of suitable quality metrics is challenging due to the absence of consensus in identifying some code smells based on a set of symptoms and also the high calibration effort in determining manually the threshold value for each metric. In this article, we propose treating the generation of code-smell detection rules as a bilevel optimization problem. Bilevel optimization problems represent a class of challenging optimization problems, which contain two levels of optimization tasks. In these problems, only the optimal solutions to the lower-level problem become possible feasible candidates to the upper-level problem. In this sense, the code-smell detection problem can be treated as a bilevel optimization problem, but due to lack of suitable solution techniques, it has been attempted to be solved as a single-level optimization problem in the past. In our adaptation here, the upper-level problem generates a set of detection rules, a combination of quality metrics, which maximizes the coverage of the base of code-smell examples and artificial code smells generated by the lower level. The lower level maximizes the number of generated artificial code smells that cannot be detected by the rules produced by the upper level. The main advantage of our bilevel formulation is that the generation of detection rules is not limited to some code-smell examples identified manually by developers that are difficult to collect, but it allows the prediction of new code-smell behavior that is different from those of the base of examples. The statistical analysis of our experiments over 31 runs on nine open-source systems and one industrial project shows that seven types of code smells were detected with an average of more than 86% in terms of precision and recall. The results confirm the outperformance of our bilevel proposal compared to state-of-art code-smell detection techniques. The evaluation performed by software engineers also confirms the relevance of detected code smells to improve the quality of software systems.

Evolutionary algorithms are becoming increasingly popular for multimodal and multi-objective optimization. Their population based nature allows them to be modified in a way so as to locate and preserve multiple optimal solutions (referred to as Pareto-optimal solutions in multi-objective optimization). These modifications are called niching methods, particularly in the context of multimodal optimization. In evolutionary multiobjective optimization, the concept of dominance and diversity preservation inherently causes niching. This paper proposes an approach to multimodal optimization which combines this power of dominance with traditional variable-space niching. The approach is implemented within the NSGA-II framework and its performance is studied on 20 benchmark problems. The simplicity of the approach and the absence of any special niching parameters are the hallmarks of this study.

Constrained optimization is one of the popular research areas since constraints are usually present in most real world optimization problems. The purpose of this work is to develop a gradient free constrained global optimization methodology to solve this type of problems. In the methodology proposed, the single objective constrained optimization problem is solved using a Multi-Objective Evolutionary Algorithm (MOEA) by considering two objectives simultaneously, the original objective function and a measure of constraint violation. The MOEA incorporates a penalty function where the penalty parameter is estimated adaptively. The use of penalty function method will enable to further improve the current best solution by decreasing the level of constraint violation, which is made using a gradient free local search method. The performance of the proposed methodology was assessed on a set of benchmark test problems. The results obtained allowed to conclude that the present approach is competitive when compared with other methods available.

The holy grail of constrained optimization is the development of an efficient, scale invariant and generic constraint handling procedure in single and multi-objective constrained optimization problems. In this paper, an individual penalty parameter based methodology is proposed to solve constrained optimization problems. The individual penalty parameter approach is a hybridization between an evolutionary method, which is responsible for estimation of penalty parameters for each constraint and the initial solution for local search. However the classical penalty function approach is used for its convergence property. The aforesaid method adaptively estimates penalty parameters linked with each constraint and it can handle any number of constraints. The method is tested over multiple runs on six mathematical test problems and a engineering design problem to verify its efficacy. The function evaluations and obtained solutions of the proposed approach is compared with three of our previous results. In addition to that, the results are also verified with some standard methods taken from literature. The results show that our method is very efficient compared to some recently developed methods.

In this paper, we apply Differential Evolution (DE) algorithm in combination with a recently proposed constraint-handling strategy and study the performance 01' the resulting algorithm on CEC'13 test suite [1], and other constrained optimization problems. The goal of this exercise is to clearly identify and highlight the challenges encountered with the DE search while solving a range of optimization problems. We emphasize that understanding and resolving fundamental issues of a search procedure and considering the nature of the optimization problems at hand is the key to effective deployment of evolutionary procedures for search and optimization.

This paper proposes the use of knee finding methods to solve cluster analysis problems from a multi-objective approach. The above proposal arises as a result of a bi-objective study of clustering problems where knee regions on the obtained Pareto-optimal fronts were observed. With increased noise in the data, these knee regions tend to get smoother but still comprise the preferred solution. Thus, being the knees what decision makers are interested in when analysing clustering problems, it makes sense to boost the search towards those regions by applying knee finding techniques.

In Evolutionary Multi-objective Optimization many solutions have to be evaluated to provide the decision maker with a diverse choice of solutions along the Pareto-front, in particular for high-dimensional optimization problems. In Simulation-based Optimization the modeled systems are complex and require long simulation times. In addition the evaluated systems are often stochastic and reliable quality assessment of system configurations by resampling requires many simulation runs. As a countermeasure for the required high number of simulation runs caused by multiple optimization objectives the optimization can be focused on interesting parts of the Pareto-front, as it is done by the Reference point-guided NSGA-II algorithm (R-NSGA-II) [9]. The number of evaluations needed for the resampling of solutions can be reduced by intelligent resampling algorithms that allocate just as much sampling budget needed in different situations during the optimization run. In this paper we propose and compare resampling algorithms that support the R-NSGA-II algorithm on optimization problems with stochastic evaluation functions.

Bilevel programming problems are often found in practice. In this paper, we handle one such bilevel application problem from the domain of environmental economics. The problem is a Stakelberg game with multiple objectives at the upper level, and a single objective at the lower level. The leader in this case is the regulating authority, and it tries to maximize its total tax revenue over multiple periods while trying to minimize the environmental damages caused by a mining company. The follower is the mining company whose sole objective is to maximize its total profit over multiple periods under the limitations set by the leader. The solution to the model contains the optimal taxation and extraction decisions to be made by the players in each of the time periods. We construct a simplistic model for the Stackelberg game and provide an analytical solution to the problem. Thereafter, the model is extended to incorporate realism and is solved using a bilevel evolutionary algorithm capable of handling multiple objectives.

The difficulties faced by existing multiobjective evolutionary algorithms (MOEAs) in handling many-objective problems relate to the inefficiency of selection operators, high computational cost, and difficulty in visualization of objective space. While many approaches aim to counter these difficulties by increasing the fidelity of the standard selection operators, the objective reduction approach attempts to eliminate objectives that are not essential to describe the Pareto-optimal front (POF). If the number of essential objectives is found to be two or three, the problem could be solved by the existing MOEAs. It implies that objective reduction could make an otherwise unsolvable (many-objective) problem solvable. Even when the essential objectives are four or more, the reduced representation of the problem will have favorable impact on the search efficiency, computational cost, and decision-making. Hence, development of generic and robust objective reduction approaches becomes important. This paper presents a principal component analysis and maximum variance unfolding based framework for linear and nonlinear objective reduction algorithms, respectively. The major contribution of this paper includes: 1) the enhancements in the core components of the framework for higher robustness in terms of applicability to a range of problems with disparate degree of redundancy; mechanisms to handle input data that poorly approximates the true POF; and dependence on fewer parameters to minimize the variability in performance; 2) proposition of an error measure to assess the quality of results; 3) sensitivity analysis of the proposed algorithms for the critical parameter involved, and the characteristics of the input data; and 4) study of the performance of the proposed algorithms vis-à-vis dominance relation preservation based algorithms, on a wide range of test problems (scaled up to 50 objectives) and two real-world problems.

Evolutionary multi-objective optimization algorithms are widely used for solving optimization problems with multiple conflicting objectives. However, basic evolutionary multi-objective optimization algorithms have shortcomings, such as slow convergence to the Pareto optimal front, no efficient termination criterion, and a lack of a theoretical convergence proof. A hybrid evolutionary multi-objective optimization algorithm involving a local search module is often used to overcome these shortcomings. But there are many issues that affect the performance of hybrid evolutionary multi-objective optimization algorithms, such as the type of scalarization function used in a local search and frequency of a local search. In this paper, we address some of these issues and propose a hybrid evolutionary multi-objective optimization framework. The proposed hybrid evolutionary multi-objective optimization framework has a modular structure, which can be used for implementing a hybrid evolutionary multi-objective optimization algorithm. A sample implementation of this framework considering NSGA-II, MOEA/D, and MOEA/D-DRA as evolutionary multi-objective optimization algorithms is presented. A gradient-based sequential quadratic programming method as a single objective optimization method for solving a scalarizing function used in a local search is implemented. Hence, only continuously differentiable functions were considered for numerical experiments. The numerical experiments demonstrate the usefulness of our proposed framework.

Land use management is increasingly becoming complex as the public and governing bodies demand more accountability and transparency in management practices that simultaneously guarantee sustainable production of goods and continued provision of ecosystem services (i.e., public goods with no markets, such as clean air). In this paper we demonstrate a novel form of decision making that will assist in meeting some of these challenges in ensuring sustainability in land use management. We apply a modified Multi-Objective Evolutionary Algorithm (MOEA), influenced by epigenetic silencing, to a farm case study. The result is a set of time-series, farm management strategies and their related spatial arrangements of land uses that satisfy 14 incommensurable and sometimes conflicting objectives, and spatial constraints. The 14 objectives cover economic (i.e. productivity and financials) and environmental issues. Choosing a single strategy from the set for implementation will require social-ethical value judgment determined from preferences and values of multiple decision-makers. This part of the decision making process is beyond the scope of this paper, but will contribute to ongoing research which will make it possible to fully account for the Triple Bottom Line (TBL), characterised by environmental, economic and social elements.

Optimization for single main objective with multi constraints is considered using a probabilistic approach coupled to evolutionary search. In this approach the problem is converted into a bi-objective problem, treating the constraint ensemble as a second objective subjected to multi-objective optimization for the formation of a Pareto front, and this is followed by a local search for the optimization of the main objective function. In this process a novel probabilistic modeling is applied to the constraint ensemble, so that the stiff constraints are effectively taken care of, while the model parameter is adaptively determined during the evolutionary search. In this way the convergence to the solution is significantly accelerated and an accurate solution is established. The improvements are demonstrated by means of example problems including comparisons with the standard benchmark problems, the solutions of which are reported in the literature.

A hybrid adaptive normalization based constraint handling approach is proposed in the present study. In most constrained optimization problems, constraints may be of different scale. Normalization of constraints is crucial for the efficient performance of a constraint handling algorithm. A growing number of researchers have proposed different strategies using bi-objective methodologies. Classical penalty function approach is another common method among both evolutionary and classical optimization research communities due to its simplicity and ease of implementation. In the present study, we propose a hybrid approach of both bi-objective method and the penalty function approach where constraints are normalized adaptively during the optimization process. The proposed bi-objective evolutionary method estimates the penalty parameter and the starting solution needed for the penalty function approach. We test and compare our algorithm on seven mathematical test problems and two engineering design problems taken from the literature. We compare our obtained results with our previous studies in terms of function evaluations and solution accuracy. The obtained optima are also compared with those of other standard algorithms. In many cases, our proposed methodology perform better than all algorithms considered in this study. Results are promising and motivate further application of the proposed adaptive normalization strategy.

Handling many-objective problems is one of the primary concerns to EMO researchers. In this paper, we discuss a number of viable directions for developing a potential EMO algorithm for many-objective optimization problems. Thereafter, we suggest a reference-point based many-objective NSGA-II (or MO-NSGA-II) that emphasizes population members which are non-dominated yet close to a set of well-distributed reference points. The proposed MO-NSGA-II is applied to a number of many-objective test problems having three to 10 objectives (constrained and unconstrained) and compared with a recently suggested EMO algorithm (MOEA/D). The results reveal difficulties of MOEA/D in solving large-sized and differently-scaled problems, whereas MO-NSGA-II is reported to show a desirable performance on all test-problems used in this study. Further investigations are needed to test MO-NSGA-II's full potential.

Evolutionary Multi-objective Optimization aims at finding a diverse set of Pareto-optimal solutions whereof the decision maker can choose the solution that fits best to her or his preferences. In case of limited time (of function evaluations) for optimization this preference information may be used to speed up the search by making the algorithm focus directly on interesting areas of the objective space. The R-NSGA-II algorithm [1] uses reference points to which the search is guided specified according to the preferences of the user. In this paper, we propose an extension to R-NSGA-II that limits the Pareto-fitness to speed up the search for a limited number of function calls. It avoids to automatically select all solutions of the first front of the candidate set into the next population. In this way non-preferred Pareto-optimal solutions are not considered thereby accelerating the search process. With focusing comes the necessity to maintain diversity. In R-NSGA-II this is achieved with the help of a clustering algorithm which keeps the found solutions above a minimum distance ε. In this paper, we propose a self-adaptive ε approach that autonomously provides the decision maker with a more diverse solution set if the found Pareto-set is situated further away from a reference point. Similarly, the approach also varies the diversity inside of the Pareto-set. This helps the decision maker to get a better overview of the available solutions and supports decisions about how to adapt the reference points.

In this paper, we propose a set of six test problems for single-objective bilevel optimization. The test-collection represents various difficulties which are commonly encountered in practical bilevel optimization problems. To support experiments with problems of different size, all of the test problems are scalable in terms of the number of variables. The problem set is also accompanied by a construction procedure, which helps to generate new test problems with controlled difficulties in convergence and interaction patterns between the two optimization levels. To provide a baseline result for easy comparisons, we have solved a 10 variable instance for each of the test problems using a simple bilevel evolutionary algorithm. The results presented may be used as a benchmark while evaluating the performance of any bilevel optimization algorithm.

In recent advancements towards handling high objective optimization problems, it is proposed to progressively integrate the decision maker with the execution of an evolutionary multi-objective optimization algorithm. Preferences from the decision maker are accepted at the intermediate steps of the algorithm and a progress towards the most preferred point is made. In this paper, we extend the work on `progressively interactive evolutionary multi-objective optimization using value function' (PI-EMO-VF) by allowing the optimization to be performed in a fixed number of interactions with the decision maker. In the PI-EMO-VF procedure, information is accepted from the decision maker, which is utilized by the evolutionary algorithm to perform a focused search in the region of interest. However, it is not possible to restrict the number of interactions required to handle an optimization problem. This paper contributes towards, solving the optimization problem in a pre-decided number of decision maker calls. Once the available budget of decision maker calls are known, it is optimally utilized to get close to the most preferred point on the Pareto-frontier. The paper evaluates the performance of the modified PI-EMOVF algorithm on two, three and five objective test problems. A comparative study is performed against the previous proposal for the PI-EMO-VF procedure.

In a multimodal optimization task, the main purpose is to find multiple optimal solutions (global and local), so that the user can have better knowledge about different optimal solutions in the search space and as and when needed, the current solution may be switched to another suitable optimum solution. To this end, evolutionary optimization algorithms (EA) stand as viable methodologies mainly due to their ability to find and capture multiple solutions within a population in a single simulation run. With the preselection method suggested in 1970, there has been a steady suggestion of new algorithms. Most of these methodologies employed a niching scheme in an existing single-objective evolutionary algorithm framework so that similar solutions in a population are deemphasized in order to focus and maintain multiple distant yet near-optimal solutions. In this paper, we use a completely different strategy in which the single-objective multimodal optimization problem is converted into a suitable bi-objective optimization problem so that all optimal solutions become members of the resulting weak Pareto-optimal set. With the modified definitions of domination and different formulations of an artificially created additional objective function, we present successful results on problems with as large as 500 optima. Most past multimodal EA studies considered problems having only a few variables. In this paper, we have solved up to 16-variable test problems having as many as 48 optimal solutions and for the first time suggested multimodal constrained test problems which are scalable in terms of number of optima, constraints, and variables. The concept of using bi-objective optimization for solving single-objective multimodal optimization problems seems novel and interesting, and more importantly opens up further avenues for research and application.

Real-world optimization problems often involve highly non-linear objectives and constraints. From an application point of view, it is usually desirable that the global optimum be achieved in such cases. Among selection, crossover and mutation operators of a genetic algorithm, the last two are responsible for search and diversity maintenance. By improving these operators, the efficiency of GAs can be improved. In this paper, we solve the problems specified in "CEC 2011 Competition on Testing Evolution Algorithms on Real World Optimization Problems" using a variation of the Simulated Binary Crossover (SBX) which adaptively shifts between parent-centric and mean-centric recombinations. The shift occurs automatically during program execution through the use of current population statistics and is expected to improve the performance of GA. Further, we also employ a self-adaptive mutation strategy developed earlier.

The task of finding crucial design interdependencies in the form of mathematical relationships (empirical or otherwise) in an engineering design problem using the Pareto optimal front is referred to as innovization. Past studies on the subject have limited themselves to a single front. In this paper we introduce the higher-level innovization task through an application of a manufacturing process simulation for the Friction Stir Welding (FSW) process where commonalities among two different Pareto-optimal fronts are analyzed. Multiple design rules are simultaneously deciphered from each front separately and compared. Important design aspects of the FSW problem are revealed in the process. The overall study aims at showing how some design principles can considerably ease the task of optimizing future enhancements to the design.

This paper proposes a novel method for using the service (field failure) data of consumer vehicles to estimate customer perception. To achieve this, relevant variables are extracted from the vehicle service data and provided as input to the proposed algorithm which then comes up with an optimized mathematical model for predicting the Customer Satisfaction Index or CSI. The methodology is then extended in a way that allows comparison of the CSIs of two or more vehicle models, thus providing a measure of the market's perceived quality of a vehicle model relative to another. Validation against the Consumer Reports data shows that customer experiences and their consequent response in surveys are indeed a reflection of the numbers the service data provides. However, it is argued that the proposed model is more generic than the Consumer Reports because: (1) it doesn't rely on consumer surveys and (2) it can be used to assess individual consumer level satisfaction.

This paper is concerned with the determination of optimum forces extracted by robot grippers on the surface of a grasped rigid object -- a matter which is crucial to guarantee the stability of the grip without causing defect or damage to the grasped object. A multi-criteria optimization of robot gripper design problem is solved with two different configurations involving two conflicting objectives and a number of constraints. The objectives involve minimization of the difference between maximum and minimum gripping forces and simultaneous minimization of the transmission ratio between the applied gripper actuator force and the force experienced at the gripping ends. Two different configurations of the robot gripper are designed by a state-of-the-art algorithm (NSGA-II) and the obtained results are compared with a previous study. Due to presence of geometric constraints, the resulting optimization problem is highly non-linear and multi-modal. For both gripper configurations, the proposed methodology outperforms the results of the previous study. The Pareto-optimal solutions are thoroughly investigated to establish some meaningful relationships between the objective functions and variable values. In addition, it is observed that one of the gripper configurations completely outperforms the other one from the point of view of both objectives, thereby establishing a complete bias towards the use of one of the configurations in practice.

For problems involving uncertainties in design variables and parameters, a bi-objective evolutionary algorithm (EA) based approach to design optimization using evidence theory is proposed and implemented in this paper. In addition to a functional objective, a plausibility measure of failure of constraint satisfaction is minimized. Despite some interests in classical optimization literature, such a consideration in EA is rare. Due to EA’s flexibility in its operators, non-requirement of any gradient, its ability to handle multiple conflicting objectives, and ease of parallelization, evidence-based design optimization using an EA is promising. Results on a test problem and a couple of engineering design prob-lems show that the modified evolutionary multi-objective optimization (EMO) algorithm is capable of finding a widely distributed trade-off frontier showing different optimal solu-tions corresponding to different levels of plausibility failure limits. Furthermore, a single-objective evidence based EA is found to produce better optimal solutions than a previously reported classical optimization procedure. Handling uncer-tainties of different types are getting increasingly popular in applied optimization studies and more such studies using EAs will make EAs more useful and pragmatic in practical optimization problem-solving tasks. Categories and Subject Descriptors J.6 [Computer Applications]: Computer-aided engineer-ing—computer-aided design

Genetic Algorithms(GAs) are suitable for parallel computing since population members fitness maybe evaluated in parallel. Most past parallel GA studies have exploited this aspect, besides resorting to different algorithms, such as island, single-population master-slave, fine-grained and hybrid models. A GA involves a number of other operations which, if parallelized, may lead to better parallel GA implementation than those currently existing. In this paper, we parallelize binary and real-coded genetic algorithms using CUDA API's with C. Although, objective and constraint violations evaluations are embarassingly parallel, other algorithmic and code optimizations have been proposed and tested. The bottlenecks in a parallel GA implementation are identified and modified suitably. The results are compared with the sequential algorithm on accuracy and clock time for varying problems by studying the effect of a number of parameters, namely: (i) population sizes, (ii) number of threads, (iii) problem sizes, and (iv) problems of differing complexities. Significant speed-ups have been observed over the sequential GA.

Real world multi-objective optimization problems are often solved with the only intention of selecting a single trade-off solution by taking up a decision-making task. The computational effort and time spent on obtaining the entire Pareto front is thus not justifiable. The Pareto solutions as a whole contain within them a lot more information than that is used. Extracting this knowledge would not only give designers a better understanding of the system, but also bring worth to the resources spent. The obtained knowledge acts as governing principles which can help solve other similar systems easily. We propose a genetic algorithm based unsupervised approach for learning these principles from the Pareto-optimal dataset of the base problem. The methodology is capable of discovering analytical relationships of a certain type between different problem entities.

Evolutionary algorithms are modified in various ways to solve constrained optimization problems. Of them, the use of a bi-objective evolutionary algorithm in which the minimization of the constraint violation is included as an additional objective, has received a significant attention. Classical penalty function approach is another common methodology which requires an appropriate knowledge of the associated penalty parameter. In this paper, we combine a bi-objective evolutionary approach with the penalty function methodology in a manner complementary to each other. The bi-objective optimization approach provides a good estimate of the penalty parameter, while the unconstrained penalty function approach using classical means provides the overall hybrid algorithm its convergence property. We demonstrate the working of the procedure on a two-variable problem and then solve a number of standard numerical test problems from the EA literature. In all cases, our proposed hybrid methodology is observed to take one or more orders of magnitude smaller number of function evaluations to find the constrained minimum solution accurately. To the best of our knowledge, no previous evolutionary constrained optimization algorithm has reported such a fast and accurate performance on the chosen problems.

Niching is an important technique for multimodal optimization in Evolutionary Computation. Most existing niching algorithms are evaluated using only 1 or 2 dimensional multimodal functions. However, it remains unclear how these niching algorithms perform on higher dimensional multimodal problems. This paper compares several schemes of PSO update rules, and examines the effects of incorporating these schemes into a lbest PSO niching algorithm using a ring topology. Subsequently a new Cauchy and Gaussian distributions based PSO (CGPSO) is proposed. Our experiments suggest that CGPSO seems to be able to locate more global peaks than other PSO variants on multimodal functions which typically have many global peaks but very few local peaks.

Genetic Algorithms (GAs) are a highly successful population based approach to solve global optimization problems. They have carved out a niche for themselves in solving optimization problems of varying difficulty levels involving single and multiple objectives. Most real-world optimization problems involve equality and / or inequality constraints and hence posed as constrained optimization problems. The most common approach to solve such problems using GAs is the method of penalty functions, which however suffers from the drawback of appropriate selection of penalty parameters for their optimal functioning. Given the nature of the problems at hand, we have used an adaptive mutation based Real-Coded GA (RGA), which uses a popular penalty parameter-less approach to handle constraints and search the feasible region effectively for the global best solution, and at the same time use an adaptive mutation strategy to maintain diversity in the population to enable creation of new solutions. We have coupled our RGA with ideas from the gradient projection method to specifically handle equality constraints. We have found our simple procedure working quite well in most of the test problems provided as part of the competition on Single-objective Constrained Real Parameter Optimization in CEC 2010 and hence simplicity remains the hallmark of our study here.

This paper advances and evaluates a recently proposed progressively interactive evolutionary multi-objective optimization algorithm. The algorithm uses preference information from the decision maker during the intermediate generations of the EMO and produces the most preferred solution on the Pareto-optimal front. The progress towards the Pareto-optimal front is made by approximating decision maker's value function. In this paper, a generalized polynomial value function has been proposed and the procedure to fit the value function to the decision maker's preference information has been described. The generality of the procedure of fitting a value function to the decision maker's preferences has been shown by using other existing value functions from the literature. The proposed generic polynomial value function has been incorporated in the PI-EMO-VF algorithm to efficiently approximate the decision maker's value function. The paper then evaluates the performance of the PI-EMO-VF algorithm on three and five objective test problems with constraints. It also evaluates the efficacy of the procedure in producing the most preferred solution when the decision maker is unable to provide perfect information, i.e., the decision maker finds certain pairs of solutions in the objective space to be incomparable. Results have been presented for three and five objective constrained test problems using the procedure.

Particle swarm optimization (PSO) has been in practice for more than 10 years now and has gained wide popularity in various optimization tasks. In the context to single objective optimization, this paper studies two aspects of PSO: (i) its ability to approach an 'optimal basin', and (ii) to find the optimum with high precision once it enters the region. of interest. We test standard PSO algorithms and discover their inability in handling both aspects efficiently. To address these issues with PSO, we propose an evolutionary algorithm (EA) which is algorithmically similar to PSO, and then borrow different EA-specific operators to enhance the PSO's performance. Our final proposed PSO contains a parent-centric recombination operator instead of usual particle update rule, but maintains PSO's individualistic trait and has a demonstrated performance comparable to a well-known GA (and outperforms the GA in some occasions). Moreover, the modified PSO algorithm is found to scale up to solve as large as 100-variable problems. This study emphasizes the need for similar such studies in establishing an equivalence between various genetic/evolutionary and other bio-inspired algorithms, a process that may lead us to better understand the scope and usefulness of various operators associated with each algorithm.

In a multimodal optimization task, the main purpose is to find multiple optimal (global and local) solutions associated with a single objective function. Starting with the preselection method suggested in 1970, most of the existing evolutionary algorithms based methodologies employ variants of niching in an existing single-objective evolutionary algorithm framework so that similar solutions in a population are de-emphasized in order to focus and maintain multiple distant yet near-optimal solutions. In this paper, we use a completely different and generic strategy in which a single-objective multimodal optimization problem in converted into a suitable bi-objective optimization problem so that all local and global optimal solutions become members of the resulting weak Pareto-optimal set. We solve up to 16-variable test-problems having as many as 48 optima and also demonstrate successful results on constrained multimodal test-problems, suggested for the first time. The concept of using multi-objective optimization for solving single-objective multimodal problems seems novel and interesting, and importantly opens further avenues for research.

With many years of research and application to real-world problems, evolutionary algorithms (EAs) have solved various problems having thousands of variables, hard heuristic constraints, and complex evaluation procedures. This paper reports another successful application of EAs in open pit mine scheduling. Typically an ore body is discretized as a 3D block model which, depending on factors such as the amount of data obtained, size of deposit, block dimensions etc. can be made up of over one million blocks, thereby requiring an optimization algorithm to handle over a million variables. Open pit mine scheduling is a complex task which is subject to very strict hard geometrical and other practical mining constraints. To the best of our knowledge there are currently no algorithm or software package that can cater for the large number of constraints and sheer scale of the data sets represented by open pit mine scheduling. Most packages are limited in the size of block model and the kind of objective and constraint functions they can efficiently handle. The proposed optimization algorithm and the resulting software (evORElution -- a trademark product of ORElogy) is developed by using the theoretical and fundamental results of evolutionary algorithms and has already been successfully used to produce complex multi-objective schedules for several large open pit iron ore mines involving hundreds of thousands to millions of variables.

This paper proposes an integrated approach to arrive at optimal build orientations, simultaneously minimizing surface roughness `Ra' and build time `T', for object manufacturing in SLS process. The optimization task is carried out by two popularly known multi-objective evolutionary optimizers - NSGA-II (non-dominated sorting genetic algorithm) and MOPSO (multi-objective particle swarm optimizer). The performance comparison of these two optimizers along with an approximation of Pareto-optimal front is done using two statistically significant performance measures. Three proposals addressing the task of decision making, i.e. selecting one solution in presence of multiple trade-off solutions, are introduced to facilitate the designer. The overall procedure is integrated into MORPE - Multi-objective Rapid Prototyping Engine. Several sample objects are considered for experimentation to demonstrate the working of MORPE. A careful study of optimal build directions for several components indicates a trend, providing insight into the SLS processes which can be regarded highly useful for various practical RP applications.

Evolutionary algorithms (EAs) are increasingly being applied to solve real-parameter optimization problems due to their flexibility in handling complexities such as non-convexity, non-differentiability, multi-modality and noise in problems. However, an EA's solution is never guaranteed to be optimal in generic problems, even for smooth problems, and importantly EAs still lack theoretically motivated termination criterion for stopping an EA run only when a near-optimal point is found. We address both these issues in this paper by integrating the Karush-Kuhn-Tucker (KKT) optimality conditions that involve first-order derivatives of objective and constraint functions with an EA. For this purpose, we define a KKT-proximity measure by relaxing the complimentary slackness condition associated with the KKT conditions. Results on a number of standard constrained test problems indicate that in spite of not using any gradient information and any theoretical optimality conditions, an EA's selection, recombination and mutation operation lead the search process to a point close to the KKT point. This suggests that the proposed KKT-proximity measure can be used termination criterion in an EA simulation.

Bilevel optimization problems involve two optimization tasks (upper and lower level), in which every feasible upper level solution must correspond to an optimal solution to a lower level optimization problem. These problems commonly appear in many practical problem solving tasks including optimal control, process optimization, game-playing strategy developments, transportation problems, and others. However, they are commonly converted into a single level optimization problem by using an approximate solution procedure to replace the lower level optimization task. Although there exist a number of theoretical, numerical, and evolutionary optimization studies involving single-objective bilevel programming problems, not many studies look at the context of multiple conflicting objectives in each level of a bilevel programming problem. In this paper, we address certain intricate issues related to solving multi-objective bilevel programming problems, present challenging test problems, and propose a viable and hybrid evolutionary-cum-local-search based algorithm as a solution methodology. The hybrid approach performs better than a number of existing methodologies and scales well up to 40-variable difficult test problems used in this study. The population sizing and termination criteria are made self-adaptive, so that no additional parameters need to be supplied by the user. The study indicates a clear niche of evolutionary algorithms in solving such difficult problems of practical importance compared to their usual solution by a computationally expensive nested procedure. The study opens up many issues related to multi-objective bilevel programming and hopefully this study will motivate EMO and other researchers to pay more attention to this important and difficult problem solving activity.

The four papers in this special issue focus on preference-based multiobjective evolutionary algorithms.

A nadir objective vector is constructed from the worst Pareto-optimal objective values in a multiobjective optimization problem and is an important entity to compute because of its significance in estimating the range of objective values in the Pareto-optimal front and also in executing a number of interactive multiobjective optimization techniques. Along with the ideal objective vector, it is also needed for the purpose of normalizing different objectives, so as to facilitate a comparison and agglomeration of the objectives. However, the task of estimating the nadir objective vector necessitates information about the complete Pareto-optimal front and has been reported to be a difficult task, and importantly an unsolved and open research issue. In this paper, we propose certain modifications to an existing evolutionary multiobjective optimization procedure to focus its search toward the extreme objective values and combine it with a reference-point based local search approach to constitute a couple of hybrid procedures for a reliable estimation of the nadir objective vector. With up to 20-objective optimization test problems and on a three-objective engineering design optimization problem, one of the proposed procedures is found to be capable of finding the nadir objective vector reliably. The study clearly shows the significance of an evolutionary computing based search procedure in assisting to solve an age-old important task in the field of multiobjective optimization.

This paper suggests a preference-based methodology, which is embedded in an evolutionary multiobjective optimization algorithm to lead a decision maker (DM) to the most preferred solution of her or his choice. The progress toward the most preferred solution is made by accepting preference based information progressively from the DM after every few generations of an evolutionary multiobjective optimization algorithm. This preference information is used to model a strictly monotone value function, which is used for the subsequent iterations of the evolutionary multiobjective optimization (EMO) algorithm. In addition to the development of the value function which satisfies DM's preference information, the proposed progressively interactive EMO-approach utilizes the constructed value function in directing EMO algorithm's search to more preferred solutions. This is accomplished using a preference-based domination principle and utilizing a preference-based termination criterion. Results on two- to five-objective optimization problems using the progressively interactive NSGA-II approach show the simplicity of the proposed approach and its future promise. A parametric study involving the algorithm's parameters reveals interesting insights of parameter interactions and indicates useful parameter values. A number of extensions to this paper are also suggested.

Genetic algorithms (GAs) have been argued to constitute a flexible search thereby enabling to solve difficult problems which classical optimization methodologies may find hard to solve. This paper is intended towards this direction and show a systematic application of a GA and its modification to solve a real-world optimization problem of sizing a solar thermal electricity plant. Despite the existence of only three variables, this problem exhibits a number of other common difficulties - black-box nature of solution evaluation, massive multi-modality, wide and non-uniform range of variable values, and terribly rugged function landscape - which prohibits a classical optimization method to find even a single acceptable solution. Both GA implementations perform well and a local analysis is performed to demonstrate the optimality of obtained solutions. This study considers both classical and genetic optimization on a fairly complex yet typical real-world optimization problems and demonstrates the usefulness and future of GAs in applied optimization activities in practice.

Many real-world problems demand a feasible solution to satisfy physical equilibrium, stability, or certain properties which require an additional lower level optimization problem to be solved. Although such bilevel problems are studied somewhat in the context of a single objective in each level, there are not many studies in which multiple conflicting objectives are considered in each level. Bilevel multi-objective optimization problems offer additional complexities, as not every lower level Pareto-optimal front has a representative solution to the upper level Pareto-optimal front and that only a tiny fraction of participating lower level fronts make it to the upper level front. A couple of recent studies by the authors have suggested a viable EMO method to handle such problems. In this paper, we analyze the difficulties which a bilevel EMO procedure may face in handling such problems and present a systematic construction procedure for bilevel optimization test problems. Based on the suggested principles, we propose five test problems which are scalable in terms of number of variables and objectives, and which enable researchers to evaluate different phases of a bilevel problem solving task. The test problem construction procedure is interesting and may motivate other researchers to extend the idea to develop further test problems.

Different approaches are in use to solve the problem of tomographic reconstruction, which is an inverse problem. Four different approaches; three variations of multiplicative algebraic reconstruction technique (MART) and a new approach based on genetic algorithms (GA), are evaluated and compared in the paper. The approaches are applied to the reconstruction of specimens from time-of-flight data collected by ultrasound transmission tomography. The time-of-flight data is simulated without taking into consideration the diffraction effects of ultrasound which is reasonably valid, only when the impedance mismatch in the specimen under consideration is small. Also it is assumed that the specimen under consideration consists of a maximum of three different materials with the goal being to identify the number, shape, and location of the inclusions in the specimen. The sensitivity of the various algorithms to the parameters involved, performance of various algorithms in terms of errors in reconstruction and time taken for the reconstruction are studied and presented here. Further the performance of the algorithms when the input data are contaminated with noise is presented. It is observed that although GA takes more time than MART, GA is reliable and accurate and scores much better than MART in dealing with problems where only limited data is available for the reconstruction.

Many objective optimization is a natural extension to multi-objective optimization where the number of objectives are significantly more than five. The performance of current state of the art algorithms (e.g. NSGA-II, SPEA2) is known to deteriorate significantly with increasing number of objectives due to the lack of adequate convergence pressure. It is of no surprise that the performance of NSGA-II on some constrained many-objective optimization problems (Deb and Saxena, 2006) (e.g., DTLZ5-(5,M), M = 10, 20) in an earlier study (Saxena, 2008) was far from satisfactory. Till date, research in many-objective optimization has focussed on two major areas (a) dimensionality reduction in the objective space and (b) preference ordering based approaches. This paper introduces a novel evolutionary algorithm powered by epsilon dominance (implemented within the framework of NSGA-II) and controlled infeasibility for improved convergence while the critical set of objectives is identified through a nonlinear dimensionality reduction scheme. Since approaching the Pareto-optimal front from within the feasible search space will need to overcome the problems associated with low selection pressure, the mechanism to approach the front from within the infeasible search space is promising as illustrated in this paper. The performance of the proposed algorithm is compared with NSGA-II (original, with crowding distance measure) and NSGA-II (epsilon dominance) on the above set of constrained multiobjective problems to highlight the benefits.

Evolutionary multi-objective optimization algorithms are commonly used to obtain a set of non-dominated solutions for over a decade. Recently, a lot of emphasis have been laid on hybridizing evolutionary algorithms with MCDM and mathematical programming algorithms to yield a computationally efficient and convergent procedure. In this paper, we test an augmented local search based EMO procedure rigorously on a test suite of constrained and unconstrained multi-objective optimization problems. The success of our approach on most of the test problems not only provides confidence but also stresses the importance of hybrid evolutionary algorithms in solving multi-objective optimization problems.

In this paper, the performance assessment of the hybrid Archive-based Micro Genetic Algorithm (AMGA) on a set of bound-constrained synthetic test problems is reported. The hybrid AMGA proposed in this paper is a combination of a classical gradient based single-objective optimization algorithm and an evolutionary multi-objective optimization algorithm. The gradient based optimizer is used for a fast local search and is a variant of the sequential quadratic programming method. The Matlab implementation of the SQP (provided by the fmincon optimization function) is used in this paper. The evolutionary multi-objective optimization algorithm AMGA is used as the global optimizer. A scalarization scheme based on the weighted objectives is proposed which is designed to facilitate the simultaneous improvement of all the objectives. The scalarization scheme proposed in this paper also utilizes reference points as constraints to enable the algorithm to solve non-convex optimization problems. The gradient based optimizer is used as the mutation operator of the evolutionary algorithm and a suitable scheme to switch between the genetic mutation and the gradient based mutation is proposed. The hybrid AMGA is designed to balance local versus global search strategies so as to obtain a set of diverse non-dominated solutions as quickly as possible. The simulation results of the hybrid AMGA are reported on the bound-constrained test problems described in the CEC09 benchmark suite.

Uncertainties in design variables and problem parameters are often inevitable and must be considered in an optimization task if reliable optimal solutions are sought. Besides a number of sampling techniques, there exist several mathematical approximations of a solution's reliability. These techniques are coupled in various ways with optimization in the classical reliability-based optimization field. This paper demonstrates how classical reliability-based concepts can be borrowed and modified and, with integrated single and multiobjective evolutionary algorithms, used to enhance their scope in handling uncertainties involved among decision variables and problem parameters. Three different optimization tasks are discussed in which classical reliability-based optimization procedures usually have difficulties, namely (1) reliability-based optimization problems having multiple local optima, (2) finding and revealing reliable solutions for different reliability indices simultaneously by means of a bi-criterion optimization approach, and (3) multiobjective optimization with uncertainty and specified system or component reliability values. Each of these optimization tasks is illustrated by solving a number of test problems and a well-studied automobile design problem. Results are also compared with a classical reliability-based methodology.

In this paper, we present a new paradigm of searching optimal strategies in the game of iterated prisoner's dilemma (IPD) using multiple-objective evolutionary algorithms. This method is more useful than the existing approaches, because it not only produces strategies that perform better in the iterated game but also finds a family of nondominated strategies, which can be analyzed to decipher properties a strategy should have to win the game in a more satisfactory manner. We present the results obtained by this new method and discuss sub-strategies found to be common among nondominated strategies. The multiobjective treatment of the IPD problem demonstrated here can be applied to other similar game-playing tasks.

This paper shows how a routine design optimization task can be enhanced to decipher important and innovative design principles which shall provide far-reaching knowledge about the problem at hand. Although the dasiainnovizationpsila task for this purpose was proposed by the first author elsewhere, the application to a brushless D.C. permanent magnet motor design is the first real application of the innovization concept to a discrete optimization problem. The model for cost and peak-torque objectives and associated constraints are borrowed from an existing study. The extent of knowledge gained in designing high-performing yet low-cost motors achieved in this study is phenomenal and should motivate other practitioners to pursue similar studies in other design and optimization related activities.

This paper offers a classification of the main representatives of interactive classical and evolutionary methods. After a crossfertilization of these two fields a new hybrid interactive reference point method is designed. The method combines the reference point idea with the relative speed of a (1+1) - EA and is implemented with a graphical user interface. Finally, it is validated on two well-known real-world test problems.

The notion ofoptimalsystemdesignholds thatinordertodasiatrulypsilamaximize/minimizeanobjectivefunction,thefeasiblesetneedstobeoptimized. Inspired by it, the attempt in our recent work was to incorporateconstraint-reductionin our earlier proposed procedures on dimensionality reduction of objectives. In that, while targetting constrained single-objective optimization problems (SOPs), we could arrive at a critical set of constraints and also their importance based rank-ordering. This information was used to study the shift from the constrained to the unconstrained optima. The methodology above was based on treating the a priori stated constraints as objectives besides the original-objective, and on applying (K. Deb et al., 2006), (D.K. Saxena et al., 2007) to this combined objective set-but-without constraints. In this work, the endeavor is to extend the above notion to the realm of multi-objective optimization problems (MOPs). Towards it, while we hire much from the above methodology, we make a fundamental shift, in that, we retain the a priori stated constraints, while evaluating the combined objective set. The motivation for this shift lies, in that, it allows more effective realization of the notion of system design than the approach in (D.K. Saxen et al., 2007). Reasonable effort has been spent on establishing this argument. Incorporating this change, a procedure for simultaneous reduction in objectives and constraints (for both SOPs, MOPs) is proposed, which also defines a realizable path towards optimal system design. Finally, the procedure is demonstrated on two test problems and one real world problem.

A new bi-objective optimization problem is formulated for generating the diverse topologies of compliant mechanisms tracing a user-defined path. Motivation behind the present study is to generate the compliant mechanisms which perform the same task of tracing a prescribed trajectory near minimum-weight solution. Therefore, the constraint are imposed at each precision point representing a prescribed path for accomplishing the tracing task. An additional constraint on stress is also included for the feasible designs. The study starts with a single objective analysis of minimum-weight of compliant mechanism and the obtained topology is referred as the reference design. Thereafter, a bi-objective optimization problem is solved by considering the objectives as minimization of weight of structure and maximization of diversity of structure with respect to the reference design. Here, the diversity is evaluated by finding the dissimilarity in the bit value at each gene position of the binary strings of the reference design and a structure evolved from the GA population. A local search based multi-objective genetic algorithm (MOGA) optimization procedure is used in which the NSGA-II is used as a global search and optimization algorithm. A parallel computing is employed in the study for evaluating nonlinear geometric FE analysis and also for the NSGA-II operations. After the NSGA-II run, a few solutions are selected from the non-dominated front and the local search is applied on them. With the help of a given optimization procedure, compliant mechanism designs tracing curvilinear and straight line trajectories are evolved and presented in the study. In both examples, compliant mechanisms are designed to have any arbitrary support and loading regions.

The game of Tic-tac-toe is one of the most commonly known games. This game does not allow one to win all the time and a significant proportion of games played results in a draw. Thus, the best a player can hope is to not lose the game. This study is aimed at evolving a number of no-loss strategies using genetic algorithms and comparing them with existing methodologies. To efficiently evolve no-loss strategies, we have developed innovative ways of representing and evaluating a solution, initializing the GA population, developing GA operators including an elite preserving scheme. Interestingly, our GA implementation is able to find more than 72 thousands no-loss strategies for playing the game. Moreover, an analysis of these solutions has given us insights about how to play the game to not lose it. Based on this experience, we have developed specialized efficient strategies having a high win-to-draw ratio. The study and its results are interesting and can be encouraging for the techniques to be applied to other board games for finding efficient strategies.

Evolutionary multi-objective optimization (EMO) methodologies, suggested in the beginning of Nineties, focussed on the task of finding a set of well-converged and well-distributed set of solutions using evolutionary optimization principles. Of the EMO methodologies, the elitist non-dominated sorting genetic algorithm or NSGA-II, suggested in 2000, is now probably the most popularly used EMO procedure. NSGA-II follows three independent principles -- domination principle, diversity preservation principle and elite preserving principle -- which make NSGA-II a flexible and robust EMO procedure in the sense of solving various multi-objective optimization problems using a common framework. In this paper, we describe NSGA-II through a functional decomposition following the implementation of these three principles and demonstrate how various multi-objective optimization tasks can be achieved by simply modifying one of the three principles. We argue that such a functionally decomposed and modular implementation of NSGA-II is probably the reason for it's popularity and robustness in solving various types of multi-objective optimization problems.

The use of a-priori information, where available, is an important step in solving an already computationally expensive tomographic imaging problem [1]. Here, an enhanced genetic algorithm based reconstruction technique is proposed that is capable of detecting the shape, size and location of multiple types of inclusions of known physical properties in a given test specimen. Preliminary results are found to be better than those reported with MART1. Simulations show that the algorithm is consistent for a wide range of grid sizes and geometries of inclusion(s). A logarithmic time complexity analysis gives a linear relationship between number of unknowns and reconstruction times, thus establishing the predictability of the algorithm.

While designing the Compliant Mechanisms (CM), an equal attention is required on both the problem formulation and the optimization algorithm used. Authors of this paper have successfully proposed the formulation of CM tracing user-defined paths based on the precision points. In this paper, authors modify the NSGA-II algorithm by incorporating (i) a helper objective and (ii) a domain specific crossover which assist in generating a diverse set of non-dominated solutions. First, the single-objective optimization problem of minimizing the weight of structure is solved and named the topology as a reference design. Thereafter, a bi-objective optimization problem is dealt to evolve 'trade-off' solutions for a primary objective of minimizing the weight and a secondary objective of maximizing the diversity with respect to the reference design. Both the optimization problems are solved using a local search based NSGA-II procedure. This study has further compared its results with another GA implementation having a different crossover operator.

In this paper, we propose a new evolutionary algorithm for multi-objective optimization. The proposed algorithm benefits from the existing literature and borrows several concepts from existing multi-objective optimization algorithms. The proposed algorithm employs a new kind of selection procedure which benefits from the search history of the algorithm and attempts to minimize the number of function evaluations required to achieve the desired convergence. The proposed algorithm works with a very small population size and maintains an archive of best and diverse solutions obtained so as to report a large number of non-dominated solutions at the end of the simulation. Improved formulation for some of the existing diversity preservation techniques is also proposed. Certain implementation aspects that facilitate better performance of the algorithm are discussed. Comprehensive benchmarking and comparison of the proposed algorithm with some of the state-of-the-art multi-objective evolutionary algorithms demonstrate the improved search capability of the proposed algorithm.

This paper describes a simulated annealing based multiobjective optimization algorithm that incorporates the concept of archive in order to provide a set of tradeoff solutions for the problem under consideration. To determine the acceptance probability of a new solution vis-a-vis the current solution, an elaborate procedure is followed that takes into account the domination status of the new solution with the current solution, as well as those in the archive. A measure of the amount of domination between two solutions is also used for this purpose. A complexity analysis of the proposed algorithm is provided. An extensive comparative study of the proposed algorithm with two other existing and well-known multiobjective evolutionary algorithms (MOEAs) demonstrate the effectiveness of the former with respect to five existing performance measures, and several test problems of varying degrees of difficulty. In particular, the proposed algorithm is found to be significantly superior for many objective test problems (e.g., 4, 5, 10, and 15 objective problems), while recent studies have indicated that the Pareto ranking-based MOEAs perform poorly for such problems. In a part of the investigation, comparison of the real-coded version of the proposed algorithm is conducted with a very recent multiobjective simulated annealing algorithm, where the performance of the former is found to be generally superior to that of the latter.

Conventional clustering algorithms based on Euclidean distance or Pearson correlation coefficient are not able to include order information in the distance metric and also unable to distinguish between random and real biological patterns. We present template based clustering algorithm for time series gene expression data. Template profiles are defined based on up-down regulation of genes between consecutive time points. Assignment of genes to templates is based on fuzzy membership function. Multi-objective evolutionary algorithm is used to determine compact clusters with varying number of templates. Statistical significance of each template is determined using permutation based non-parametric test. Statistically significant profiles are further tested for their biological relevance using gene ontology analysis. The algorithm was able to distinguish between real and noisy pattern when tested on artificial and real biological data. The proposed algorithm has shown better or similar performance compared to STEM and better than k-means on a real biological data.

In this paper a new approach to search for diverse solutions for a multi-objective problem is presented. Commonly, a search for solutions for a multi-objective problem, which is aimed at optimization, results in a set of Pareto optimal solutions. There are cases where more solutions should be also considered, nonetheless preserving the optimization inspiration. These solutions should not resemble the Pareto set, so as to provide diversity within the design space, and therefore they might not always be found by taking an epsilon-Pareto approach. With this motivation in mind, an already established method, which searches for diverse solutions, which are not all necessarily optimal, is herewith discussed and its shortages are highlighted. In contrast to the already established design method, the approach taken in this paper is to solve the multi-objective problem repeatedly, adding (automatically or interactively) at each run constraints, which are constructed, based on the obtained Pareto set. The motivation for the introduced approach comes from the need to generate a set of robot paths, which allow a mobile robot operator, flexibility in complying with different planning demands and a rapid response to a developing scenario. The methodology and the applicability of the approach are explained and demonstrated by utilizing multi-objective path planning problems.

In this paper, we combine reliability-based optimization with a multi-objective evolutionary algorithm for handling uncertainty in decision variables and parameters. This work is an extension to a previous study by the second author and his research group to more accurately compute a multi-constraint reliability. This means that the overall reliability of a solution regarding all constraints is examined, instead of a reliability computation of only one critical constraint. First, we present a brief introduction into this so-called 'structural reliability' aspects. Thereafter, we introduce a method for identifying inactive constraints according to the reliability evaluation. With this method, we show that with less number of constraint evaluations, an identical solution can be achieved. Furthermore, we apply our approach to a number of problems including a real-world car side impact design problem to illustrate our method.

For the past decade or so, evolutionary multi-objective optimization (EMO) methodologies have earned wide popularity for solving complex practical optimization problems, simply due to their ability to find a representative set of Pareto-optimal solutions for mostly two, three, and some extent to four and five-objective optimization problems. Recently, emphasis has been made in addressing the decision-making activities in arriving at a single preferred solution. The multiple criteria decision making (MCDM) literature offers a number of possibilities for such a task involving user preferences which can be supplied in different forms. This paper presents an interactive methodology for finding a preferred set of solutions, instead of the complete Pareto-optimal frontier, by incorporating preference information of the decision maker. Particularly, we borrow the concept of light beam search and combine it with the NSGA-II procedure. The working of this procedure has been demonstrated on a set of test problems and on engineering design problems having two to ten objectives, where the obtained solutions are found to match with the true Pareto-optimal solutions. The results highlight the utility of this approach towards eventually facilitating a better and more reliable optimization-cum-decision-making task.

Despite having a wide-spread applicability of evolutionary optimization procedures over the past few decades, EA researchers still face criticism about the theoretical optimality of obtained solutions. In this paper, we address this issue for problems for which gradients of objectives and constraints can be computed either exactly, or numerically or through subdifferentials. We suggest a systematic procedure of analyzing a representative set of Pareto-optimal solutions for their closeness to satisfying Karush-Kuhn-Tucker (KKT) points, which every Pareto-optimal solution must also satisfy. The procedure involves either a least-square solution or an optimum solution to a set of linear system of equations involving Lagrange multipliers. The procedure is applied to a number of differentiable and non-differentiable test problems and to a highly nonlinear engineering design problem. The results clearly show that EAs are capable of finding solutions close to theoretically optimal solutions in various problems. As a by-product, the error metric suggested in this paper can also be used as a termination condition for an EA application. Hopefully, this study will bring EAs and its research closer to classical optimization studies.

In this paper, a neural network approach is presented to expand the Pareto-optimal front for multiobjective optimization problems. The network is trained using results obtained from the nondominated sorting genetic algorithm (NSGA-II) on a set of well-known benchmark multiobjective problems. Its performance is evaluated against NSGA-II, and the neural network is shown to perform extremely well. Using the same number of function evaluations, the neural network produces many times more non-dominated solutions than NSGA-II.

Despite the existence of a number of procedures for multi-objective optimization using evolutionary algorithms, there is still the need for a systematic and unbiased comparison of different approaches on a carefully chosen set of test problems. In this paper, a hybrid approach using PCX based NSGA- II and sequential quadratic programming (SQP) is applied on 19 benchmark test problems consisting of two, three and five objectives. PCX-NSGA-II is used as a population based algorithm where SQP is used as a local search procedure. A population based approach helps in finding the non-dominated set of solutions with a good spread, whereas SQP improves the obtained set of non-dominated solutions locally. The results obtained by the present approach shows mixed performance on the chosen test problems.

In this paper, we present 3D offline path planner for Unmanned Aerial Vehicles (UAVs) using Multiobjective Evolutionary Algorithms for finding solutions corresponding to conflicting goals of minimizing length of path and maximizing margin of safety. In particular, we have chosen the commonlyused NSGA-II algorithm for this purpose. The algorithm generates a curved path which is represented using B-Spline curves. The control points of the B-Spline curve are the decision variables in the genetic algorithm. In particular, we solve two problems, assuming the normal flight envelope restriction: i) Path planning for UAV when no other constraint is assumed to be present and ii) Path planning for UAV if the vehicle has to necessarily pass through a particular point in the space. The use of a multiobjective evolutionary algorithm helps in generating a number of feasible paths with different trade-offs between the objective functions. The availability of a number of trade-off solutions allows the user to choose a path according to his/her needs easily, thereby making the approach more pragmatic. Although an automated decision-making aid is the next immediate need of research, we defer it for another study.

Preferences are ubiquitous in real life - many problems are over-constrained and would not be solvable if we insist that all our requirements are strictly met. This paper portrays ‘constraints’ in an unconventional perspective, based on the realization that in order to truly maximize/minimize objective function(s), one has to optimize the feasible set. On a broader level, this paper acknowledges the need to move from ‘optimizing the given’, towards ‘designing the optimal’. Here, the constraints are treated as objectives over and above the stated objective(s) and no other restrictions are used to ‘constrain’ the search space. We then evaluate this enhanced set of objectives, in terms of their criticality or redundancy. To this effect, we utilize our earlier proposed dimensionality reduction procedures [3], [7], to obtain a minimal set of objectives, which would characterize the original system with reasonable accuracy (from dimensionality reduction perspective) but with enhanced effectiveness (from a ‘system design’ perspective).

Most real-world search and optimization problems involve multiple conflicting objectives and results in a Pareto-optimal set. Various multi-objective optimization algorithms have been proposed for solving such problems with the goals of finding as many trade-off solutions as possible and maintaining diversity among them. Since last decade, evolutionary multi-objective optimization (EMO) algorithms have been applied successfully to various test and real-world optimization problems. These population based algorithms provide a diverse set of non-dominated solutions. The obtained non-dominated set is close to the true Pareto-optimal front but it's convergence to the true Pareto-optimal front is not guaranteed. Hence to ensure the same, a local search method using classical algorithm can be applied. In the present work, SBX based NSGA-II is used as a population based approach and the sequential quadratic programming (SQP) method is used as a local search procedure. This hybridization of evolutionary and classical algorithms approach provides a confidence of converging near to the true Pareto-optimal set with a good diversity. The proposed procedure is successfully applied to 13 test problems consisting two, three and five objectives. The obtained results validate our motivation of hybridizing evolutionary and classical methods.

In this paper, we borrow the concept of reference direction approach from the multi-criterion decision-making literature and combine it with an EMOprocedure to develop an algorithm for finding a single preferred solution in a multi-objective optimization scenario efficiently. EMO methodologies are adequately used to find a set of representative efficient solutions over the past decade. This study is timely in addressing the issue of optimizing and choosing a single solution using certain preference information. In this approach, the user supplies one or more reference directions in the objective space. The population approach of EMO methodologies is exploited to find a set of efficient solutions corresponding to a number of representative points along the reference direction. By using a utility function, a single solution is chosen for further analysis. This procedure is continued till no further improvement is possible. The working of the procedure is demonstrated on a set of test problems having two to ten objectives and on an engineering design problem. Results are verified with theoretically exact solutions on two-objective test problems.

Simulated binary crossover (SBX) is a real-parameter recombinationoperator which is commonly used in the evolutionary algorithm (EA) literature. The operatorinvolves a parameter which dictates the spread of offspring solutionsvis-a-vis that of the parent solutions. In all applications of SBX sofar, researchers have kept a fixed value throughout a simulation run. In this paper, we suggest a self-adaptive procedure of updating theparameter so as to allow a smooth navigation over the functionlandscape with iteration. Some basic principles of classicaloptimization literature are utilized for this purpose. The resultingEAs are found to produce remarkable and much better results comparedto the original operator having a fixed value of the parameter. Studieson both single and multiple objective optimization problems are madewith success.

In this paper, we present a new design of the Hindi1 keyboard for convenient typing. We describe the ergonomic criterion we have used to evaluate and compare keyboards. This criterion is a mathematical formulation of keyboard optimality in terms of the distribution of the typing effort among the ten fingers, accessibility of commonly used keys and various other factors. Measured against this criterion, our keyboard performs more than twice as well as the standard Hindi keyboard. We also describe a genetic algorithm based optimization framework which we use to arrive at our new keyboard design. Finally, we perform some sensitivity analysis on our optimization procedure and demonstrate that our results conform to intuitive expectations.

Power system operation is subject to many uncertainties since acquired data are subject to inaccuracies due to inaccuracies in the process of measuring and forecasting of input data and changes of unit performance during the period between measuring and operation. In the environmental/economic dispatch problem, both fuel cost and emission are to be simultaneously minimized. In this paper, in order to obtain a solution closer to real-world situations, a constrained Monte Carlo sampling scheme is considered with stochastic decision variables, power system loads and objective functions whereby NSGA-II is used for solving the resulting stochastic environmental/economic dispatch problem. Simulation results presented for the standard IEEE 30-bus system show that the optimized system is reliable if stochastic variables are correlated.

In this paper an algorithm for pruning a set of non-dominated solutions is proposed. The algorithm is based on the crowding distance calculation used in the elitist non-dominated sorting genetic algorithm (NSGA-II). The time complexity class of the new algorithm is estimated and in most cases it is the same as for the original pruning algorithm. Numerical results also support this estimate. For used bi-objective test problems, the proposed pruning algorithm is demonstrated to provide better distribution compared to the original pruning algorithm of NSGA-II. However, with tri-objective test problems there is no improvement and this study reveals that crowding distance does not estimate crowdedness well in this case and presumably also in cases of more objectives.

Despite the existence of a number of procedures for constrained real-parameter optimization using evolutionary algorithms, there is still the need for a systematic and unbiased comparison of different approaches on a carefully chosen set of test problems. In this paper, we suggest a parent centric procedure for constrained real-parameter optimization. The algorithm so developed is applied to a set of 24 test problems and the results are presented. The proposed procedure is able to find the exact optimum within the specified number of function evaluations for 22 of the 24 test problems. In the remaining two problems, the proposed algorithm shows steady progress towards the respective optima, but it was unable to solve within the specified number of evaluations. It is also noteworthy that the algorithm was able to find solutions, better than the ones specified in the original problem description (http://www.ntu.edu.sg/home/EPNSugan/) for a number of test problems.

Nadir point plays an important role in multi-objective optimization because of its importance in estimating the range of objective values corresponding to desired Pareto-optimal solutions and also in using many classical interactive optimization techniques. Since this point corresponds to the worst Pareto-optimal solution of each objective, the task of estimating the nadir point necessitates information about the whole Pareto optimal frontier and is reported to be a difficult task using classical means. In this paper, for the first time, we have proposed a couple of modifications to an existing evolutionary multi-objective optimization procedure to focus its search towards the extreme objective values front-wise. On up to 20-objective optimization problems, both proposed procedures are found to be capable of finding a near nadir point quickly and reliably. Simulation results are interesting and should encourage further studies and applications in estimating the nadir point, a process which should lead to a better interactive procedure of finding and arriving at a desired Pareto-optimal solution.

Evolutionary multi-objective optimization (EMO) methodologies have been amply applied to find a representative set of Pareto-optimal solutions in the past decade and beyond. Although there are advantages of knowing the range of each objective for Pareto-optimality and the shape of the Pareto-optimal frontier itself in a problem for an adequate decision-making, the task of choosing a single preferred Pareto-optimal solution is also an important task which has received a lukewarm attention so far. In this paper, we combine one such preference based strategy with an EMO methodology and demonstrate how, instead of one solution, a preferred set solutions near the reference points can be found parallely. We propose a modified EMO procedure based on the elitist non-dominated sorting GAor NSGA-II. On two-objective to 10-objective optimization problems, the modified NSGA-II approach shows its efficacy in finding an adequate set of Pareto-optimal points. Such procedures will provide the decision-maker with a set of solutions near her/his preference so that a better and a more reliable decision can be made.

This paper introduces a new design methodology (we call it "innovization") in the context of finding new and innovative design principles by means of optimization techniques. Although optimization algorithms are routinely used to find an optimal solution corresponding to an optimization problem, the task of innovization stretches the scope beyond an optimization task and attempts to unveil new, innovative, and important design principles relating to decision variables and objectives, so that a deeper understanding of the problem can be obtained. The variety of problems chosen in the paper and the resulting innovations obtained for each problem amply demonstrate the usefulness of the innovization task. The results should encourage a wide spread applicability of the proposed innovization procedure (which is not simply an optimization procedure) to other problem-solving tasks.

Existing test problems for multi-objective optimization are criticized for not having adequate linkages among variables. In most problems, the Pareto-optimal solutions correspond to a fixed value of certain variables and diversity of solutions comes mainly from a random variation of certain other variables. In this paper, we introduce explicit linkages among variables so as to develop difficult two and multi-objective test problems along the lines of ZDT and DTLZ problems. On a number of such test problems, this paper compares the performance of a number of EMO methodologies having (i) variable-wise versus vector-wise recombination operators and (ii) spatial versus unidirectional recombination operators. Interesting and useful conclusions on the use of above operators are made from the study.

Many engineering optimization tasks involve finding more than one optimum solution. The present study provides a comprehensive review of the existing work done in the field of multi-modal function optimization and provides a critical analysis of the existing methods. Existing niching methods are analyzed and an improved niching method is proposed. To achieve this purpose, we first give an introduction to niching and diversity preservation, followed by discussion of a number of algorithms. Thereafter, a comparison of clearing, clustering, deterministic crowding, probabilistic crowding, restricted tournament selection, sharing, species conserving genetic algorithms is made. A modified niching-based technique -- modified clearing approach -- is introduced and also compared with existing methods. For comparison, a versatile hump test function is also proposed and used together with two other functions. The ability of the algorithms in finding, locating, and maintaining multiple optima is judged using two performance measures: (i) number of peaks maintained, and (ii) computational time. Based on the results, we conclude that the restricted tournament selection and the proposed modified clearing approaches are better in terms of finding and maintaining the multiple optima.

In optimization studies including multi-objective optimization, the main focus is placed on finding the global optimum or global Pareto-optimal solutions, representing the best possible objective values. However, in practice, users may not always be interested in finding the so-called global best solutions, particularly when these solutions are quite sensitive to the variable perturbations which cannot be avoided in practice. In such cases, practitioners are interested in finding the robust solutions which are less sensitive to small perturbations in variables. Although robust optimization is dealt with in detail in single-objective evolutionary optimization studies, in this paper, we present two different robust multi-objective optimization procedures, where the emphasis is to find a robust frontier, instead of the global Pareto-optimal frontier in a problem. The first procedure is a straightforward extension of a technique used for single-objective optimization and the second procedure is a more practical approach enabling a user to set the extent of robustness desired in a problem. To demonstrate the differences between global and robust multi-objective optimization principles and the differences between the two robust optimization procedures suggested here, we develop a number of constrained and unconstrained test problems having two and three objectives and show simulation results using an evolutionary multi-objective optimization (EMO) algorithm. Finally, we also apply both robust optimization methodologies to an engineering design problem.

Robust multi-objective optimization has emerged as an active research. A recent study proposed two different definitions of robust solutions in the context of multi-objective optimization. In this paper, we extend the concepts for finding robust solutions in the presence of active constraints. The meaning of robust solutions for constrained problems is demonstrated by suggesting three test problems and simulating an evolutionary multi-objective optimization method using the two definitions of robustness. The inclusion of constraint handling strategies makes the multi-objective robust optimization procedure more pragmatic and the procedure is now ready to be applied to real-world problems.

Despite the existence of a number of procedures for real-parameter optimization using evolutionary algorithms, there is still a need of a systematic and unbiased comparison of different approaches on a carefully chosen set of test problems. In this paper, we develop a steady-state, population-based optimization algorithm which allows the main search principles to be independently designed. The algorithm so developed is applied to a set of 25 test problems and results on 10 and 30 dimensions are presented. Although the proposed procedure cannot find the exact optimum within the specified number of function evaluations, in most problems, the algorithm shows steady progress towards the optimum. Moreover, it is also observed that the performance of the algorithm does not get affected by the rotation of the functions, discontinuity and embedded noise in function description.

In the evolutionary optimization field, there exist some algorithms taking advantage of the known property of the benchmark functions, such as local optima lying along the coordinate axes, global optimum having the same values for many variables and so on. Multiagent genetic algorithm (MAGA) is an example for this class of algorithms. In this paper, we identify shortcomings associated with the existing test functions. Novel hybrid benchmark functions, whose complexity and properties can be controlled easily, are introduced and several evolutionary algorithms are evaluated with the novel test functions.

Since the suggestion of a computing procedure of multiple Pareto-optimal solutions in multi-objective optimization problems in the early Nineties, researchers have been on the look out for a procedure which is computationally fast and simultaneously capable of finding a well-converged and well-distributed set of solutions. Most multi-objective evolutionary algorithms (MOEAs) developed in the past decade are either good for achieving a well-distributed solutions at the expense of a large computational effort or computationally fast at the expense of achieving a not-so-good distribution of solutions. For example, although the Strength Pareto Evolutionary Algorithm or SPEA (Zitzler and Thiele, 1999) produces a much better distribution compared to the elitist non-dominated sorting GA or NSGA-II (Deb et al., 2002a), the computational time needed to run SPEA is much greater. In this paper, we evaluate a recently-proposed steady-state MOEA (Deb et al., 2003) which was developed based on the epsilon-dominance concept introduced earlier(Laumanns et al., 2002) and using efficient parent and archive update strategies for achieving a well-distributed and well-converged set of solutions quickly. Based on an extensive comparative study with four other state-of-the-art MOEAs on a number of two, three, and four objective test problems, it is observed that the steady-state MOEA is a good compromise in terms of convergence near to the Pareto-optimal front, diversity of solutions, and computational time. Moreover, the epsilon-MOEA is a step closer towards making MOEAs pragmatic, particularly allowing a decision-maker to control the achievable accuracy in the obtained Pareto-optimal solutions.

Evolutionary multi-objective optimization (EMO) may be computationally quite demanding, because instead of searching for a single optimum, one generally wishes to find the whole front of Pareto-optimal solutions. For that reason, parallelizing EMO is an important issue. Since we are looking for a number of Pareto-optimal solutions with different tradeoffs between the objectives, it seems natural to assign different parts of the search space to different processors. We propose the idea of cone separation which is used to divide up the search space by adding explicit constraints for each process. We show that the approach is more efficient than simple parallelization schemes, and that it also works on problems with a non-convex Pareto-optimal front.

Protein fold recognition (PFR) is an important approach to structure discovery without relying on sequence similarity. In pattern recognition terminology, PFR is a multiclass classification problem to be solved by employing feature analysis and pattern classification techniques. This work reformulates PFR into a multiobjective optimization problem and proposes a multiobjective feature analysis and selection algorithm (MOFASA). We use support vector machines as the classifier. Experimental results on the structural classification of protein (SCOP) data set indicate that MOFASA is capable of achieving comparable performances to the existing results. In addition, MOFASA identifies relevant features for further biological analysis.

After demonstrating adequately the usefulness of evolutionary multiobjective optimization (EMO) algorithms in finding multiple Pareto-optimal solutions for static multiobjective optimization problems, there is now a growing need for solving dynamic multiobjective optimization problems in a similar manner. In this paper, we focus on addressing this issue by developing a number of test problems and by suggesting a baseline algorithm. Since in a dynamic multiobjective optimization problem, the resulting Pareto-optimal set is expected to change with time (or, iteration of the optimization process), a suite of five test problems offering different patterns of such changes and different difficulties in tracking the dynamic Pareto-optimal front by a multiobjective optimization algorithm is presented. Moreover, a simple example of a dynamic multiobjective optimization problem arising from a dynamic control loop is presented. An extension to a previously proposed direction-based search method is proposed for solving such problems and tested on the proposed test problems. The test problems introduced in this paper should encourage researchers interested in multiobjective optimization and dynamic optimization problems to develop more efficient algorithms in the near future.

This paper presents a concept of combining genetic algorithms (GAs) with an approximate evaluation technique to achieve a computationally effective search and optimization procedure. The major objective of this work is to enable the use of GAs on computationally expensive problems, while retaining their basic robust search capabilities. Starting with a coarse approximation model of the problems, GAs successively use finer models, thereby allowing the proposed algorithm to find the optimal or a near-optimal solution of computationally expensive problems faster. A general methodology is proposed for combining any approximating technique with GA. The proposed methodology is also tested in conjunction with one particular approximating technique, namely the artificial neural network, on a B-spline curve fitting problem successfully. Savings in the exact function evaluation up to 32% are achieved. The computational advantage demonstrated here should encourage the use of the proposed approach to more complex and computationally demanding real-world problems.

After adequately demonstrating the ability to solve different two-objective optimization problems, multi-objective evolutionary algorithms (MOEAs) must show their efficacy in handling problems having more than two objectives. In this paper, we suggest three different approaches for systematically designing test problems for this purpose. The simplicity of construction, scalability to any number of decision variables and objectives, knowledge of exact shape and location of the resulting Pareto-optimal front, and ability to control difficulties in both converging to the true Pareto-optimal front and maintaining a widely distributed set of solutions are the main features of the suggested test problems. Because of these features, they should be useful in various research activities on MOEAs, such as testing the performance of a new MOEA, comparing different MOEAs, and having a better understanding of the working principles of MOEAs.

The paper follows the line of the design and evaluation of new evolutionary algorithms for constrained multi-objective optimization. The evolutionary algorithm proposed (ENORA) incorporates the Pareto concept of multi-objective optimization with a constraint handling technique and with a powerful diversity mechanism to obtain multiple nondominated solutions through the simple run of the algorithm. Constraint handling is carried out in an evolutionary way and using the min-max formulation, while the diversity technique is based on the partitioning of search space in a set of radial slots along which are positioned the successive populations generated by the algorithm. A set of test problems recently proposed for the evaluation of this kind of algorithm has been used in the evaluation of the algorithm presented. The results obtained with ENORA were very good and considerably better than those obtained with algorithms recently proposed by other authors.

Due to increasing interest in solving real-world optimization problems using evolutionary algorithms (EAs), researchers have recently developed a number of real-parameter genetic algorithms (GAs). In these studies, the main research effort is spent on developing an efficient recombination operator. Such recombination operators use probability distributions around the parent solutions to create an offspring. Some operators emphasize solutions at the center of mass of parents and some around the parents. In this paper, we propose a generic parent-centric recombination operator (PCX) and a steady-state, elite-preserving, scalable, and computationally fast population-alteration model (we call the G3 model). The performance of the G3 model with the PCX operator is investigated on three commonly used test problems and is compared with a number of evolutionary and classical optimization algorithms including other real-parameter GAs with the unimodal normal distribution crossover (UNDX) and the simplex crossover (SPX) operators, the correlated self-adaptive evolution strategy, the covariance matrix adaptation evolution strategy (CMA-ES), the differential evolution technique, and the quasi-Newton method. The proposed approach is found to consistently and reliably perform better than all other methods used in the study. A scale-up study with problem sizes up to 500 variables shows a polynomial computational complexity of the proposed approach. This extensive study clearly demonstrates the power of the proposed technique in tackling real-parameter optimization problems.

Over the past few years, the research on evolutionary algorithms has demonstrated their niche in solving multiobjective optimization problems, where the goal is to find a number of Pareto-optimal solutions in a single simulation run. Many studies have depicted different ways evolutionary algorithms can progress towards the Pareto-optimal set with a widely spread distribution of solutions. However, none of the multiobjective evolutionary algorithms (MOEAs) has a proof of convergence to the true Pareto-optimal solutions with a wide diversity among the solutions. In this paper, we discuss why a number of earlier MOEAs do not have such properties. Based on the concept of epsilon-dominance, new archiving strategies are proposed that overcome this fundamental problem and provably lead to MOEAs that have both the desired convergence and distribution properties. A number of modifications to the baseline algorithm are also suggested. The concept of epsilon-dominance introduced in this paper is practical and should make the proposed algorithms useful to researchers and practitioners alike.

Multi-objective evolutionary algorithms (MOEAs) that use non-dominated sorting and sharing have been criticized mainly for: (1) their O(MN/sup 3/) computational complexity (where M is the number of objectives and N is the population size); (2) their non-elitism approach; and (3) the need to specify a sharing parameter. In this paper, we suggest a non-dominated sorting-based MOEA, called NSGA-II (Non-dominated Sorting Genetic Algorithm II), which alleviates all of the above three difficulties. Specifically, a fast non-dominated sorting approach with O(MN/sup 2/) computational complexity is presented. Also, a selection operator is presented that creates a mating pool by combining the parent and offspring populations and selecting the best N solutions (with respect to fitness and spread). Simulation results on difficult test problems show that NSGA-II is able, for most problems, to find a much better spread of solutions and better convergence near the true Pareto-optimal front compared to the Pareto-archived evolution strategy and the strength-Pareto evolutionary algorithm - two other elitist MOEAs that pay special attention to creating a diverse Pareto-optimal front. Moreover, we modify the definition of dominance in order to solve constrained multi-objective problems efficiently. Simulation results of the constrained NSGA-II on a number of test problems, including a five-objective, seven-constraint nonlinear problem, are compared with another constrained multi-objective optimizer, and the much better performance of NSGA-II is observed.

Self-adaptation is an essential feature of natural evolution. However, in the context of function optimization, self-adaptation features of evolutionary search algorithms have been explored mainly with evolution strategy (ES) and evolutionary programming (EP). In this paper, we demonstrate the self-adaptive feature of real-parameter genetic algorithms (GAs) using a simulated binary crossover (SBX) operator and without any mutation operator. The connection between the working of self-adaptive ESs and real-parameter GAs with the SBX operator is also discussed. Thereafter, the self-adaptive behavior of real-parameter GAs is demonstrated on a number of test problems commonly used in the ES literature. The remarkable similarity in the working principle of real-parameter GAs and self-adaptive ESs shown in this study suggests the need for emphasizing further studies on self-adaptive GAs.

Due to the flexibility in adapting to different fitness landscapes, self-adaptive evolutionary algorithms (SA-EAs) have been gaining popularity in the recent past. In this paper, we postulate the properties that SA-EA operators should have for successful applications in real-valued search spaces. Specifically, population mean and variance of a number of SA-EA operators such as various real-parameter crossover operators and self-adaptive evolution strategies are calculated for this purpose. Simulation results are shown to verify the theoretical calculations. The postulations and population variance calculations explain why self-adaptive genetic algorithms and evolution strategies have shown similar performance in the past and also suggest appropriate strategy parameter values, which must be chosen while applying and comparing different SA-EAs.

In this paper, we provide a systematic comparison of various evolutionary approaches to multiobjective optimization using six carefully chosen test functions. Each test function involves a particular feature that is known to cause difficulty in the evolutionary optimization process, mainly in converging to the Pareto-optimal front (e.g., multimodality and deception). By investigating these different problem features separately, it is possible to predict the kind of problems to which a certain technique is or is not well suited. However, in contrast to what was suspected beforehand, the experimental results indicate a hierarchy of the algorithms under consideration. Furthermore, the emerging effects are evidence that the suggested test functions provide sufficient complexity to compare multiobjective optimizers. Finally, elitism is shown to be an important factor for improving evolutionary multiobjective search.

The experimental results reported in many papers suggest that making an appropriate a priori choice of an evolutionary method for a nonlinear parameter optimization problem remains an open question. It seems that the most promising approach at this stage of research is experimental, involving the design of a scalable test suite of constrained optimization problems, in which many features could be tuned easily. It would then be possible to evaluate the merits and drawbacks of the available methods, as well as to test new methods efficiently. In this paper, we propose such a test-case generator for constrained parameter optimization techniques. This generator is capable of creating various test problems with different characteristics including: 1) problems with different relative sizes of the feasible region in the search space; 2) problems with different numbers and types of constraints; 3) problems with convex or nonconvex evaluation functions, possibly with multiple optima; and 4) problems with highly nonconvex constraints consisting of (possibly) disjoint regions. Such a test-case generator is very useful for analyzing and comparing different constraint-handling techniques.

Goal programming is a technique often used in engineering design activities primarily to find a compromised solution which will simultaneously satisfy a number of design goals. In solving goal programming problems, classical methods reduce the multiple goal-attainment problem into a single objective of minimizing a weighted sum of deviations from goals. In this paper, we pose the goal programming problem as a multi-objective optimization problem of minimizing deviations from individual goals. This procedure eliminates the need of having extra constraints needed with classical formulations and also eliminates the need of any user-defined weight factor for each goal. The proposed technique can also solve goal programming problems having a non-convex trade-off region, which are difficult to solve using classical methods. The efficacy of the proposed method is demonstrated by solving a number of test problems and by solving an engineering design problem. The results suggest that the proposed approach is a unique, effective, and practical tool for solving goal programming problems.

The experimental results reported in many papers suggest that making an appropriate a priori choice of an evolutionary method for a nonlinear parameter optimization problem remains an open question. It seems that the most promising approach at this stage of research is experimental, involving a design of a scalable test suite of constrained optimization problems, in which many features could be easily tuned. Then it would be possible to evaluate merits and drawbacks of the available methods as well as test new methods efficiently. In this paper we discuss a recently proposed test-case generator for constrained parameter optimization techniques. This generator is capable of creating various test cases with different characteristics and is very useful for analyzing and comparing different constraint-handling techniques.

Most real-world search and optimization problems are faced with constraints, which must be satisfied by any acceptable solution. Although a plethora of research is spent on handling constraints in genetic algorithms (GAs), the same is not the case in evolution strategies (ESs). However, this does not say that ESs have not been applied to real-world problems. In fact, in the absence of an efficient constraint-handling technique, ES practitioners have mostly made sure that their ESs started from feasible solutions, a matter which allowed them to apply the commonly-used rejection scheme. In this paper, we borrow a constraint-handling scheme from the GA literature and implement it with standard ES paradigm. The resulting algorithm does not require initial feasible solutions and is found to yield a faster progress in the cylindrical corridor model and be efficient in solving a couple of complicated test problems. The results are interesting and suggest further use of the proposed technique in real-world search and optimization problems.

The paper describes a fuzzy genetic algorithm in which a fuzzy logic controller (FLC) is used with genetic algorithms (GAs) to find obstacle-free paths in a number of find-path problems of a mobile robot. In this algorithm, an obstacle-free direction for the movement of a robot locally is created using an FLC and the extent of travel along obstacle-free direction is determined by a GA. Here, the fuzzy logic approach is used to create initial population and GA crossover and mutation operators. This algorithm is found to perform better than the popular steepest descent approach. The proposed algorithm also finds solutions close to the best known tangent graph with A* algorithm from the accuracy point of view. However, the proposed algorithm finds a near-optimal solution faster than the tangent graph and A* algorithm. Moreover, the proposed approach shows how genetic operators can be modified with problem-specific information to create a search algorithm which is efficient for the particular application.

In this paper, we study the problem features that may cause a multi-objective genetic algorithm (GA) difficulty in converging to the true Pareto-optimal front. Identification of such features helps us develop difficult test problems for multi-objective optimization. Multi-objective test problems are constructed from single-objective optimization problems, thereby allowing known difficult features of single-objective problems (such as multi-modality, isolation, or deception) to be directly transferred to the corresponding multi-objective problem. In addition, test problems having features specific to multi-objective optimization are also constructed. More importantly, these difficult test problems will enable researchers to test their algorithms for specific aspects of multi-objective optimization.

info:eu-repo/semantics/published

Scheduling of a bus transit system must be formulated as an optimization problem, if the level of service to passengers is to be maximized within the available resources. In this paper, we present a formulation of a transit system scheduling problem with the objective of minimizing the overall waiting time of transferring and nontransferring passengers while satisfying a number of resource- and service-related constraints. It is observed that the number of variables and constraints for even a simple transit system (a single bus station with three routes) is too large to tackle using classical mixed-integer optimization techniques. The paper shows that genetic algorithms (GAs) are ideal for these problems, mainly because they (i) naturally handle binary variables, thereby taking care of transfer decision variables, which constitute the majority of the decision variables in the transit scheduling problem; and (ii) allow procedure-based declarations, thereby allowing complex algorithmic approaches (involving if then-else conditions) to be handled easily. The paper also shows how easily the same GA procedure with minimal modifications can handle a number of other more pragmatic extensions to the simple transit scheduling problem: buses with limited capacity, buses that do not arrive exactly as per scheduled times, and a multiple-station transit system having common routes among bus stations. Simulation results show the success of GAs in all these problems and suggest the application of GAs in more complex scheduling problems.

A Markov chain framework is developed for analyzing a wide variety of selection techniques used in genetic algorithms (GAs) and evolution strategies (ESs). Specifically, we consider linear ranking selection, probabilistic binary tournament selection, deterministic s-ary (s = 3,4, …) tournament selection, fitness-proportionate selection, selection in Whitley's GENITOR, selection in (μ, λ)-ES, selection in (μ + λ)-ES, (μ, λ)-linear ranking selection in GAs, (μ + λ)-linear ranking selection in GAs, and selection in Eshelman's CHC algorithm. The analysis enables us to compare and contrast the various selection algorithms with respect to several performance measures based on the probability of takeover. Our analysis is exact—we do not make any assumptions or approximations. Finite population sizes are considered. Our approach is perfectly general, and following the methods of this paper, it is possible to analyze any selection strategy in evolutionary algorithms.

We approach the difficult task of analyzing the complex behavior of even the simplest learning classifier system (LCS) by isolating one crucial subfunction in the LCS learning algorithm: covering through niching. The LCS must maintain a population of diverse rules that together solve a problem (e.g., classify examples). To maintain a diverse population while applying the GAs selection operator, the LCS must incorporate some kind of niching mechanism. The natural way to accomplish niching in an LCS is to force competing rules to share resources (i.e., rewards). This implicit LCS fitness sharing is similar to the explicit fitness sharing used in many niched GAs. Indeed, the LCS implicit sharing algorithm can be mapped onto explicit fitness sharing with a one-to-one correspondence between algorithm components. This mapping is important because several studies of explicit fitness sharing, and of niching in GAs generally, have produced key insights and analytical tools for understanding the interaction of the niching and selection forces. We can now bring those results to bear in understanding the fundamental type of cooperation (a.k.a. weak cooperation) that an LCS must promote.

In trying to solve multiobjective optimization problems, many traditional methods scalarize the objective vector into a single objective. In those cases, the obtained solution is highly sensitive to the weight vector used in the scalarization process and demands that the user have knowledge about the underlying problem. Moreover, in solving multiobjective problems, designers may be interested in a set of Pareto-optimal points, instead of a single point. Since genetic algorithms (GAs) work with a population of points, it seems natural to use GAs in multiobjective optimization problems to capture a number of solutions simultaneously. Although a vector evaluated GA (VEGA) has been implemented by Schaffer and has been tried to solve a number of multiobjective problems, the algorithm seems to have bias toward some regions. In this paper, we investigate Goldberg's notion of nondominated sorting in GAs along with a niche and speciation method to find multiple Pareto-optimal points simultaneously. The proof-of-principle results obtained on three problems used by Schaffer and others suggest that the proposed method can be extended to higher dimensional and more difficult multiobjective problems. A number of suggestions for extension and application of the algorithm are also discussed.