Experienced users often have useful knowledge and intuition in solving real-world optimization problems. User knowledge can be formulated as intervariable relationships to assist an optimization algorithm in finding good solutions faster. Such intervariable interactions can also be automatically learned from high-performing solutions discovered at intermediate iterations in an optimization run—a process called innovization. These relations, if vetted by the users, can be enforced among newly generated solutions to steer the optimization algorithm toward practically promising regions in the search space. Challenges arise for large-scale problems where the number of such variable relationships may be high. This article proposes an interactive knowledge-based evolutionary multiobjective optimization (IK-EMO) framework that extracts hidden variable-wise relationships as knowledge from evolving high-performing solutions, shares them with users to receive feedback, and applies them back to the optimization process to improve its effectiveness. The knowledge extraction process uses a systematic and elegant graph analysis method which scales well with the number of variables. The working of the proposed IK-EMO is demonstrated on three large-scale real-world engineering design problems. The simplicity and elegance of the proposed knowledge extraction process and the achievement of high-performing solutions quickly indicate the power of the proposed framework. The results presented should motivate further such interaction-based optimization studies for their routine use in practice.

This paper proposes a machine learning (ML) based unified innovized progress (UIP) operator to simultaneously enhance the convergence and diversity capabilities of reference vector based evolutionary multi-and many-objective optimization algorithms, namely, RV-EMâOAs. Recent studies have demonstrated that ML intervention could help enhance convergence of RV-EMâOAs by capturing efficient search directions, through mapping of inter-generational solutions along the different reference vectors (RVs). This paper first demonstrates that ML intervention can also help enhance the diversity capability of RV-EMâOAs through mapping of intra-generational solutions across the RVs. Subsequently, the UIP operator integrates the convergence and diversity enhancement capabilities in a manner that is generic -applicable to different RV-EMâOAs, and practicable -not requiring any extra solution evaluations over the base RV-EMâOAs. Based on 24,056 experimental runs on multi-and many-objective problems, the UIP operator, when integrated with different RV-EMâOAs, has provided statistically better performance in about 36% instances, and better or equivalent in about 92% instances, compared to the respective base RV-EMâOAs.

Automated machine learning (AutoML) greatly eases human efforts in architecture engineering. However, mainstream AutoML methods like neural architecture search (NAS) are customized for well-designed search spaces wherein promising architectures are densely distributed. In contrast, AutoML-Zero builds machine-learning algorithms using basic primitives and can explore novel architectures beyond human knowledge. AutoML-Zero shows the potential to deploy machine learning systems by not taking advantage of either feature engineering or architectural engineering. In its current form, it only optimizes a single objective like accuracy and has no mechanism to ensure that the constraints of real-world applications are satisfied. We propose a multi-objective variant of AutoML-Zero called MOAZ, that distributes solutions on a Pareto front by trading off accuracy against the computational complexity of the machine learning algorithm. In addition to generating different Pareto-optimal solutions, MOAZ can effectively explore the sparse search space to improve search efficiency. Experimental results on linear regression tasks show MOAZ reduces the median complexity by 87.4% compared to AutoML-Zero while accelerating the median target performance achievement speed by 82%. In addition, our preliminary results on non-linear regression tasks show the potential for further improvements in search accuracy and for reducing the need for human intervention in AutoML.

Surrogate-based-constrained optimization for some optimization problems involving computationally expensive objective functions and constraints is still a great challenge in the optimization field. Its difficulties are of two primary types. One is how to handle the constraints, especially, equality constraints; another is how to sample a good point to improve the prediction of the surrogates in the feasible region. Overcoming these difficulties requires a reliable constraint-handling method and an efficient infill-sampling strategy. To perform inequality- and equality-constrained optimization of expensive black-box systems, this work proposes a hybrid surrogate-based-constrained optimization method (HSBCO), and the main innovation is that a new constraint-handling method is proposed to map the feasible region into the origin of the Euclidean subspace. Thus, if the constraint violation of an infeasible solution is large, then it is far from the origin in the Euclidean subspace. Therefore, all constraints of the problem can be transformed into an equivalent equality constraint, and the distance between an infeasible point and the origin in the Euclidean subspace represents the constraint violation of the infeasible solution. Based on the distance, the objective function of the problem can be penalized by a Gaussian penalty function, and the original constrained optimization problem becomes an unconstrained optimization problem. Thus, the feasible solutions of the original minimization problem always have a lower objective function value than any infeasible solution in the penalized objective space. To improve the optimization performance, kriging-based efficient global optimization (EGO) is used to find a locally optimal solution in the first phase of HSBCO, and starting from this locally optimal solution, RBF-model-based global search and local search strategies are introduced to seek global optimal solutions. Such a hybrid optimization strategy can help the optimization process converge to the global optimal solution within a given maximum number of function evaluations, as demonstrated in the experimental results on 23 test problems. The method is shown to achieve the global optimum more closely and efficiently than other leading methods.

Evolutionary multiobjective clustering (MOC) algorithms have shown promising potential to outperform conventional single-objective clustering algorithms, especially when the number of clusters k is not set before clustering. However, the computational burden becomes a tricky problem due to the extensive search space and fitness computational time of the evolving population, especially when the data size is large. This article proposes a new, hierarchical, topology-based cluster representation for scalable MOC, which can simplify the search procedure and decrease computational overhead. A coarse-to-fine-trained topological structure that fits the spatial distribution of the data is utilized to identify a set of seed points/nodes, then a tree-based graph is built to represent clusters. During optimization, a bipartite graph partitioning strategy incorporated with the graph nodes helps in performing a cluster ensemble operation to generate offspring solutions more effectively. For the determination of the final result, which is underexplored in the existing methods, the usage of a cluster ensemble strategy is also presented, whether k is provided or not. Comparison experiments are conducted on a series of different data distributions, revealing the superiority of the proposed algorithm in terms of both clustering performance and computing efficiency.

In the past several years, it has become apparent that the effectiveness of Pareto-dominance-based multiobjective evolutionary algorithms deteriorates progressively as the number of objectives in the problem, given by M , grows. This is mainly due to the poor discriminability of Pareto optimality in many-objective spaces (typically M ≥ 4 ). As a consequence, research efforts have been driven in the general direction of developing solution ranking methods that do not rely on Pareto dominance (e.g., decomposition-based techniques), which can provide sufficient selection pressure. However, it is still a nontrivial issue for many existing non-Pareto-dominance-based evolutionary algorithms to deal with unknown irregular Pareto front shapes. In this article, a new many-objective evolutionary algorithm based on the generalization of Pareto optimality (GPO) is proposed, which is simple, yet effective, in addressing many-objective optimization problems. The proposed algorithm used an "( M-1 ) + 1" framework of GPO dominance, ( M-1 )-GPD for short, to rank solutions in the environmental selection step, in order to promote convergence and diversity simultaneously. To be specific, we apply M symmetrical cases of ( M-1 )-GPD, where each enhances the selection pressure of M-1 objectives by expanding the dominance area of solutions, while remaining unchanged for the one objective left out of that process. Experiments demonstrate that the proposed algorithm is very competitive with the state-of-the-art methods to which it is compared, on a variety of scalable benchmark problems. Moreover, experiments on three real-world problems have verified that the proposed algorithm can outperform the others on each of these problems.

Innovization is a task of learning common relationships among some or all of the Pareto-optimal (PO) solutions in multi- and many-objective optimization problems. A recent study has shown that a chronological sequence of nondominated solutions obtained along the successive generations of an optimizer possesses salient patterns that can be learnt using a Machine Learning (ML) model, and can help the offspring solutions progress in useful directions. This article enhances each constitutive module of the above approach, including novel interventions on management of the convergence-diversity tradeoff while mapping the solutions from the previous and current generation; use of a computationally more efficient ML method, namely, Random Forest (RF); and changing the manner and extent to which the learnt ML model is utilized toward advancement of the offspring. The proposed modules constitute what is called the enhanced innovized progress (IP2) operator. To investigate the search efficacy provided by the IP2 operator, it is integrated with multi-and many-objective optimization algorithms, such as NSGA-II, NSGA-III, MOEA/D, and MaOEA-IGD, and tested on a range of two- to ten-objective test problems, and five real-world problems. Since the IP2 operator utilizes the history of gradual and progressive improvements in solutions over generations, without requiring any additional solution evaluations, it opens up a new direction for ML-assisted evolutionary optimization.

Learning effective problem information from already explored search space in an optimization run, and utilizing it to improve the convergence of subsequent solutions, have represented important directions in Evolutionary Multi-objective Optimization (EMO) research. In this article, a machine learning (ML)-assisted approach is proposed that: (a) maps the solutions from earlier generations of an EMO run to the current non-dominated solutions in the decision space ; (b) learns the salient patterns in the mapping using an ML method, here an artificial neural network (ANN); and (c) uses the learned ML model to advance some of the subsequent offspring solutions in an adaptive manner. Such a multi-pronged approach, quite different from the popular surrogate-modeling methods, leads to what is here referred to as the Innovized Progress (IP) operator. On several test and engineering problems involving two and three objectives, with and without constraints, it is shown that an EMO algorithm assisted by the IP operator offers faster convergence behavior, compared to its base version independent of the IP operator. The results are encouraging, pave a new path for the performance improvement of EMO algorithms, and set the motivation for further exploration on more challenging problems.

Neural architecture search (NAS) has emerged as a promising avenue for automatically designing task-specific neural networks. Existing NAS approaches require one complete search for each deployment specification of hardware or objective. This is a computationally impractical endeavor given the potentially large number of application scenarios. In this paper, we propose Neural Architecture Transfer (NAT) to overcome this limitation. NAT is designed to efficiently generate task-specific custom models that are competitive under multiple conflicting objectives. To realize this goal we learn task-specific supernets from which specialized subnets can be sampled without any additional training. The key to our approach is an integrated online transfer learning and many-objective evolutionary search procedure. A pre-trained supernet is iteratively adapted while simultaneously searching for task-specific subnets. We demonstrate the efficacy of NAT on 11 benchmark image classification tasks ranging from large-scale multi-class to small-scale fine-grained datasets. In all cases, including ImageNet, NATNets improve upon the state-of-the-art under mobile settings ( ≤ 600M Multiply-Adds). Surprisingly, small-scale fine-grained datasets benefit the most from NAT. At the same time, the architecture search and transfer is orders of magnitude more efficient than existing NAS methods. Overall, experimental evaluation indicates that, across diverse image classification tasks and computational objectives, NAT is an appreciably more effective alternative to conventional transfer learning of fine-tuning weights of an existing network architecture learned on standard datasets. Code is available at https://github.com/human-analysis/neural-architecture-transfer.

When solving constrained multiobjective optimization problems (CMOPs), the most commonly used way of measuring constraint violation is to calculate the sum of all constraint violations of a solution as its distance to feasibility. However, this kind of constraint violation measure may not reflect the distance of an infeasible solution from feasibility for some problems, for example, when an infeasible solution closer to a feasible region does not have a smaller constraint violation than the one farther away from a feasible region. Unfortunately, no set of artificial benchmark problems focusing on this area exists. To remedy this issue, a set of CMOPs with deceptive constraints is introduced in this article. It is the first attempt to consider CMOPs with deceptive constraints (DCMOPs). Based on our previous work, which designed a set of directed weight vectors to solve CMOPs, this article proposes a cooperative framework with an improved version of directed weight vectors to solve DCMOPs. Specifically, the cooperative framework consists of two switchable phases. The first phase uses two subpopulations-one to explore feasible regions and the other to explore the entire space. The two subpopulations provide useful information about the optimal direction of objective improvement to each other. The second phase aims mainly at finding Pareto-optimal solutions. Then an infeasibility utilization strategy is used to improve the objective function values. The two phases are switchable based on the information found to date at any time in the evolutionary process. The experimental results show that this method significantly outperforms the algorithms with which it is compared on most of the DCMOPs, in terms of reliability and stability in finding a set of well-distributed optimal solutions.

Convolutional neural networks (CNNs) are the backbones of deep learning paradigms for numerous vision tasks. Early advancements in CNN architectures are primarily driven by human expertise and by elaborate design processes. Recently, neural architecture search was proposed with the aim of automating the network design process and generating task-dependent architectures. While existing approaches have achieved competitive performance in image classification, they are not well suited to problems where the computational budget is limited for two reasons: 1) the obtained architectures are either solely optimized for classification performance, or only for one deployment scenario and 2) the search process requires vast computational resources in most approaches. To overcome these limitations, we propose an evolutionary algorithm for searching neural architectures under multiple objectives, such as classification performance and floating point operations (FLOPs). The proposed method addresses the first shortcoming by populating a set of architectures to approximate the entire Pareto frontier through genetic operations that recombine and modify architectural components progressively. Our approach improves computational efficiency by carefully down-scaling the architectures during the search as well as reinforcing the patterns commonly shared among past successful architectures through Bayesian model learning. The integration of these two main contributions allows an efficient design of architectures that are competitive and in most cases outperform both manually and automatically designed architectures on benchmark image classification datasets: CIFAR, ImageNet, and human chest X-ray. The flexibility provided from simultaneously obtaining multiple architecture choices for different compute requirements further differentiates our approach from other methods in the literature.

Many design optimization problems from practice involve a large number of variables. In handling such problems, optimization algorithms, in general, suffer from the well-known ”curse of dimensionality” issue. One of the ways to alleviate the issue somewhat is to use problem information to update the optimization algorithm so that more meaningful solutions are evolved quickly. In this paper, we consider a solid rocket motor design problem involving hundreds of integer variables and two conflicting objectives - minimize the error in matching developed thrust with a desired time-dependent thrust profile and simultaneously minimize the unburnt residue of propellant at the end of the burning process. The evaluation of both objectives involve a detailed burn simulation from the core to the shell of the rocket. After finding a set of trade-off solutions using an evolutionary multi-objective optimization algorithm, we use two learning-based optimization methods (akin to the concept of innovization) to find similar set of solutions using a fraction of the overall solution evaluations. The proposed methods are applied to seven different thrust profiles. Besides solving the large-scale problem quicker, a by-product of our approach is that learnt innovized principles stay as new and innovative knowledge for solving the solid rocket design problem, a matter which is extremely useful to the practitioners.

Convolutional neural networks (CNNs) are the backbones of deep learning paradigms for numerous vision tasks. Early advancements in CNN architectures are primarily driven by human expertise and elaborate design. Recently, neural architecture search (NAS) was proposed with the aim of automating the network design process and generating task-dependent architectures. This paper introduces NSGA-Net -- an evolutionary search algorithm that explores a space of potential neural network architectures in three steps, namely, a population initialization step that is based on prior-knowledge from hand-crafted architectures, an exploration step comprising crossover and mutation of architectures, and finally an exploitation step that utilizes the hidden useful knowledge stored in the entire history of evaluated neural architectures in the form of a Bayesian Network. The integration of these components allows an efficient design of architectures that are competitive and in many cases outperform both manually and automatically designed architectures on CIFAR-10 classification task. The flexibility provided from simultaneously obtaining multiple architecture choices for different compute requirements further differentiates our approach from other methods in the literature.

Multiobjective evolutionary algorithms (MOEAs) have progressed significantly in recent decades, but most of them are designed to solve unconstrained multiobjective optimization problems. In fact, many real-world multiobjective problems contain a number of constraints. To promote research on constrained multiobjective optimization, we first propose a problem classification scheme with three primary types of difficulty, which reflect various types of challenges presented by real-world optimization problems, in order to characterize the constraint functions in constrained multiobjective optimization problems (CMOPs). These are feasibility-hardness, convergence-hardness, and diversity-hardness. We then develop a general toolkit to construct difficulty adjustable and scalable CMOPs (DAS-CMOPs, or DAS-CMaOPs when the number of objectives is greater than three) with three types of parameterized constraint functions developed to capture the three proposed types of difficulty. In fact, the combination of the three primary constraint functions with different parameters allows the construction of a large variety of CMOPs, with difficulty that can be defined by a triplet, with each of its parameters specifying the level of one of the types of primary difficulty. Furthermore, the number of objectives in this toolkit can be scaled beyond three. Based on this toolkit, we suggest nine difficulty adjustable and scalable CMOPs and nine CMaOPs, to be called DAS-CMOP1-9 and DAS-CMaOP1-9, respectively. To evaluate the proposed test problems, two popular CMOEAs-MOEA/D-CDP (MOEA/D with constraint dominance principle) and NSGA-II-CDP (NSGA-II with constraint dominance principle) and two popular constrained many-objective evolutionary algorithms (CMaOEAs)-C-MOEA/DD and C-NSGA-III-are used to compare performance on DAS-CMOP1-9 and DAS-CMaOP1-9 with a variety of difficulty triplets, respectively. The experimental results reveal that mechanisms in MOEA/D-CDP may be more effective in solving convergence-hard DAS-CMOPs, while mechanisms of NSGA-II-CDP may be more effective in solving DAS-CMOPs with simultaneous diversity-, feasibility-, and convergence-hardness. Mechanisms in C-NSGA-III may be more effective in solving feasibility-hard CMaOPs, while mechanisms of C-MOEA/DD may be more effective in solving CMaOPs with convergence-hardness. In addition, none of them can solve these problems efficiently, which stimulates us to continue to develop new CMOEAs and CMaOEAs to solve the suggested DAS-CMOPs and DAS-CMaOPs.

Dynamic multiobjective optimization problems (DMOPs) challenge multiobjective evolutionary algorithms (MOEAs) because those problems change rapidly over time. The class of DMOPs whose objective functions change over time steps, in ways that exhibit some hidden patterns has gained much attention. Their predictability indicates that the problem exhibits some correlations between solutions obtained in sequential time periods. Most of the current approaches use linear models or similar strategies to describe the correlations between historical solutions obtained, and predict the new solutions in the following time period as an initial population from which the MOEA can begin searching in order to improve its efficiency. However, nonlinear correlations between historical solutions and current solutions are more common in practice, and a linear model may not be suitable for the nonlinear case. In this paper, we present a support vector regression (SVR)-based predictor to generate the initial population for the MOEA in the new environment. The basic idea of this predictor is to map the historical solutions into a high-dimensional feature space via a nonlinear mapping, and to do linear regression in this space. SVR is used to implement this process. We incorporate this predictor into the MOEA based on decomposition (MOEA/D) to construct a novel algorithm for solving the aforementioned class of DMOPs. Comprehensive experiments have shown the effectiveness and competitiveness of our proposed predictor, comparing with the state-of-the-art methods.

This paper introduces NSGA-Net --- an evolutionary approach for neural architecture search (NAS). NSGA-Net is designed with three goals in mind: (1) a procedure considering multiple and conflicting objectives, (2) an efficient procedure balancing exploration and exploitation of the space of potential neural network architectures, and (3) a procedure finding a diverse set of trade-off network architectures achieved in a single run. NSGA-Net is a population-based search algorithm that explores a space of potential neural network architectures in three steps, namely, a population initialization step that is based on prior-knowledge from hand-crafted architectures, an exploration step comprising crossover and mutation of architectures, and finally an exploitation step that utilizes the hidden useful knowledge stored in the entire history of evaluated neural architectures in the form of a Bayesian Network. Experimental results suggest that combining the dual objectives of minimizing an error metric and computational complexity, as measured by FLOPs, allows NSGA-Net to find competitive neural architectures. Moreover, NSGA-Net achieves error rate on the CIFAR-10 dataset on par with other state-of-the-art NAS methods while using orders of magnitude less computational resources. These results are encouraging and shows the promise to further use of EC methods in various deep-learning paradigms.

For a many-objective optimization problem with redundant objectives, we propose two novel objective reduction algorithms for linearly and, nonlinearly degenerate Pareto fronts. They are called LHA and NLHA respectively. The main idea of the proposed algorithms is to use a hyperplane with non-negative sparse coefficients to roughly approximate the structure of the PF. This approach is quite different from the previous objective reduction algorithms that are based on correlation or dominance structure. Especially in NLHA, in order to reduce the approximation error, we transform a nonlinearly degenerate Pareto front into a nearly linearly degenerate Pareto front via a power transformation. In addition, an objective reduction framework integrating a magnitude adjustment mechanism and a performance metric [Formula: see text] are also proposed here. Finally, to demonstrate the performance of the proposed algorithms, comparative experiments are done with two correlation-based algorithms, LPCA and NLMVUPCA, and with two dominance-structure-based algorithms, PCSEA and greedy [Formula: see text]MOSS, on three benchmark problems: DTLZ5(I,M), MAOP(I,M), and WFG3(I,M). Experimental results show that the proposed algorithms are more effective.

This paper introduces a differential prediction model to predict the varying Pareto-Optimal Solutions (POS) when solving dynamic multiobjective optimization problems (DMOPs). In dynamic multiobjective optimization problems, several competing objective functions and/or constraints change over time. As a consequence, the Pareto-Optimal Solutions and/or Pareto-Optimal Front may vary over time. The differential prediction model is used to forecast the shift vector in the decision space of the centroid in the population through the centroid's historical locations in three previous environments. This differential prediction model is incorporated into a multiobjective evolutionary algorithm based on decomposition to solve DMOPs. After detecting the environmental change, half of individuals in the population are forecasted their new positions in the decision space by using the differential prediction model and the others' positions are retained. The proposed model is tested on a number of typical benchmark problems with several dynamic characteristics. Experimental results show that the proposed model is competitively in comparisons with the other state-of-the-art models or approaches that were proposed for solving DMOPs.

Multi-objective optimization can be used to solve land-use allocation problems involving multiple conflicting objectives. In this paper, we show how genetic algorithms can be improved in order to effectively and efficiently solve multi-objective land-use allocation problems. Our focus lies on improving crossover and mutation operators of the genetic algorithms. We tested a range of different approaches either based on the literature or proposed for the first time. We applied them to a land-use allocation problem in Switzerland including two conflicting objectives: ensuring compact urban development and reducing the loss of agricultural productivity. We compared all approaches by calculating hypervolumes and by analysing the spread of the produced non-dominated fronts. Our results suggest that a combination of different mutation operators, of which at least one includes spatial heuristics, can help to find well-distributed fronts of non-dominated solutions. The tested modified crossover operators did not significantly improve the results. These findings provide a benchmark for multi-objective optimization of land-use allocation problems with promising prospectives for solving complex spatial planning problems.

Supply-chain management problems are common to most industries and they involve a hierarchy of subtasks, which must be coordinated well to arrive at an overall optimal solution. Such problems involve a hierarchy of decision-makers, each having its own objectives and constraints, but importantly requiring a coordination of their actions to make the overall supply chain process optimal from cost and quality considerations. In this paper, we consider a specific supply-chain management problem from a company, which involves two levels of coordination: (i) yearly strategic planning in which a decision on establishing an association of every destination point with a supply point must be made so as to minimize the yearly transportation cost, and (ii) weekly operational planning in which, given the association between a supply and a destination point, a decision on the preference of available transport carriers must be made for multiple objectives: minimization of transport cost and maximization of service quality and satisfaction of demand at each destination point. We propose a customized multi-objective bilevel evolutionary algorithm, which is computationally tractable. We then present results on state-level and ZIP-level accuracy (involving about 40,000 upper level variables) of destination points over the mainland USA. We compare our proposed method with current non-optimization based practices and report a considerable cost saving.

There are two main tasks involved in addressing a multiobjective optimization problem (MOP) by evolutionary multiobjective (EMO) algorithms: 1) make the population converge close to the Pareto-optimal front and 2) maintain adequate population diversity. However, most state-of-the-art EMO algorithms are designed based on the “convergence first and diversity second” principle. It has been observed that although these EMO algorithms have been successful in optimizing many real-world MOPs, they fail to solve certain problems that feature a severe imbalance between diversity preservation and achieving convergence. This paper characterizes an imbalanced MOP by clearly defining properties and indicating the reasons for the existing EMO algorithms' difficulties in solving them. We then present 14 imbalanced problems, with and without constraints. Computational results using four existing EMO algorithms-elitist non-dominated sorting genetic algorithm (NSGA-II), multiobjective evolutionary algorithm based on decomposition (MOEA/D), strength Pareto evolutionary algorithm 2 (SPEA2), and S metric selection EMO algorithm (SMS-EMOA) and a proposed generalized vector-evaluated genetic algorithm are then presented. It is seen that these EMO algorithms cannot solve these imbalanced problems, but they are able to solve the problems when augmented by multiobjective to multiobjective (M2M), an approach that decomposes the population into several interacting subpopulations. These results and the successful application of the EMO methods with the M2M approach even on standard so-called balanced problems indicate the usefulness of using the M2M approach.

The vast majority of multiobjective evolutionary algorithms presented to date are Pareto-based. Usually, these algorithms perform well for problems with few (two or three) objectives. However, due to the poor discriminability of Pareto-optimality in many-objective spaces (typically four or more objectives), their effectiveness deteriorates progressively as the problem dimension increases. This paper generalizes Pareto-optimality both symmetrically and asymmetrically by expanding the dominance area of solutions to enhance the scalability of existing Pareto-based algorithms. The generalized Pareto-optimality (GPO) criteria are comparatively studied in terms of the distribution of ranks, the ranking landscape, and the convergence of the evolutionary process over several benchmark problems. The results indicate that algorithms equipped with a generalized optimality criterion can acquire the flexibility of changing their selection pressure within certain ranges, and achieve a richer variety of ranks to attain faster and better convergence on some subsets of the Pareto optima. To compensate for the possible diversity loss induced by the generalization, a distributed evolution framework with adaptive parameter setting is also proposed and briefly discussed. Empirical results indicate that this strategy is quite promising in diversity preservation for algorithms associated with the GPO.

This paper presents a foreground detection algorithm that is robust against illumination changes and noise, and provides a novel and practical choice for intelligent video surveillance systems using static cameras. This paper first introduces an online expectation-maximization algorithm that is developed from a basic batch version to update Gaussian mixture models in real time. Then, a spherical K-means clustering method is combined to provide a more accurate direction for the update when illumination is unstable. The combination is supported by the linearity of RGB color reflected from object surfaces, which is both theoretically proved by spectral reflection theory and experimentally validated in several observations. Foreground detection is carried out using a statistical framework with regional judgment. Noise in the detection stage is further reduced by a Bayesian iterative decision-making step. The experiments show that the proposed algorithm outcompetes several classical methods on several datasets, both in detection performance and in robustness to perturbations from illumination changes.

Land use management is increasingly becoming complex as the public and governing bodies demand more accountability and transparency in management practices that simultaneously guarantee sustainable production of goods and continued provision of ecosystem services (i.e., public goods with no markets, such as clean air). In this paper we demonstrate a novel form of decision making that will assist in meeting some of these challenges in ensuring sustainability in land use management. We apply a modified Multi-Objective Evolutionary Algorithm (MOEA), influenced by epigenetic silencing, to a farm case study. The result is a set of time-series, farm management strategies and their related spatial arrangements of land uses that satisfy 14 incommensurable and sometimes conflicting objectives, and spatial constraints. The 14 objectives cover economic (i.e. productivity and financials) and environmental issues. Choosing a single strategy from the set for implementation will require social-ethical value judgment determined from preferences and values of multiple decision-makers. This part of the decision making process is beyond the scope of this paper, but will contribute to ongoing research which will make it possible to fully account for the Triple Bottom Line (TBL), characterised by environmental, economic and social elements.

This work proposes a fast background learning algorithm for foreground detection under changing illumination. Gaussian Mixture Model (GMM) is an effective statistical model in background learning. We first focus on Titterington's online EM algorithm that can be used for real-time unsupervised GMM learning, and then advocate a deterministic data assignment strategy to avoid Bayesian computation. The color of the foreground is apt to be influenced by the environmental illumination that usually produce undesirable effect for GMM updating, however, a collinear feature of pixel intensity under changing light is discovered in RGB color space. This feature is afterward used as a reliable clue to decide which part of mixture to update under changing light. A foreground detection step proposed in early version of this work is employed to extract foreground objects by comparing the estimated background model with the current video frame. Experiments have shown the proposed method is able to achieve satisfactory static background images of scenes as well as is also superior to some mainstream methods in detection performance under both indoor and outdoor scenes.

This paper investigates the issue of evolutionary design of open-ended plants for hybrid dynamical systems, i.e., both their topologies and parameters. Hybrid bond graphs (HBGs) are used to represent dynamical systems involving both continuous and discrete system dynamics. Genetic programming, with some special mechanisms incorporated, is used as a search tool to explore the open-ended design space of hybrid bond graphs. Combination of these two tools, i.e., HBGs and genetic programming, leads to an approach called HBGGP that can automatically generate viable design candidates of hybrid dynamical systems that fulfill predefined design specifications. A comprehensive investigation of a case study of DC-DC converter design demonstrates the feasibility and effectiveness of the HBGGP approach. Important characteristics of the approach are also discussed, with some future research directions pointed out.

In this paper, we present an alternative representation for solving multiobjective Flexible Job-shop Scheduling Problems (FJSP). In FJSP, there may be a choice of machines that can perform any given operation. In order to schedule an operation, it needs to be assigned a machine first, a process known as routing. Most previous approaches to solving FJSP assigned machines to all schedules before beginning any scheduling. In our approach, Adaptive Representation (AdRep), we assign a machine to an operation just at the time it is ready to be scheduled, allowing the routing process to incorporate information from the scheduling environment. Experimental results show that although AdRep performance does not scale as well with problem size as some other approaches that are not simultaneously searching for machine assignments, it is able to find all best published solutions on a three-objective Pareto front including makespan, total workload, and maximum workload on any machine, while its simultaneous routing search opens up new possibilities for optimality of rescheduling in response to machine failure.

This paper presents a background modeling algorithm and a foreground detecting method which is robust against illumination change, providing a novel and practical choice for intelligent video surveillance systems using static cameras. This paper first introduces an online Expectation Maximization algorithm which is developed from the basic batch edition to update the mixture models in real time. Then a spherical K-means clustering method is used to provide more accurate direction for the update of Gaussian Mixture Models after a deep study of RGB space features under illumination changes. Foreground detection is carried out using a statistical framework and RGB pixel intensity judgments. The results show the proposed algorithm outcompete several classic methods in efficiency, accuracy, and robustness to perturbations from illumination changes, on a sampling of problems.

It is popular that there exist multiple objectives in practical control system. To solve this problem, a dynamic multi-objective control algorithm based on NSGA-II is presented. Based on the multi-objective evolutionary algorithm and the tight relation between the system states of the neighboring sampling instants, a multi-objective iterative compatible control algorithm is proposed which can cope with both the convex/non-convex control problem as well as improve the computing speed. Considering the two objectives speed and energy cost in the control of IPMCs propelled robotic fish, the algorithm is successfully applied to it to illuminate its validity. The result also shows the potential for the multi-objective evolutionary algorithm to the real-time control field.

In this paper, we propose a methodology to improve the performance of the standard Differential Evolution (DE) in constraint optimization applications, in terms of accelerating its search speed, and improving the success rate. One critical mechanism embedded in the approach is applying Stochastic Ranking (SR) to rank the whole population of individuals with both objective value and constraint violation to be compared. The ranked population is then in a better shape to provide useful information e.g. direction to guide the search process. The strength of utilizing the directional information can be further controlled by a parameter - population partitioning factor, which is adjusted according to the evolution stage and generations. Because the adaptive adjustment of the parameter is predefined and does not need user input, the resulting algorithm is free of definition of this extra parameter and easier to implement. The performance of the proposed approach, which we call SRaDE (Stochastic Ranking based Adaptive Differential Evolution) is investigated and compared with standard DE. The experimental results show that SRDE significantly outperforms, or at least is comparable with standard DE in all the tested benchmark functions. We also conducted an experiment to compare SRaDE with SRDE - a variant of Stochastic Ranking based Differential Evolution without adaptive adjustment of the population partitioning factor. Experimental results show that SRaDE can also achieve improved performance over SRDE.

In the field of mechanical engineering, synthesizing a mechanism to perform an intended task is deceptively complex. In this paper, a novel approach to automated mechanism synthesis is described which uses an evolutionary search algorithm and a technique called "convertible agents" to simultaneously find the most appropriate mechanism type for a given problem, while finding an optimum set of dimensions for that mechanism to complete a specified task. The search was limited to four-bar, Stephenson, and Watt types of planar, single-degree-of-freedom mechanisms, although the method is readily scalable to include any number of different types. Several case studies are described which illustrate the effectiveness of the method. The developed convertible agent approach is well suited for evolutionary design applications in which there are a small number of distinct topological possibilities each with parametric variables to be optimized.

Use of the Building Block Hypothesis to illuminate GA search behavior, as pursued by J. H. Holland and D. E. Goldberg, invites additional investigation. This paper re-examines the space actually searched by a GA, in light of the Building Block Hypothesis, GA sampling and population size, in an effort to develop more quantitative measures of GA di±culty for problems where building block sizes can be estimated. A Practical Search Index (PSI) is defined, related to the size of the space actively searched by the GA, in terms of sizes and numbers of building blocks. When BBs are hierarchical, the PSI can be used at various stages of BB assembly. Difficulty depends strongly on the sizes of the largest building blocks, rather than on the size of the entire search space, for GAs dominated by crossover. Premature convergence prevails when population size is not adequate to allow sampling and assembly of building blocks. Appropriate sizing depends on balancing the BB sampling and mixing costs. A set of simple GA experiments on classical test functions with clear building block structures (One-Max, RR1, RR2, RRJH, HIFF, etc.) at various population sizes, illustrates the relationship between the PSI, population size, and efficiency of search.

A new two-layer multi-objective compatible control algorithm is proposed for a class of control problems with two conflicting control objectives, control error and energy consumption. The first layer is devoted to obtaining a user’s desired controlled objectives region, assured to be not only achievable but also Pareto-optimal. The second layer is devoted to designing an effective controller by optimizing the most important controlled objective (such as the energy consumption), subject to system constraints from the controlled objectives region in the first layer. This control algorithm provides an effective robust controller design method for multiobjective control problems with precise models and uncertain initial conditions. Simulations illustrate that the two-layer multi-objective compatible control (MOCC) algorithm has some advantages over traditional multi-objective control methods.

This paper describes the use of a genetic algorithm (GA) in tuning a double-level modular fuzzy logic controller (DLMFLC), which can expand its control working zone to a larger spectrum than a single-level FLC. The first-level FLCs are tuned by a GA so that the input parameters of their membership functions and fuzzy rules are optimized according to their individual working zones. The second-level FLC is then used to adjust contributions of the first-level FLCs to the final output signal of the whole controller, i.e., DLMFLC, so that it can function in a wider spectrum covering all individual working zones of the first-level FLCs. The second-level FLC is again optimized by a GA. An inverted pendulum system (IPS) is used to demonstrate the feasibility of the approach.

In the classical binary genetic algorithm, although crossover within a building block (BB) does not always cause a decrease in fitness, any decrease in fitness results from the destruction of some building blocks, in problems where such structures are well defined, such as those considered here. Those crossovers that cause both offspring to be worse, or one to be worse and one unchanged, are here designated as failed crossovers. Counting the failure frequency of singlepoint crossovers performed at each locus reveals something of the BB structure. Guided by the failure record, GA operators could choose appropriate points for crossover, in order to work more efficiently and effectively. Experiments on test

In this paper, we describe a general-purpose, systematic algorithm, consisting of a genetic programming module and a k-nearest neighbor classifier to automatically create artificial features-features that are computer-crafted and may not have a known physical meaning-directly from the reconstructed state-space trajectories of the EEG signals that reveal patterns indicative of epileptic seizure onset. The algorithm was evaluated in three patients and validation experiments were carried out using 267.6 hours of EEG recordings. The results with the artificial features compare favorably with previous benchmark work that used a handcrafted feature.

Most existing research on robust design using evolutionary algorithms (EA) follows the paradigm of traditional robust design, in which parameters of a design solution are tuned to improve the robustness of the system. However, the topological structure of a system may set a limit on the possible robustness achievable through parameter tuning. This paper proposes a new robust design paradigm that exploits the open-ended topological synthesis capability of genetic programming to evolve more robust systems. As a case study, a methodology for automated synthesis of dynamic systems, based on genetic programming and bond graph modeling (GPBG), is applied to evolve robust low-pass and high-pass analog filters. Compared with a traditional robust design approach based on a state-of-the-art real-parameter genetic algorithm (GA), it is shown that open-ended topology search by genetic programming with a fitness criterion rewarding robustness can evolve more robust systems with respect to parameter perturbations than what was achieved through parameter tuning alone, for our test problems.

This paper introduces a redesign method for an air pump system using bond graphs and genetic programming to maximize outflow subject to a constraint specifying maximum power consumption. The redesign process can alter the topological connections among components and can introduce additional components. The air pump system is a mixed-domain system that includes electromagnetic, mechanical and pneumatic elements. Bond graphs are domain independent, allow free composition, and are efficient for classification and analysis of models. Genetic programming is well recognized as a powerful tool for open-ended search. The combination of these two powerful methods, BG/GP, was applied for redesign of an air pump system.

Many current Evolutionary Algorithms (EAs) suffer from a tendency to converge prematurely or stagnate without progress for complex problems. This may be due to the loss of or failure to discover certain valuable genetic material or the loss of the capability to discover new genetic material before convergence has limited the algorithm's ability to search widely. In this paper, the Hierarchical Fair Competition (HFC) model, including several variants, is proposed as a generic framework for sustainable evolutionary search by transforming the convergent nature of the current EA framework into a non-convergent search process. That is, the structure of HFC does not allow the convergence of the population to the vicinity of any set of optimal or locally optimal solutions. The sustainable search capability of HFC is achieved by ensuring a continuous supply and the incorporation of genetic material in a hierarchical manner, and by culturing and maintaining, but continually renewing, populations of individuals of intermediate fitness levels. HFC employs an assembly-line structure in which subpopulations are hierarchically organized into different fitness levels, reducing the selection pressure within each subpopulation while maintaining the global selection pressure to help ensure the exploitation of the good genetic material found. Three EAs based on the HFC principle are tested - two on the even-10-parity genetic programming benchmark problem and a real-world analog circuit synthesis problem, and another on the HIFF genetic algorithm (GA) benchmark problem. The significant gain in robustness, scalability and efficiency by HFC, with little additional computing effort, and its tolerance of small population sizes, demonstrates its effectiveness on these problems and shows promise of its potential for improving other existing EAs for difficult problems. A paradigm shift from that of most EAs is proposed: rather than trying to escape from local optima or delay convergence at a local optimum, HFC allows the emergence of new optima continually in a bottom-up manner, maintaining low local selection pressure at all fitness levels, while fostering exploitation of high-fitness individuals through promotion to higher levels.

This paper describes a unified network synthesis approach for the conceptual stage of mechatronic systems design using bond graphs. It facilitates knowledge interaction with evolutionary computation significantly by encoding the structure of a bond graph in a genetic programming tree representation. On the one hand, since bond graphs provide a succinct set of basic design primitives for mechatronic systems modeling, it is possible to extract useful modular design knowledge discovered during the evolutionary process for design creativity and reusability. On the other hand, design knowledge gained from experience can be incorporated into the evolutionary process to improve the topologically open-ended search capability of genetic programming for enhanced search efficiency and design feasibility. This integrated knowledge-based design approach is demonstrated in a quarter-car suspension control system synthesis and a MEMS bandpass filter design application.

We discuss the hierarchy that is involved in a typical MEMS design and how evolutionary approaches can be used to automate the hierarchical design and synthesis process for MEMS. At the system level, the approach combining bond graphs and genetic programming can lead to satisfactory design candidates of system level models that meet the predefined behavioral specifications for designers to tradeoff. At the physical layout synthesis level, the selection of geometric parameters for component devices is formulated as a constrained optimization problem and addressed using a constrained GA approach. A multiple-resonator microsystem design is used to illustrate the integrated design automation idea using evolutionary approaches.

The wireless access point configuration problem in wireless LAN deployment can be formulated as a nonlinear optimization problem with a variable number of parameters. In this paper, strongly-typed genetic programming is applied to solve an abstract version of this problem successfully. It is argued that this problem can be used as a potential benchmark problem for evaluating techniques and investigating issues in strongly typed genetic programming, topologically open-ended synthesis by genetic programming, and simultaneous topological and parametric search.

The HFC model for evolutionary computation is inspired by the stratified competition often seen in society and biology. Subpopulations are stratified by fitness. Individuals move from low-fitness subpopulations to higher-fitness subpopulations if and only if they exceed the fitness-based admission threshold of the receiving subpopulation, but not of a higher one. HFC's balanced exploration and exploitation, while avoiding premature convergence, is shown on a genetic programming example.

Pattern recognition generally requires that objects be described in terms of a set of measurable features. The selection and quality of the features representing each pattern affect the success of subsequent classification. Feature extraction is the process of deriving new features from original features to reduce the cost of feature measurement, increase classifier efficiency, and allow higher accuracy. Many feature extraction techniques involve linear transformations of the original pattern vectors to new vectors of lower dimensionality. While this is useful for data visualization and classification efficiency, it does not necessarily reduce the number of features to be measured since each new feature may be a linear combination of all of the features in the original pattern vector. Here, we present a new approach to feature extraction in which feature selection and extraction and classifier training are performed simultaneously using a genetic algorithm. The genetic algorithm optimizes a feature weight vector used to scale the individual features in the original pattern vectors. A masking vector is also employed for simultaneous selection of a feature subset. We employ this technique in combination with the k nearest neighbor classification rule, and compare the results with classical feature selection and extraction techniques, including sequential floating forward feature selection, and linear discriminant analysis. We also present results for the identification of favorable water-binding sites on protein surfaces.

Evolutionary programming (EP) has historically used a number of approaches for selection of the mutation step size. Current EP implementations typically use self-adaptive meta-parameters for mutation step size selection. However, one of the potential drawbacks of this scheme is that it is not directly responsive to the variance reduction caused by selection. We investigate an alternate method for mutative step size selection that reacts directly to the variance-reducing effects of selection.

Many black box optimization algorithms have sufficient flexibility to allow them to adapt to the varying circumstances they encounter. These capabilities are of two primary sorts: user-determined choices among alternative parameters, operations, and logic structures; and the algorithm-determined alternative paths chosen during the process of seeking a solution to a particular problem. We discuss the process of algorithm design and operation, with the intent of integrating the seemingly distinct aspects described above within a unified framework. We relate this algorithmic optimization process to the field of dynamic process control. An approach is proposed toward the optimization of a process for controlling a specific class of systems, and its application to dynamic adjustment of the algorithm used in the search problem. An instance of this approach in genetic algorithms is demonstrated. The experimental results show the adaptability and robustness of the proposed approach.

As part of the development of SPRAYTOOL, an accurate simulator of robotically applied spray coatings on sculptured surfaces, a tabular technique was developed for representing with arbitrary accuracy the spray pattern from a robot sprayer. The method allows parameterization of this spray distribution using a single spray test pattern applied by the robot, without symmetry assumptions. The thickness of the test pattern is measured at a user-specified grid of points. These data are used to calculate a least-squares solution to an overdetermined set of linear equations, with optional user-scalable smoothing (via regularization) to suppress effects of measurement noise. In closed-loop testing of the parameterization and simulation process using pseudorandom noise in the measurement data, the solution table entries typically contain less noise than the individual thickness measurements on which they are based.>

The paper begins with a good formal introduction to iterative arrays, discussing briefly their relation to other automata, particularly the "tessellation structures" of Moore [1] and von Neumann [2]. Attention is then restricted to iterative arrays viewed as real-time tape acceptors. The author proves a speedup theorem, which shows how to speed up an array by a constant factor k. The speedup is done by using a length k encoding of the input tapes, and realizing blocks of the array as finite-state machines in a new array which operates k times as fast. The complexity classes of arrays defined by the author ignore the complexity of the modules of which the array is composed. Thus, the speeded-up array is a member of the same class of arrays as the array whose behavior it imitates. The author then proves that the pattern of interconnection ("stencil") of any array may be reduced to allow direct communication only between nearest neighbors without reducing the real-time computing power of the array. This again involves increasing the complexity of the finite-state machines in the array.