Solving real-world optimization problems in engineering and design involves various practical challenges. They include simultaneously optimizing multiple conflicting objective functions that may involve computationally expensive simulations. Failed simulations introduce another practical challenge, as it is not always possible to set constraints a priori to avoid failed simulations. Failed simulations are typically ignored during optimization, which leads to wasting computation resources. When the optimization problem has multiple objective functions, failed simulations can also be misleading for the decision maker while choosing the most preferred solution. Utilizing data collected from previous simulations and enabling the optimization algorithm to avoid failed simulations can reduce the computational requirements. We consider data-driven multiobjective optimization of the diffusor of an axial pump and propose an approach to reduce the number of solutions that fail in expensive computational fluid dynamics simulations. The proposed approach utilizes Kriging surrogate models to approximate the objective functions and is inexpensive to evaluate. We utilize a probabilistic selection approach with constraints in a multiobjective evolutionary algorithm to find solutions with better objective function values, lower uncertainty, and lower probability of failing. Finally, a domain expert chooses the most preferred solution using one’s preferences. Numerical tests show significant improvement in the ratio of feasible solutions to all the available solutions without special treatment of failed simulations. The solutions also have a higher quality (hypervolume) and accuracy than the other tested approaches. The proposed approach provides an efficient way of reducing the number of failed simulations and utilizing offline data in multiobjective design optimization.

No abstract available.

For offline data-driven multiobjective optimization problems (MOPs), no new data is available during the optimization process. Approximation models (or surrogates) are first built using the provided offline data, and an optimizer, for example, a multiobjective evolutionary algorithm, can then be utilized to find Pareto optimal solutions to the problem with surrogates as objective functions. In contrast to online data-driven MOPs, these surrogates cannot be updated with new data and, hence, the approximation accuracy cannot be improved by considering new data during the optimization process. Gaussian process regression (GPR) models are widely used as surrogates because of their ability to provide uncertainty information. However, building GPRs becomes computationally expensive when the size of the dataset is large. Using sparse GPRs reduces the computational cost of building the surrogates. However, sparse GPRs are not tailored to solve offline data-driven MOPs, where good accuracy of the surrogates is needed near Pareto optimal solutions. Treed GPR (TGPR-MO) surrogates for offline data-driven MOPs with continuous decision variables are proposed in this paper. The proposed surrogates first split the decision space into subregions using regression trees and build GPRs sequentially in regions close to Pareto optimal solutions in the decision space to accurately approximate tradeoffs between the objective functions. TGPR-MO surrogates are computationally inexpensive because GPRs are built only in a smaller region of the decision space utilizing a subset of the data. The TGPR-MO surrogates were tested on distance-based visualizable problems with various data sizes, sampling strategies, numbers of objective functions, and decision variables. Experimental results showed that the TGPR-MO surrogates are computationally cheaper and can handle datasets of large size. Furthermore, TGPR-MO surrogates produced solutions closer to Pareto optimal solutions compared to full GPRs and sparse GPRs.

Optimisation problems often have multiple conflicting objectives that can be computationally and/or financially expensive. Mono-surrogate Bayesian optimisation (BO) is a popular model-based approach for optimising such black-box functions. It combines objective values via scalarisation and builds a Gaussian process (GP) surrogate of the scalarised values. The location which maximises a cheap-to-query acquisition function is chosen as the next location to expensively evaluate. While BO is an effective strategy, the use of GPs is limiting. Their performance decreases as the problem input dimensionality increases, and their computational complexity scales cubically with the amount of data. To address these limitations, we extend previous work on BO by density-ratio estimation (BORE) to the multi-objective setting. BORE links the computation of the probability of improvement acquisition function to that of probabilistic classification. This enables the use of state-of-the-art classifiers in a BO-like framework. In this work we present MBORE: multi-objective Bayesian optimisation by density-ratio estimation, and compare it to BO across a range of synthetic and real-world benchmarks. We find that MBORE performs as well as or better than BO on a wide variety of problems, and that it outperforms BO on high-dimensional and real-world problems.

Visualizing the search behavior of a series of points or populations in their native domain is critical in understanding biases and attractors in an optimization process. Distance-based many-objective optimization test problems have been developed to facilitate visualization of search behavior in a 2-D design space with arbitrarily many objective functions. Previous works have proposed a few commonly seen problem characteristics into this problem framework, such as the definition of disconnected Pareto sets and dominance resistant regions of the design space. The authors’ previous work has advanced this research further by providing a problem generator to automatically create user-defined problem instances featuring any combination of these problem features as well as newly introduced ones, such as landscape discontinuities, varying objective ranges, and neutrality. This work makes a number of additional contributions including the proposal of an enhanced, open-source feature-rich problem generator that can create user-defined problem instances exhibiting a range of problem features—some of which are newly introduced here or form extensions of existing features. A comprehensive validation of the problem generator is also provided using popular multiobjective optimization algorithms, and some problem generator settings to create instances exhibiting different challenges for an optimizer are identified.

In offline data-driven multiobjective optimization, no new data are available during the optimization process. Approximation models, also known as surrogates, are built using the provided offline data. A multiobjective evolutionary algorithm can be utilized to find solutions by using these surrogates. The accuracy of the approximated solutions depends on the surrogates and approximations typically involve uncertainties. In this article, we propose probabilistic selection approaches that utilize the uncertainty information of the Kriging models (as surrogates) to improve the solution process in offline data-driven multiobjective optimization. These approaches are designed for decomposition-based multiobjective evolutionary algorithms and can, thus, handle a large number of objectives. The proposed approaches were tested on distance-based visualizable test problems and the DTLZ suite. The proposed approaches produced solutions with a greater hypervolume, and a lower root mean squared error compared to generic approaches and a transfer learning approach that do not use uncertainty information.

Scalarizing functions have been widely used to convert a multiobjective optimization problem into a single objective optimization problem. However, their use in solving computationally expensive multi-and many-objective optimization problems using Bayesian multiobjective optimization is scarce. Scalarizing functions can play a crucial role on the quality and number of evaluations required when doing the optimization. In this article, we compare 15 different scalarizing functions in the framework of Bayesian multiobjective optimization and build Gaussian process models on them. We use the expected improvement as infill criterion (or acquisition function) to update the models. In particular, we analyze the performance of different scalarizing functions on several benchmark problems with different number of objectives to be optimized. The review and experiments on different functions provide useful insights in using a scalarizing function, especially for problems with a large number of objectives.

We formulate and solve a real-world shape design optimization problem of an air intake ventilation system in a tractor cabin by using a preference-based surrogate-assisted evolutionary multi-objective optimization algorithm. We are motivated by practical applicability and focus on two main challenges faced by practitioners in industry: 1) meaningful formulation of the optimization problem reflecting the needs of a decision maker and 2) finding a desirable solution based on a decision maker's preferences when solving a problem with computationally expensive function evaluations. For the first challenge, we describe the procedure of modelling a component in the air intake ventilation system with commercial simulation tools. The problem to be solved involves time consuming computational fluid dynamics simulations. Therefore, for the second challenge, we extend a recently proposed Kriging-assisted evolutionary algorithm K-RVEA to incorporate a decision maker's preferences. Our numerical results indicate efficiency in using the computing resources available and the solutions obtained reflect the decision maker's preferences well. Actually, two of the solutions dominate the baseline design (the design provided by the decision maker before the optimization process). The decision maker was satisfied with the results and eventually selected one as the final solution.

In optimiser analysis and design it is informative to visualise how a search point/population moves through the design space over time. Visualisable distance-based many-objective optimisation problems have been developed whose design space is in two-dimensions with arbitrarily many objective dimensions. Previous work has shown how disconnected Pareto sets may be formed, how problems can be projected to and from arbitrarily many design dimensions, and how dominance resistant regions of design space may be defined. Most recently, a test suite has been proposed using distances to lines rather than points. However, active use of visualisable problems has been limited. This may be because the type of problem characteristics available has been relatively limited compared to many practical problems (and non-visualisable problem suites). Here we introduce the mechanisms required to embed several widely seen problem characteristics in the existing problem framework. These include variable density of solutions in objective space, landscape discontinuities, varying objective ranges, neutrality, and non-identical disconnected Pareto set regions. Furthermore, we provide an automatic problem generator (as opposed to hand-tuned problem definitions). The flexibility of the problem generator is demonstrated by analysing the performance of popular optimisers on a range of sampled instances.

Many-objective optimization problems (MaOPs) contain four or more conflicting objectives to be optimized. A number of efficient decomposition-based evolutionary algorithms have been developed in the recent years to solve them. However, computationally expensive MaOPs have been scarcely investigated. Typically, surrogate-assisted methods have been used in the literature to tackle computationally expensive problems, but such studies have largely focused on problems with 1-3 objectives. In this paper, we present an approach called hybrid surrogate-assisted many-objective evolutionary algorithm to solve computationally expensive MaOPs. The key features of the approach include: 1) the use of multiple surrogates to effectively approximate a wide range of objective functions; 2) use of two sets of reference vectors for improved performance on irregular Pareto fronts (PFs); 3) effective use of archive solutions during offspring generation; and 4) a local improvement scheme for generating high quality infill solutions. Furthermore, the approach includes constraint handling which is often overlooked in contemporary algorithms. The performance of the approach is benchmarked extensively on a set of unconstrained and constrained problems with regular and irregular PFs. A statistical comparison with the existing techniques highlights the efficacy and potential of the approach.

Most evolutionary optimization algorithms assume that the evaluation of the objective and constraint functions is straightforward. In solving many real-world optimization problems, however, such objective functions may not exist. Instead, computationally expensive numerical simulations or costly physical experiments must be performed for fitness evaluations. In more extreme cases, only historical data are available for performing optimization and no new data can be generated during optimization. Solving evolutionary optimization problems driven by data collected in simulations, physical experiments, production processes, or daily life are termed data-driven evolutionary optimization. In this paper, we provide a taxonomy of different data driven evolutionary optimization problems, discuss main challenges in data-driven evolutionary optimization with respect to the nature and amount of data, and the availability of new data during optimization. Real-world application examples are given to illustrate different model management strategies for different categories of data-driven optimization problems.

We consider multiobjective optimization problems where objective functions have different (or heterogeneous) evaluation times or latencies. This is of great relevance for (computationally) expensive multiobjective optimization as there is no reason to assume that all objective functions should take an equal amount of time to be evaluated (particularly when objectives are evaluated separately). To cope with such problems, we propose a variation of the Kriging-assisted reference vector guided evolutionary algorithm (K-RVEA) called heterogeneous K-RVEA (short HK-RVEA). This algorithm is a merger of two main concepts designed to account for different latencies: A single-objective evolutionary algorithm for selecting training data to train surrogates and K-RVEA's approach for updating the surrogates. HK-RVEA is validated on a set of biobjective benchmark problems varying in terms of latencies and correlations between the objectives. The results are also compared to those obtained by previously proposed strategies for such problems, which were embedded in a non-surrogate-assisted evolutionary algorithm. Our experimental study shows that, under certain conditions, such as short latencies between the two objectives, HK-RVEA can outperform the existing strategies as well as an optimizer operating in an environment without latencies.

We propose a surrogate-assisted reference vector guided evolutionary algorithm (EA) for computationally expensive optimization problems with more than three objectives. The proposed algorithm is based on a recently developed EA for many-objective optimization that relies on a set of adaptive reference vectors for selection. The proposed surrogate-assisted EA (SAEA) uses Kriging to approximate each objective function to reduce the computational cost. In managing the Kriging models, the algorithm focuses on the balance of diversity and convergence by making use of the uncertainty information in the approximated objective values given by the Kriging models, the distribution of the reference vectors as well as the location of the individuals. In addition, we design a strategy for choosing data for training the Kriging model to limit the computation time without impairing the approximation accuracy. Empirical results on comparing the new algorithm with the state-of-the-art SAEAs on a number of benchmark problems demonstrate the competitiveness of the proposed algorithm.

We tackle three different challenges in solving a real-world industrial problem: formulating the optimization problem, connecting different simulation tools and dealing with computationally expensive objective functions. The problem to be optimized is an air intake ventilation system of a tractor and consists of three computationally expensive objective functions. We describe the modeling of the system and its numerical evaluation with a commercial software. To obtain solutions in few function evaluations, a recently proposed surrogate-assisted evolutionary algorithm K-RVEA is applied. The diameters of four different outlets of the ventilation system are considered as decision variables. From the set of nondominated solutions generated by K-RVEA, a decision maker having substance knowledge selected the final one based on his preferences. The final selected solution has better objective function values compared to the baseline solution of the initial design. A comparison of solutions with K-RVEA and RVEA (which does not use surrogates) is also performed to show the potential of using surrogates.