Benchmarking is essential to the effective conduct of algorithm development and selection. However, current benchmark suites in constrained multiobjective optimisation, consisting of constrained multiobjective optimisation problems (CMOPs), are lacking in many areas, particularly regarding their diversity and ability to differentiate among algorithms. This limits the ability to adequately conduct benchmarking in this domain. Despite the inadequacies of existing benchmark suites, there do exist CMOPs which are satisfactory according to certain criteria. This paper proposes a benchmark suite called the Selected Test Problems (STEP) suite. It is derived from a methodology for selecting the satisfactory CMOPs from the set of all CMOPs provided by the existing benchmark suites. The STEP suite is analysed in both the feature and performance spaces and is found to be diverse and able to distinguish among algorithms, in addition to other desirable characteristics.

We analyze a fundamental class of multiobjective constrained problems where the objectives are spherical functions and the constraints are convex. As an application from the projection theorem on closed convex sets, we prove that the constrained Pareto set corresponds to the orthogonal projection of the unconstrained Pareto set onto the feasible region. We establish this fundamental geometric property and illustrate its implications using visualizations of Pareto sets and fronts under various constraint configurations. Furthermore, we assess the performance of NSGA-II on these problems, examining its ability to approximate the constrained Pareto set across different dimensions. Our findings highlight the importance of theoretically grounded and understood benchmark problems for assessing algorithmic behavior and contribute to a deeper understanding of constrained multiobjective landscapes.

Constrained multiobjective optimization has gained much interest in the past few years. However, constrained multiobjective optimization problems (CMOPs) are still unsatisfactorily understood. Consequently, the choice of adequate CMOPs for benchmarking is difficult and lacks a formal background. This paper takes a step towards addressing this issue by exploring CMOPs from a performance space perspective. First, it presents a novel performance assessment approach designed explicitly for constrained multiobjective optimization. This methodology offers a first attempt at simultaneously measuring the performance in approximating the Pareto front and constraint satisfaction. Secondly, it proposes an approach to measure the capability of the given optimization problem to differentiate among algorithm performances. Finally, this approach is used to compare eight frequently used artificial test suites of CMOPs. The experimental results reveal which suites are more efficient in discerning between four well-known multiobjective optimization algorithms.

A research problem studied extensively in recent years is the prediction of optimization algorithm performance. A common approach is using the landscape features of optimization problems to train machine learning models. These models are then used to predict algorithm performance. Due to the small number of constrained multiobjective optimization problems (CMOPs) available for benchmarking, training a machine learning model to predict algorithm performance is a hard task. To address this issue, this study uses the functions from the bbob and bbob-constrained benchmark problems to generate new CMOPs. These are then used as additional training examples for the machine learning models. Given the large number of generated CMOPs, the experiments in this study are limited to those with two objectives and two variables. The obtained results are promising. Using additional problems in the training phase improves the predictions in half of the defined classification tasks.

Several test function suites are being used for numerical benchmarking of multiobjective optimization algorithms. While they have some desirable properties, such as well-understood Pareto sets and Pareto fronts of various shapes, most of the currently used functions possess characteristics that are arguably underrepresented in real-world problems such as separability, optima located exactly at the boundary constraints, and the existence of variables that solely control the distance between a solution and the Pareto front. Via the alternative construction of combining existing single-objective problems from the literature, we describe the bbob-biobj test suite with 55 bi-objective functions in continuous domain, and its extended version with 92 bi-objective functions (bbob-biobj-ext). Both test suites have been implemented in the COCO platform for black-box optimization benchmarking and various visualizations of the test functions are shown to reveal their properties. Besides providing details on the construction of these problems and presenting their (known) properties, this article also aims at giving the rationale behind our approach in terms of groups of functions with similar properties, objective space normalization, and problem instances. The latter allows us to easily compare the performance of deterministic and stochastic solvers, which is an often overlooked issue in benchmarking.

Despite the increasing interest in constrained multiobjective optimization in recent years, constrained multiobjective optimization problems (CMOPs) are still insufficiently understood and characterized. For this reason, the selection of appropriate CMOPs for benchmarking is difficult and lacks a formal background. We address this issue by extending landscape analysis to constrained multiobjective optimization. By employing four exploratory landscape analysis techniques, we propose 29 landscape features (of which 19 are novel) to characterize CMOPs. These landscape features are then used to compare eight frequently used artificial test suites against a recently proposed suite consisting of real-world problems based on physical models. The experimental results reveal that the artificial test problems fail to adequately represent some realistic characteristics, such as strong negative correlation between the objectives and the overall constraint violation. Moreover, our findings show that all the studied artificial test suites have advantages and limitations, and that no “perfect” suite exists. Additionally, the effectiveness of the proposed features at predicting algorithm performance is demonstrated for two multiobjective optimization algorithms. Benchmark designers can use the obtained results to select or generate appropriate CMOP instances based on the characteristics they want to explore.

We present concepts and recipes for the anytime performance assessment when benchmarking optimization algorithms in a blackbox scenario. We consider runtime—oftentimes measured in the number of blackbox evaluations needed to reach a target quality—to be a universally measurable cost for solving a problem. Starting from the graph that depicts the solution quality versus runtime, we argue that runtime is the only performance measure with a generic, meaningful, and quantitative interpretation. Hence, our assessment is solely based on runtime measurements. We discuss proper choices for solution quality indicators in single- and multi-objective optimization, as well as in the presence of noise and constraints. We also discuss the choice of the target values, budget-based targets, and the aggregation of runtimes by using simulated restarts, averages, and empirical cumulative distributions which generalize convergence graphs of single runs. The presented performance assessment is to a large extent implemented in the comparing continuous optimizers (COCO) platform freely available athttps://github.com/numbbo/coco.

We introduce two suites of mixed-integer benchmark problems to be used for analyzing and comparing black-box optimization algorithms. They contain problems of diverse difficulties that are scalable in the number of decision variables. The bbob-mixint suite is designed by partially discretizing the established BBOB (Black-Box Optimization Benchmarking) problems. The bi-objective problems from the bbob-biobj-mixint suite are, on the other hand, constructed by using the bbob-mixint functions as their separate objectives. We explain the rationale behind our design decisions and show how to use the suites within the COCO (Comparing Continuous Optimizers) platform. Analyzing two chosen functions in more detail, we also provide some unexpected findings about their properties.

Despite a large interest in real-world problems from the research field of evolutionary optimisation, established benchmarks in the field are mostly artificial. We propose to use game optimisation problems in order to form a benchmark and implement function suites designed to work with the established COCO benchmarking framework. Game optimisation problems are real-world problems that are safe, reasonably complex and at the same time practical, as they are relatively fast to compute. We have created four function suites based on two optimisation problems previously published in the literature (TopTrumps and MarioGAN). For each of the applications, we implemented multiple instances of several scalable single- and multi-objective functions with different characteristics and fitness landscapes. Our results prove that game optimisation problems are interesting and challenging for evolutionary algorithms.

In multiobjective optimization, many techniques are used to visualize the results, ranging from traditional general-purpose data visualization techniques to approaches tailored to the specificities of multiobjective optimization. The number of specialized approaches rapidly grows in the recent years. To assist both the users and developers in this field, we propose a taxonomy of methods for visualizing Pareto front approximations. It builds on the nature of the visualized data and the properties of visualization methods rather than on the employed visual representations. It covers the methods for visualizing individual approximation sets resulting from a single algorithm run as well as multiple approximation sets produced in repeated runs. The proposed taxonomy categories are characterized and illustrated with selected examples of visualization methods. We expect that proposed taxonomy will be insightful to the multiobjective optimization community, make the communication among the participants easier and help focus further development of visualization methods.

In evolutionary multiobjective optimization, it is very important to be able to visualize approximations of the Pareto front (called approximation sets) that are found by multiobjective evolutionary algorithms. While scatter plots can be used for visualizing 2-D and 3-D approximation sets, more advanced approaches are needed to handle four or more objectives. This paper presents a comprehensive review of the existing visualization methods used in evolutionary multiobjective optimization, showing their outcomes on two novel 4-D benchmark approximation sets. In addition, a visualization method that uses prosection (projection of a section) to visualize 4-D approximation sets is proposed. The method reproduces the shape, range, and distribution of vectors in the observed approximation sets well and can handle multiple large approximation sets while being robust and computationally inexpensive. Even more importantly, for some vectors, the visualization with prosections preserves the Pareto dominance relation and relative closeness to reference points. The method is analyzed theoretically and demonstrated on several approximation sets.

To address the needs of rapidly changing energy markets, an energy data management system capable of supporting higher utilization of renewable energy sources is being developed. The system receives flexible offers from producers and consumers of energy, aggregates them on a regional level and schedules the aggregated flexible offers to balance forecast energy supply and demand. This paper focuses on formulating and solving the optimization problem of scheduling aggregated flexible offers within such a system. Three metaheuristic scheduling algorithms (a randomized greedy search, an evolutionary algorithm and a hybrid between the two) tailored to this problem are introduced and their performance is assessed on a benchmark test problem and two realistic problems. The best results are achieved by the evolutionary algorithms, which can efficiently handle thousands of aggregated flex-offers.

In ideal multiobjective optimization, the result produced by an optimizer is a set of nondominated solutions approximating the Pareto optimal front. Visualization of this approximation set can help assess its quality as well as present various features of the problem. Most often, scatter plots are used to visualize 2D and 3D approximation sets, while no scatter plot equivalent exists for visualization in higher dimensions. This paper presents a method for visualizing 4D approximation sets which performs dimension reduction using prosections (projections of a section). The method yields a prosection matrix---a matrix of intuitive 3D scatter plots that well reproduce the shape, range and distribution of vectors in the observed approximation set. The performance of visualization with prosections is analyzed theoretically and demonstrated on two examples with approximation sets of state-of-the-art test optimization problems.

Tuning parameters of an evolutionary algorithm is the essential phase of a problem solving process since the parameter values significantly influence the algorithm efficiency. A traditional parameter tuning approach finds a setting of parameter values that is then used for solving various problem instances. Clearly, such parameter values may not perform well on specific problem instances. This paper suggests finding several parameter settings which are suitable for specific problem instances. However, this is not aimed at the level of each individual instance, but rather for specific types of problem instances. A new problem instance can then be solved using the tuned parameter values for its type. We demonstrate the approach by tuning parameters of an evolutionary algorithm for commodity transportation optimization with very heterogeneous problem instances. Numerical experiments show that the procedure improves the algorithm performance. Moreover, the analysis of empirical results reveals that there exist relations between the tuned parameter values and that they vary over types of problem instances.