In this paper, we address the problem of efficiently computing finite-size approximations of the set of -locally optimal solutions of a given multi-objective optimization problem (MOP). Such sets are in particular interesting in the context of multi-objective multimodal optimization (MMO). To this end, we first propose a bounded archiver, ArchiveUpdateLQ,∈B, that is, a modification of a previously proposed unbounded archiver. These archivers can be used as external archivers to in principle any multi-objective evolutionary algorithm (MOEA). In order to reduce the computational cost compared to such archive equipped MOEAs, we propose, in a next step LQ,∈MOEA. This evolutionary algorithm directly uses ArchiveUpdateLQ,∈B for the selection process and hence does not need an external archive for the computation of ∈-locally optimal solutions. We further propose a hybrid of LQ,∈MOEA with a multi-objective continuation method, which significantly improves the accuracy of the obtained solutions in case the gradient information is at hand. Finally, we show some numerical results that demonstrate the benefit of both the bounded archiver and the new MOEAs.

In many optimization domains, there are multiple different solvers that contribute to the overall state-of-the-art, each performing better on some, and worse on other types of problem instances. Metaalgorithmic approaches, such as instance-based algorithm selection, configuration and scheduling, aim to close this gap by extracting the most performance possible from a set of (configurable) optimizers. In this context, the best performing individual algorithms are often hand-crafted hybrid heuristics which perform many restarts of fast local optimization approaches. However, data-driven techniques to create optimized restart schedules have not yet been extensively studied.

In this article, we address the problem of computing all locally optimal solutions of a given multiobjective problem whose images are sufficiently close to the Pareto front. Such$\epsilon $-locally optimal solutions are particularly interesting in the context of multiobjective multimodal optimization (MMO). To accomplish this task, we first define a new set of interest,$L_{Q,\epsilon }$, that is strongly related to the recently proposed set of$\epsilon $-acceptable solutions. Next, we propose a new unbounded archiver,$ArchiveUpdateL_{Q,\epsilon }$, aiming to capture$L_{Q,\epsilon }$in the limit. This archiver can in principle be used in combination with any multiobjective evolutionary algorithm (MOEA). Further, we equip numerous MOEAs with$ArchiveUpdateL_{Q,\epsilon }$, investigate their performances across several benchmark functions, and compare the enhanced MOEAs with their archive-free counterparts. For our experiments, we utilize the well-established metrics HV, IGDX, and$\Delta _{p}$. Additionally, we propose and use a new performance indicator,$I_{\mathrm { EDR}}$, which results in comparable performances but which is applicable to problems defined in higher dimensions (in particular in decision variable space).

Artificial benchmark functions are commonly used in optimization research because of their ability to rapidly evaluate potential solutions, making them a preferred substitute for real-world problems. However, these benchmark functions have faced criticism for their limited resemblance to real-world problems. In response, recent research has focused on automatically generating new benchmark functions for areas where established test suites are inadequate. These approaches have limitations, such as the difficulty of generating new benchmark functions that exhibit exploratory landscape analysis (ELA) features beyond those of existing benchmarks.

Abstract Single-objective continuous optimization can be challenging, especially when dealing with multimodal problems. This work sheds light on the effects that multi-objective optimization may have in the single-objective space. For this purpose, we examine the inner mechanisms of the recently developed sophisticated local search procedure SOMOGSA. This method solves multimodal single-objective continuous optimization problems based on first expanding the problem with an additional objective (e.g., a sphere function) to the bi-objective domain and subsequently exploiting local structures of the resulting landscapes. Our study particularly focuses on the sensitivity of this multiobjectivization approach w.r.t. (1) the parametrization of the artificial second objective, as well as (2) the position of the initial starting points in the search space. As SOMOGSA is a modular framework for encapsulating local search, we integrate Nelder–Mead local search as optimizer in the respective module and compare the performance of the resulting hybrid local search to its original single-objective counterpart. We show that the SOMOGSA framework can significantly boost local search by multiobjectivization. Hence, combined with more sophisticated local search and metaheuristics, this may help solve highly multimodal optimization problems in the future.

Recent advances in the visualization of continuous multimodal multi-objective optimization (MMMOO) landscapes brought a new perspective to their search dynamics. Locally eficient (LE) sets, often considered as traps for local search, are rarely isolated in the decision space. Rather, intersections by superposing attraction basins lead to further solution sets that at least partially contain better solutions. The Multi-Objective Gradient Sliding Algorithm (MOGSA) is an algorithmic concept developed to exploit these superpositions. While it has promising performance on many MMMOO problems with linear LE sets, closer analysis of MOGSA revealed that it does not sufficiently generalize to a wider set of test problems. Based on a detailed analysis of shortcomings of MOGSA, we propose a new algorithm, the Multi-Objective Landscape Explorer (MOLE). It is able to efficiently model and exploit LE sets in MMMOO problems. An implementation of MOLE is presented for the bi-objective case, and the practicality of the approach is shown in a benchmarking experiment on the Bi-Objective BBOB testbed.

Abstract Hyperparameter optimization (HPO) is a key component of machine learning models for achieving peak predictive performance. While numerous methods and algorithms for HPO have been proposed over the last years, little progress has been made in illuminating and examining the actual structure of these black-box optimization problems. Exploratory landscape analysis (ELA) subsumes a set of techniques that can be used to gain knowledge about properties of unknown optimization problems. In this paper, we evaluate the performance of five different black-box optimizers on 30 HPO problems, which consist of two-, three- and five-dimensional continuous search spaces of the XGBoost learner trained on 10 different data sets. This is contrasted with the performance of the same optimizers evaluated on 360 problem instances from the black-box optimization benchmark (BBOB). We then compute ELA features on the HPO and BBOB problems and examine similarities and differences. A cluster analysis of the HPO and BBOB problems in ELA feature space allows us to identify how the HPO problems compare to the BBOB problems on a structural meta-level. We identify a subset of BBOB problems that are close to the HPO problems in ELA feature space and show that optimizer performance is comparably similar on these two sets of benchmark problems. We highlight open challenges of ELA for HPO and discuss potential directions of future research and applications.

Traditionally, visualizing benchmark problems is an integral task in the domain of evolutionary algorithms development. Researchers get inspired for new search heuristics by challenges observed in functional landscapes. Moreover, landscape characteristics, features, and even terminology to describe them are derived from visualizations. And most importantly, benchmark designers need visualizations for identifying diverse problems that potentially challenge different aspects of optimization algorithms. As easy as it is to visualize single-objective problems, until recently there were hardly any approaches for gaining similar insights for multi-objective problems. Also, there have been no seamlessly accessible tools to support such visualizations. This article presents a comprehensive overview of the available visualization techniques from literature, including two interactive techniques to visualize 3-D problems, as well as two novel techniques which are suitable to scale some visualization properties to even higher-dimensional spaces. All presented techniques are integrated into a single tool, the moPLOT-dashboard, which enables users to perform landscape analyses in an interactive manner. Finally, the value of the tool and the visualizations is demonstrated in a series of usage scenarios on well-known benchmark problems.