We address the challenging problem of multiobjective optimization via stochastic simulation over a discrete design space. We consider the setting where objective functions are expensive black-box simulations corrupted by heteroscedastic noise, and the decision space is usually too large for exhaustive enumeration. Existing methods often struggle to balance three competing needs: scalable surrogate modeling on discrete domains, principled handling of simulation noise (specifically regarding the uncertainty of the current best solution), and efficient navigation of the multiobjective landscape. Our proposed framework extends the single-objective Complete Expected Improvement acquisition function to the bi-objective case. Our contribution is threefold: (1) we employ Gaussian Markov Random Field surrogates to exploit the integer lattice structure; (2) we use ParEGO-style scalarizations but restrict them to linear to preserve the Gaussianity of the posterior, allowing us to derive a closed-form scalarized acquisition function that explicitly accounts for the covariance between the candidate solution and the noisy incumbent. (3) To maximize this acquisition function, we integrate a discrete Genetic Algorithm with specialized local and jump mutation operators as the inner optimizer. We benchmark our approach against an adaptation of state-of-the-art methods on noisy variants of standard test functions, showing faster early convergence while retaining computational tractability.

Real-world problems often consist of multiple conflicting objectives to be optimized simultaneously, featuring a set of Pareto-optimal solutions. Estimating the entire Pareto front can be computationally expensive, and is not always necessary, as decision makers (DMs) will likely be interested only in specific regions of the Pareto front. In the absence of knowledge about the DM preferences, the so-called knees in the Pareto front are considered to be particularly attractive. In this article, we propose using Thompson sampling in the Bayesian optimization framework to estimate the location of the knee regions in a data-efficient manner. Our experimental results show that the proposed methods accurately locate the knee regions after a very small number of evaluations, providing a computationally efficient approach to single- and multiknee detection in multiobjective optimization.

We present Parallel Feasible Pareto Frontier Entropy Search ($\{\mathrm{PF}\}^2$ES) — a novel information-theoretic acquisition function for multi-objective Bayesian optimization supporting unknown constraints and batch queries. Due to the complexity of characterizing the mutual information between candidate evaluations and (feasible) Pareto frontiers, existing approaches must either employ crude approximations that significantly hamper their performance or rely on expensive inference schemes that substantially increase the optimization’s computational overhead. By instead using a variational lower bound, $\{\mathrm{PF}\}^2$ES provides a low-cost and accurate estimate of the mutual information. We benchmark $\{\mathrm{PF}\}^2$ES against other information-theoretic acquisition functions, demonstrating its competitive performance for optimization across synthetic and real-world design problems.

Open-set recognition (OSR) has achieved significant importance in recent years. For a robust recognition system, we need to identify the right class from a myriad of knowns and unknowns. In this work, we build and compare OSR systems for patient activity recognition (PAR) using compact radar sensors in a hospital setting. Radar sensors are an important part of a privacy-preserving monitoring system. Specifically, the proposed approach is based on a deep discriminative representation network (DDRN) trained using the large margin cosine loss (LMCL) and triplet loss (TL). A probability of an inclusion model in the embedding space based on the Weibull distribution is able to separate knowns from unknowns. This overall approach limits the risk of open space and enables us to easily identify any unknown activities. Our experiments show that the proposed approach is significantly better for open-set human activity recognition (HAR) with radar when compared with the state-of-the-art open-set approaches.

We study the inference of mean-variance robustness measures to quantify input uncertainty under the Gaussian Process (GP) framework. These measures are widely used in applications where the robustness of the solution is of interest, for example, in engineering design. While the variance is commonly used to characterize the robustness, Bayesian inference of the variance using GPs is known to be challenging. In this paper, we propose a Spectral Representation of Robustness Measures based on the GP’s spectral representation, i.e., an analytical approach to approximately infer both robustness measures for normal and uniform input uncertainty distributions. We present two approximations based on different Fourier features and compare their accuracy numerically. To demonstrate their utility and efficacy in robust Bayesian Optimization, we integrate the analytical robustness measures in three standard acquisition functions for various robust optimization formulations. We show their competitive performance on numerical benchmarks and real-life applications.

This work develops and benchmarks surrogate models for Dynamic Phasor (DP) simulation of electrical drives. DP simulations of complex systems may be time-consuming due to the increased number of equations. Thus, it is desirable to have a data-driven approach to compute the critical state/control variables and power losses. The surrogate models are intended to be used as a steady-state equivalent of the DP simulation model. We consider the Gaussian Process (GP), Multi Layer Perceptron, and Random Forest as surrogate models. Among other techniques, GPs are found to have good accuracy. Moreover, GPs are data-efficient and have desirable properties, such as built-in uncertainty quantification. The study shows that the GP performs better compared to other techniques in terms of the Mean Squared Error of the prediction, while still being very fast to evaluate. We illustrate the potential of these surrogate models to also predict transient behavior.

Trends in the automotive industry confirm that the demand for testing of embedded systems, especially advanced driver assistance systems (ADAS), will grow dramatically in the near future. This paper proposes a new solution that automates the detection of software defects in embedded systems. The solution consists of a data-driven sampling algorithm to intelligently sample the testing space by sequentially generating test cases. Moreover, it segregates different defects from each other and identifies the signals that trigger each. The results are compared against other automated methods for defect identification and analysis, and it is found that this novel solution is able to identify defects more rapidly. In addition, it correctly separates defects and reliably reproduces each distinct defect.

Given their substantial success in addressing a wide range of computer vision challenges, Convolutional Neural Networks (CNNs) are increasingly being used in smart home applications, with many of these applications relying on the automatic recognition of human activities. In this context, low-power radar devices have recently gained in popularity as recording sensors, given that the usage of these devices allows mitigating a number of privacy concerns, a key issue when making use of conventional video cameras. Another concern that is often cited when designing smart home applications is the resilience of these applications against cyberattacks. It is, for instance, well-known that the combination of images and CNNs is vulnerable against adversarial examples, mischievous data points that force machine learning models to generate wrong classifications during testing time. In this paper, we investigate the vulnerability of radar-based CNNs to adversarial attacks, and where these radar-based CNNs have been designed to recognize human gestures. Through experiments with four unique threat models, we show that radar-based CNNs are susceptible to both white- and black-box adversarial attacks. We also expose the existence of an extreme adversarial attack case, where it is possible to change the prediction made by the radar-based CNNs by only perturbing the padding of the inputs, without touching the frames where the action itself occurs. Moreover, we observe that gradient-based attacks exercise perturbation not randomly, but on important features of the input data. We highlight these important features by making use of Grad-CAM, a popular neural network interpretability method, hereby showing the connection between adversarial perturbation and prediction interpretability.

Often in the design process of an engineer, the design specifications of the system are not completely known initially. However, usually there are some physical constraints which are already known, corresponding to a region of interest in the design space that is called feasible. These constraints often have no analytical form but need to be characterised based on expensive simulations or measurements. Therefore, it is important that the feasible region can be modeled sufficiently accurate using only a limited amount of samples. This can be solved by using active learning techniques that minimize the amount of samples w.r.t. what we try to model. Most active learning strategies focus on classification models or regression models with classification accuracy and regression accuracy in mind respectively. In this work, regression models of the constraints are used, but only the (in) feasibility is of interest. To tackle this problem, an information-theoretic sampling strategy is constructed to discover these regions. The proposed method is then tested on two synthetic examples and one engineering example and proves to outperform the current state-of-the-art.

Contemporary surveillance systems mainly use video cameras as their primary sensor. However, video cameras possess fundamental deficiencies, such as the inability to handle low-light environments, poor weather conditions, and concealing clothing. In contrast, radar devices are able to sense in pitch-dark environments and to see through walls. In this paper, we investigate the use of micro-Doppler (MD) signatures retrieved from a low-power radar device to identify a set of persons based on their gait characteristics. To that end, we propose a robust feature learning approach based on deep convolutional neural networks. Given that we aim at providing a solution for a real-world problem, people are allowed to walk around freely in two different rooms. In this setting, the IDentification with Radar data data set is constructed and published, consisting of 150 min of annotated MD data equally spread over five targets. Through experiments, we investigate the effectiveness of both the Doppler and time dimension, showing that our approach achieves a classification error rate of 24.70% on the validation set and 21.54% on the test set for the five targets used. When experimenting with larger time windows, we are able to further lower the error rate.

Constructing surrogate models of high-dimensional complex black-box systems from simulation-based data requires an appropriate choice of surrogate model type, as well as identification of the most influential input parameters. As including irrelevant input parameters results in a longer surrogate model training process and potentially increases the risk of overfitting, it is important to identify a small set of relevant parameters during the adaptive modeling phase of the surrogate modeling process. A multi-objective optimization step is proposed to identify both the appropriate model type as well as a parameters subset. The obtained model can be used for evaluation intensive applications such as exploration, sensitivity analysis or optimization.

High Dimensional Model Representation (HDMR) offers efficient ways to approximate computation-intensive high- dimensional black-box functions. The distinctive nature of HDMR allows a high-dimensional problem to be decomposed into a low-dimensional function or a combination of various low-dimensional functions, thus making it more attractive than other popular metamodelling approaches such as Kriging, Radial basis function, etc. However, the computational cost of HDMR is still a bottleneck for high-dimensional problems. In this work, a hybrid sequential sampling based Kriging metamodelling technique is integrated with HDMR to improve the computational efficiency of HDMR for high-dimensional problems. The performance of the proposed metamodelling approach for high-dimensional problems is validated with various benchmark mathematical problems of a wide scope of dimensionalities.

Wireless systems often need to optimize multiple conflicting objectives (low delay, high reliability, and low cost), which are difficult to fulfill simultaneously. In such cases, the wireless system exhibits multiple optimal operation points, referred to as the optimal Pareto front (OPF). However, due to the large number of parameter settings to be evaluated and the time-consuming nature of performing wireless experiments, it is typically not possible to identify the OPF by exhaustively evaluating all possible settings. Instead, for many use cases, an approximation is good enough. To this end, this paper applies a multi-objective surrogate-based optimization (MOSBO) toolbox to efficiently optimize wireless systems and approximate the OPF using a limited number of iterations. Moreover, a real Wi-Fi conferencing scenario is optimized that has two conflicting objectives (exposure and audio quality) and four configurable parameters (Tx-Power, Tx-Rate, Codec Bit-Rate, and Codec Frame-Length). The benefits of using the MOSBO approach for such a network problem is demonstrated by approximating the OPF using 94 iterations instead of requiring the exploration of 6528 different parameter combinations, while still dominating 96.58% of the complete design space.

A large amount of research focuses on experimentally optimizing the performance of wireless solutions. Finding the optimal performance settings typically requires investigating all possible combinations of design parameters, while the number of required experiments increases exponentially for each considered design parameter. The aim of this paper is to analyze the applicability of global optimization techniques to reduce the optimization time of wireless experimentation. In particular, the paper applies the Efficient Global Optimization (EGO) algorithm implemented in the SUrrogate MOdeling (SUMO) toolbox inside a wireless testbed. Moreover, to cope with the unpredictable nature of wireless testbeds, the paper applies an experiment outlier detection which monitors outside interference and verifies the validity of conducted experiments. The proposed techniques are implemented and evaluated in a wireless testbed using a realistic wireless conferencing scenario. The performance gain and experimentation time of a SUMO optimized experiment is compared against an exhaustively searched experiment. In our proof of concept, it is shown that the proposed SUMO optimizer reaches 99.79% of the global optimum performance while requiring 8.67 times less experiments compared to the exhaustive search experiment.

Surrogate models or metamodels are widely used in the realm of engineering for design optimization to minimize the number of computationally expensive simulations. Most practical problems often have conflicting objectives, which lead to a number of competing solutions which form a Pareto front. Multi-objective surrogate-based constrained optimization algorithms have been proposed in literature, but handling constraints directly is a relatively new research area. Most algorithms proposed to directly deal with multi-objective optimization have been evolutionary algorithms (Multi-Objective Evolutionary Algorithms - MOEAs). MOEAs can handle large design spaces but require a large number of simulations, which might be infeasible in practice, especially if the constraints are expensive. A multi-objective constrained optimization algorithm is presented in this paper which makes use of Kriging models, in conjunction with multi-objective probability of improvement (PoI) and probability of feasibility (PoF) criteria to drive the sample selection process economically. The efficacy of the proposed algorithm is demonstrated on an analytical benchmark function, and the algorithm is then used to solve a microwave filter design optimization problem.

The use of Surrogate Based Optimization (SBO) is widely spread in engineering design to reduce the number of computational expensive simulations. However, “real-world” problems often consist of multiple, conflicting objectives leading to a set of equivalent solutions (the Pareto front). The objectives are often aggregated into a single cost function to reduce the computational cost, though a better approach is to use multiobjective optimization methods to directly identify a set of Pareto-optimal solutions, which can be used by the designer to make more efficient design decisions (instead of making those decisions upfront). Most of the work in multiobjective optimization is focused on MultiObjective Evolutionary Algorithms (MOEAs). While MOEAs are well-suited to handle large, intractable design spaces, they typically require thousands of expensive simulations, which is prohibitively expensive for the problems under study. Therefore, the use of surrogate models in multiobjective optimization, denoted as MultiObjective Surrogate-Based Optimization (MOSBO), may prove to be even more worthwhile than SBO methods to expedite the optimization process. In this paper, the authors propose the Efficient Multiobjective Optimization (EMO) algorithm which uses Kriging models and multiobjective versions of the expected improvement and probability of improvement criterions to identify the Pareto front with a minimal number of expensive simulations. The EMO algorithm is applied on multiple standard benchmark problems and compared against the wellknown NSGA-II and SPEA2 multiobjective optimization methods with promising results.