Data-driven surrogate models provide fast and fully differentiable approximations of complex dynamical systems. In this work, we develop such surrogates for the Rayleigh–Bénard convection (RBC), which governs thermally driven flows in natural and industrial environments. Specifically, the proposed models approximate the discrete-time flow map of the RBC system, advancing the full system state by a fixed time step. We train Fourier Neural Operator (FNO)–based models to learn the dynamics of RBC in two and three dimensions and compare them to a convolutional U-Net baseline and a Koopman-based Linear Recurrent Autoencoder Network (LRAN). The two-dimensional system serves as a baseline for the more challenging three-dimensional case, which exhibits increased spatial complexity and turbulent dynamics. Across all settings, FNO-based models consistently outperform the LRAN, while achieving performance comparable to the U-Net in several regimes. Incorporating spatio-temporal inputs via FNOs leads to improved long-term prediction accuracy, particularly for turbulent flows. The physical fidelity of the predictions is assessed using convective heat flux statistics, profiles, and fluctuations, showing that FNOs most closely reproduce the ground-truth flow statistics. In addition, we demonstrate that FNOs enable zero-shot super-resolution across unseen spatial discretizations, a capability not shared by the convolutional baselines. These results highlight the potential of neural operator–based models as accurate, physically consistent, and resolution-independent surrogates for downstream tasks such as flow control.

Backdoor attacks pose a serious threat to the security of face recognition systems. These involve the insertion of poisoned inputs into the training data to manipulate the model's behavior at inference time and can cause severe consequences, such as unauthorized access to secure systems or impersonation of legitimate users. Previous works on backdoor attacks have primarily focused on closed-set classification systems. However, open-set face recognition systems are commonly utilized in practical applications, which operate fundamentally differently from closed-set systems. In this paper, we propose two main contributions. First, we demonstrate that closed-set backdoor attacks are effective in basic classification scenarios but fail to perform well in complex open-set face recognition tasks. Second, we introduce Feature Stabilized Trigger Loss (FSTL), a novel loss function designed to facilitate the learning of backdoors in open-set recognition models. The experiments were conducted on two large-scale datasets using a variety of high-performing face recognition systems and by training with both physical and digital triggers. Since developing effective attack countermeasures requires knowledge of effective attacks, this work will enable future works on developing more secure recognition systems.

Different conflicting optimization criteria arise naturally in various Deep Learning scenarios. These can address different main tasks (i.e., in the setting of Multi-Task Learning), but also main and secondary tasks such as loss minimization versus sparsity. The usual approach is a simple weighting of the criteria, which formally only works in the convex setting. In this paper, we present a Multi-Objective Optimization algorithm using a modified Weighted Chebyshev scalarization for training Deep Neural Networks (DNNs) with respect to several tasks. By employing this scalarization technique, the algorithm can identify all optimal solutions of the original problem while reducing its complexity to a sequence of single-objective problems. The simplified problems are then solved using an Augmented Lagrangian method, enabling the use of popular optimization techniques such as Adam and Stochastic Gradient Descent, while efficaciously handling constraints. Our work aims to address the (economical and also ecological) sustainability issue of DNN models, with a particular focus on Deep Multi-Task models, which are typically designed with a very large number of weights to perform equally well on multiple tasks. Through experiments conducted on two Machine Learning datasets, we demonstrate the possibility of adaptively sparsifying the model during training without significantly impacting its performance, if we are willing to apply task-specific adaptations to the network weights. Code is available at https://github.com/salomonhotegni/MDMTN.

Abstract Regularization is used in many different areas of optimization when solutions are sought which not only minimize a given function, but also possess a certain degree of regularity. Popular applications are image denoising, sparse regression and machine learning. Since the choice of the regularization parameter is crucial but often difficult, path-following methods are used to approximate the entire regularization path, i.e., the set of all possible solutions for all regularization parameters. Due to their nature, the development of these methods requires structural results about the regularization path. The goal of this article is to derive these results for the case of a smooth objective function which is penalized by a piecewise differentiable regularization term. We do this by treating regularization as a multiobjective optimization problem. Our results suggest that even in this general case, the regularization path is piecewise smooth. Moreover, our theory allows for a classification of the nonsmooth features that occur in between smooth parts. This is demonstrated in two applications, namely support-vector machines and exact penalty methods.

We present a novel algorithm that allows us to gain detailed insight into the effects of sparsity in linear and nonlinear optimization. Sparsity is of great importance in many scientific areas such as image and signal processing, medical imaging, compressed sensing, and machine learning, as it ensures robustness against noisy data and yields models that are easier to interpret due to the small number of relevant terms. It is common practice to enforce sparsity by adding the$\ell _1$-norm as a penalty term. In order to gain a better understanding and to allow for an informed model selection, we directly solve the corresponding multiobjective optimization problem (MOP) that arises when minimizing the main objective and the$\ell _1$-norm simultaneously. As this MOP is in general non-convex for nonlinear objectives, the penalty method will fail to provide all optimal compromises. To avoid this issue, we present a continuation method specifically tailored to MOPs with two objective functions one of which is the$\ell _1$-norm. Our method can be seen as a generalization of homotopy methods for linear regression problems to the nonlinear case. Several numerical examples – including neural network training – demonstrate our theoretical findings and the additional insight gained by this multiobjective approach.

Abstract It is a challenging task to identify the objectives on which a certain decision was based, in particular if several, potentially conflicting criteria are equally important and a continuous set of optimal compromise decisions exists. This task can be understood as the inverse problem of multiobjective optimization, where the goal is to find the objective function vector of a given Pareto set. To this end, we present a method to construct the objective function vector of an unconstrained multiobjective optimization problem (MOP) such that the Pareto critical set contains a given set of data points with prescribed KKT multipliers. If such an MOP can not be found, then the method instead produces an MOP whose Pareto critical set is at least close to the data points. The key idea is to consider the objective function vector in the multiobjective KKT conditions as variable and then search for the objectives that minimize the Euclidean norm of the resulting system of equations. By expressing the objectives in a finite-dimensional basis, we transform this problem into a homogeneous, linear system of equations that can be solved efficiently. Potential applications of this approach include the identification of objectives (both from clean and noisy data) and the construction of surrogate models for expensive MOPs.