The accuracy of Machine Learning (ML) models is highly dependent on the hyperparameters that have to be chosen by the user before the training. However, finding the optimal set of hyperparameters is a complex process, as many different parameter combinations need to be evaluated, and obtaining the accuracy of each combination usually requires a full training run. It is therefore of great interest to reduce the computational runtime of this process. On High-Performance Computing (HPC) systems, several configurations can be evaluated in parallel to speed up this Hyperparameter Optimization (HPO). State-of-the-art HPO methods follow a bandit-based approach and build on top of successive halving, where the final performance of a combination is estimated based on a lower than fully trained fidelity performance metric and more promising combinations are assigned more resources over time. Frequently, the number of epochs is treated as a resource, letting more promising combinations train longer. Another option is to use the number of workers as a resource and directly allocate more workers to more promising configurations via data-parallel training. This article proposes a novel Resource-Adaptive Successive Doubling Algorithm (RASDA), which combines a resource-adaptive successive doubling scheme with the plain Asynchronous Successive Halving Algorithm (ASHA). Scalability of this approach is shown on up to 1,024 Graphics Processing Units (GPUs) on modern HPC systems. It is applied to different types of Neural Networks (NNs) and trained on large datasets from the Computer Vision (CV), Computational Fluid Dynamics (CFD), and Additive Manufacturing (AM) domains, where performing more than one full training run is usually infeasible. Empirical results show that RASDA outperforms ASHA by a factor of up to 1.9 with respect to the runtime. At the same time, the solution quality of final ASHA models is maintained or even surpassed by the implicit batch size scheduling of RASDA. With RASDA, systematic HPO is applied to a terabyte-scale scientific dataset for the first time in the literature, enabling efficient optimization of complex models on massive scientific data.

The robustness of an artificial neural network that performs model order reduction for flow field data is studied. The network is trained with a large-scale distributed learning approach using up to 6,259 nodes of the supercomputer Fugaku. Flow around two square cylinders with a varying distance between their centers is investigated. The network is trained and tested with data from numerical simulations. First, the capability to reproduce flow fields with 2, 12, and 24 modes is investigated by comparing the reconstructed flow data to simulated data. It is shown, that reconstructions based on 2 modes cannot capture both, low- and high-frequency flow structures correctly, whereas predictions based on 12 and 24 modes yield improved flow fields, especially in the case of high-frequency waves in the vicinity of the square cylinders. Reconstructions with 24 modes provide smooth velocity fields that reproduce all relevant low- and high-frequency waves for all variations of the distance between the two square cylinders. Second, the performance of the machine learning-based reconstructions are compared to proper orthogonal decomposition, which is a commonly used reduced order model technique. The comparison only includes flow fields based on 24 modes. For all geometric variations, the mean squared errors of the reconstructions by the conventional method are higher than those of the machine learning model. This underlines the advantage of artificial neural networks over linear methods like proper orthogonal decomposition for tasks like reconstructing flow fields that are characterized by non-linear governing equations.

Accurately computing the flow in the nasal cavity with computational fluid dynamics (CFD) simulations requires highly resolved computational meshes based on anatomically realistic geometries. Such geometries can only be obtained from computer tomography (CT) data with high spatial resolution, i.e., featuring a ≤1mm slice thickness. In practice, CTs are, however, recorded at a lower resolution to not expose patients to high radiation and to reduce the overall costs. To overcome this problem and to provide patients with a detailed physics-based diagnosis, e.g., for surgery planning, the potential of super-resolution networks (SRNs) to increase the CT resolution is analyzed. Therefore, an SRN is developed and trained on CT data. Its predictive performance is improved by an automated hyperparameter optimization technique. The training time is further reduced without predictive accuracy degradation by oversampling images with challenging regions. The performance of the SRN is assessed by an analysis of the reconstructed 3D surfaces of the human upper airway and by comparing results of CFD simulations. That is, surfaces and simulation results based on SRN-generated CT data at 1mm resolution are compared to those obtained from unmodified CT data-sets at low (3mm) and high (1mm) resolution, as well as from CT data interpolated to a 1mm resolution from coarse data. The findings reveal the SRN-based approach to have the lowest deviations in the physics-based CFD results when compared to those based on the original high-resolution data. The pressure loss between the inflow (nostrils) and outflow (pharynx) regions averaged for three test patients differs by only 1.3%, compared to 8.7% and 8.8% in the coarse and interpolated cases. It is concluded that the SRN-based method is a promising tool to enhance underresolved CT data to yield reliable numerical results of respiratory flows.

The application of Artificial Neural Networks (ANNs) has been extensively investigated for fluid dynamic problems. A specific form of ANNs are Physics-Informed Neural Networks (PINNs). They incorporate physical laws in the training and have increasingly been explored in the last few years. In this work, the prediction accuracy of PINNs is compared with that of conventional Deep Neural Networks (DNNs). The accuracy of a DNN depends on the amount of data provided for training. The change in prediction accuracy of PINNs and DNNs is assessed using a varying amount of training data. To ensure the correctness of the training data, they are obtained from analytical and numerical solutions of classical problems in fluid mechanics. The objective of this work is to quantify the fraction of training data relative to the maximum number of data points available in the computational domain, such that the accuracy gained with PINNs justifies the increased computational cost. Furthermore, the effects of the location of sampling points in the computational domain and noise in training data are analyzed. In the considered problems, it is found that PINNs outperform DNNs when the sampling points are positioned in the Regions of Interest. PINNs for predicting potential flow around a Rankine oval have shown a better robustness against noise in training data compared to DNNs. Both models show higher prediction accuracy when sampling points are randomly positioned in the flow domain as compared to a prescribed distribution of sampling points. The findings reveal new insights on the strategies to massively improve the prediction capabilities of PINNs with respect to DNNs.

Deep Learning models have proven necessary in dealing with the challenges posed by the continuous growth of data volume acquired from satellites and the increasing complexity of new Remote Sensing applications. To obtain the best performance from such models, it is necessary to fine-tune their hyperparameters. Since the models might have massive amounts of parameters that need to be tuned, this process requires many computational resources. In this work, a method to accelerate hyperparameter optimization on a High-Performance Computing system is proposed. The data batch size is increased during the training, leading to a more efficient execution on Graphics Processing Units. The experimental results confirm that this method reduces the runtime of the hyperparameter optimization step by a factor of 3 while achieving the same validation accuracy as a standard training procedure with a fixed batch size.

Abstract Many simulation workflows require to prepare the data for the simulation manually. This is time consuming and leads to a massive bottleneck when a large number of numerical simulations is requested. This bottleneck can be overcome by an automated data processing pipeline. Such a novel pipeline is developed for a medical use case from rhinology, where computer tomography recordings are used as input and flow simulation data define the results. Convolutional neural networks are applied to segment the upper airways and to detect and prepare the in- and outflow regions for accurate boundary condition prescription in the simulation. The automated process is tested on three cases which have not been used to train the networks. The accuracy of the pipeline is evaluated by comparing the network-generated output surfaces to those obtained from a semi-automated procedure performed by a medical professional. Except for minor deviations at interfaces between ethmoidal sinuses, the network-generated surface is sufficiently accurate. To further analyze the accuracy of the automated pipeline, flow simulations are conducted with a thermal lattice-Boltzmann method for both cases on a high-performace computing system. The comparison of the results of the respiratory flow simulations yield averaged errors of less than 1% for the pressure loss between the in- and outlets, and for the outlet temperature. Thus, the pipeline is shown to work accurately and the geometrical deviations at the ethmoidal sinuses to be negligible.