Stencil codes are valuable methods to solve partial differential equations of models in a wide range of applications in science and engineering. Graphics processing units (GPUs) provide a highly parallel architecture with fast directly accessible memory that is desirable to run stencil codes. They enable larger and more complex simulations that are solved faster compared to simulating on CPUs. We provide a solution for users to run highly parallel stencil codes on GPUs, that can also be efficiently used on a distributed GPU cluster.In this work, we present an extension to the multi-disciplinary framework NAStJA originally designed to efficiently run stencil codes on the CPUs in current high-performance computing (HPC) systems, to also be able to run on GPUs. We describe different methods to increase the performance of stencil codes on GPUs like a border exchange method which is transparent to the user, as well as a buffer for gradient values that are needed multiple times. We show their performance using the phase-field method as a stencil code example.With this GPU extension and optimizations implemented into the NAStJA framework, we can show highly improved performance compared to the CPU implementation, and an efficiency of 92% (weak scaling) for a large-scale example simulation on 64 GPUs on the ForHLR II HPC system.

Abstract Background Discoveries in cellular dynamics and tissue development constantly reshape our understanding of fundamental biological processes such as embryogenesis, wound-healing, and tumorigenesis. High-quality microscopy data and ever-improving understanding of single-cell effects rapidly accelerate new discoveries. Still, many computational models either describe few cells highly detailed or larger cell ensembles and tissues more coarsely. Here, we connect these two scales in a joint theoretical model. Results We developed a highly parallel version of the cellular Potts model that can be flexibly applied and provides an agent-based model driving cellular events. The model can be modular extended to a multi-model simulation on both scales. Based on the NAStJA framework, a scaling implementation running efficiently on high-performance computing systems was realized. We demonstrate independence of bias in our approach as well as excellent scaling behavior. Conclusions Our model scales approximately linear beyond 10,000 cores and thus enables the simulation of large-scale three-dimensional tissues only confined by available computational resources. The strict modular design allows arbitrary models to be configured flexibly and enables applications in a wide range of research questions. Cells in Silico (CiS) can be easily molded to different model assumptions and help push computational scientists to expand their simulations to a new area in tissue simulations. As an example we highlight a 10003 voxel-sized cancerous tissue simulation at sub-cellular resolution.

In the last decades, simulations have been established in several fields of science and industry to study various phenomena by solving, inter alia, partial differential equations. For an efficient use of current and future high performance computing systems, with many thousands of computation ranks, high node-level performance, scalable communication, and the omission of unnecessary calculations are of high priority in the development of new solvers. The challenge of contemporary simulation applications is to bridge the gap between the scales of the various physical processes. We introduce the NAStJA framework, a block-based MPI parallel solver for arbitrary algorithms, based on stencil code or other regular grid methods. NAStJA decomposes the domain of spatially complex structures into small cuboid blocks. A special feature of NAStJA is the dynamic block adaption which modifies the calculation domain around the region where the computation currently takes place, and hence avoids unnecessary calculations. This often occurs, inter alia, in phase-field simulations. Block creation and deletion is managed autonomously within local neighborhoods. A basic load balancing mechanism allows a re-distribution of newly created blocks to the involved computing ranks. The use of a multi-hop network, to distribute information to the entire domain, avoids collective all-gather communications. Thus, we can demonstrate excellent scaling. The present scaling tests substantiate the enormous advantage of this adaptive method. For certain simulation scenarios, we can show that the calculation effort and memory consumption can be reduced to only 3.5 percent, compared to the classical full-domain reference simulation. The overhead of 70-100 percent for the dynamic adapting block creation is significantly lower than the gain. The approach is not restricted to phase-field simulations, and can be employed in other domains of computational science to exploit sparsity of computing regions.

Microstructures forming during ternary eutectic directional solidification processes have significant influence on the macroscopic mechanical properties of metal alloys. For a realistic simulation, we use the well established thermodynamically consistent phase-field method and improve it with a new grand potential formulation to couple the concentration evolution. This extension is very compute intensive due to a temperature dependent diffusive concentration. We significantly extend previous simulations that have used simpler phase-field models or were performed on smaller domain sizes. The new method has been implemented within the massively parallel HPC framework waLBerla that is designed to exploit current supercomputers efficiently. We apply various optimization techniques, including buffering techniques, explicit SIMD kernel vectorization, and communication hiding. Simulations utilizing up to 262,144 cores have been run on three different supercomputing architectures and weak scalability results are shown. Additionally, a hierarchical, mesh-based data reduction strategy is developed to keep the I/O problem manageable at scale.