In weather and climate research, latitude–longitude grid data are typically used for analysis and visualization, and remapping from model native grids to latitude–longitude grids typically requires a significant amount of time. Here, we developed a series of parallelized remapping algorithms for NICAM, a global weather and climate model with an icosahedral grid system, and demonstrated their performance with global 14–0.87-km mesh model data on the supercomputer Fugaku. The original remapping tool in NICAM supports parallelization only in reading and interpolating data. In our proposed algorithms, the process of data writing is parallelized by separating output files or using the MPI-IO library, both of which enable us to remap 0.87-km mesh data with 670 million horizontal grid points and 94 vertical levels. The benchmark with 14-km mesh data shows that the developed algorithms significantly outperform the original algorithm in terms of elapsed time (by 7.4–8.7 times) and memory usage (by 2.8–5.0 times). Among the proposed algorithms, the separation of output files, along with reduced MPI communication size, leads to a better performance in the elapsed time and its scalability, and the use of the MPI-IO library leads to a better performance in memory usage. The remapping year per wall-clock day, assuming a six-hourly output interval, is up to 0.56 with 3.5-km mesh data, demonstrating the feasibility of handling global cloud-resolving climate simulation data in a practical time. This study demonstrates the importance of IO performance, including MPI-IO, in accelerating weather and climate research on future supercomputers.

Real-time 30-second-refresh numerical weather prediction (NWP) was performed with exclusive use of 11,580 nodes (~7%) of supercomputer Fugaku during Tokyo Olympics and Paralympics in 2021. Total 75,248 forecasts were disseminated in the 1-month period mostly stably with time-to-solution less than 3 minutes for 30-minute forecast. Japan's Big Data Assimilation (BDA) project developed the novel NWP system for precise prediction of hazardous rains toward solving the global climate crisis. Compared with typical 1-hour-refresh systems, the BDA system offered two orders of magnitude increase in problem size and revealed the effectiveness of 30-second refresh for highly nonlinear, rapidly evolving convective rains. To achieve the required time-to-solution for real-time 30-second refresh with high accuracy, the core BDA software incorporated single precision and enhanced parallel I/O with properly selected configurations of 1000 ensemble members and 500-m-mesh weather model. The massively parallel, I/O intensive real-time BDA computation demonstrated a promising future direction.

We have been carrying out the FLAGSHIP 2020 Project to develop the Japanese next-generation flagship supercomputer, the Post-K, recently named “Fugaku”. We have designed an original many core processor based on Armv8 instruction sets with the Scalable Vector Extension (SVE), an A64FX processor, as well as a system including interconnect and a storage subsystem with the industry partner, Fujitsu. The “co-design” of the system and applications is a key to making it power efficient and high performance. We determined many architectural parameters by reflecting an analysis of a set of target applications provided by applications teams. In this paper, we present the pragmatic practice of our co-design effort for “Fugaku”. As a result, the system has been proven to be a very power-efficient system, and it is confirmed that the performance of some target applications using the whole system is more than 100 times the performance of the K computer.

Numerical weather prediction (NWP) supports our daily lives. Weather models require higher spatiotemporal resolutions to prepare for extreme weather disasters and reduce the uncertainty of predictions. The accuracy of the initial state of the weather simulation is also critical; thus, we need more advanced data assimilation (DA) technology. By combining resolution and ensemble size, we have achieved the world’s largest weather DA experiment using a global cloud-resolving model and an ensemble Kalman filter method. The number of grid points was $\sim$4.4 trillion, and 1.3 PiB of data was passed from the model simulation part to the DA part. We adopted a data-centric application design and approximate computing to speed up the overall system of DA. Our DA system, named NICAM-LETKF, scales to 131,072 nodes (6,291,456 cores) of the supercomputer Fugaku with a sustained performance of 29 PFLOPS and 79 PFLOPS for the simulation and DA parts, respectively.

Summary On the verge of the convergence between high‐performance computing and Big Data processing, it has become increasingly prevalent to deploy large‐scale data analytics workloads on high‐end supercomputers. Such applications often come in the form of complex workflows with various different components, assimilating data from scientific simulations as well as from measurements streamed from sensor networks, such as radars and satellites. For example, as part of the Flagship 2020 (post‐K) supercomputer project of Japan, RIKEN is investigating the feasibility of a highly accurate weather forecasting system that would provide a real‐time outlook for severe guerrilla rainstorms. One of the main performance bottlenecks of this application is the lack of efficient communication among workflow components, which currently takes place over the parallel file system.In this paper, we present an initial study of a direct communication framework designed for complex workflows that eliminates unnecessary file I/O among components. Specifically, we propose an I/O arbitration layer that provides direct parallel data transfer (both synchronous and asynchronous) among job components that rely on the netCDF interface for performing I/O operations. Our solution requires only minimal modifications to application code. Moreover, we propose a configuration file–based approach that allows users to specify the desired data transfer pattern among workflow components, offering a general solution for different application contexts. We present a preliminary evaluation of the proposed framework on the K Computer (running on up to 4800 compute nodes) using RIKEN's experimental weather forecasting workflow as a case study.

We propose a cost-effective online nesting procedure (CONeP) for regional atmospheric models to improve computational efficiency. The conventional procedure of online nesting is ineffective because computations are executed sequentially for each domain, and it does not enable users freely to determine the number of computational nodes. However, CONeP can completely avoid this limitation through three actions: 1) splitting the processes into multiple subgroups; 2) making each subgroup manage just one domain; and 3) executing the computations for each domain simultaneously. Since users can assign an optimal number of nodes to each domain, the model with CONeP is computationally efficient. We demonstrate the computational advantage of CONeP over the conventional procedure, comparing the elapsed times with both procedures on a supercomputer. The elapsed time with CONeP is markedly shorter than that observed with the conventional procedure using the same number of computational nodes. This advantage becomes more significant as the number of nesting domains increases.

Stencil computation has many applications in science and engineering, thus many optimization techniques such as temporal blocking have been developed. They are, however, rarely used in real-world applications, since a large amount of careful programming is required for even the simplest of stencils. We introduce Formura, a domain specific language that provides easy access to optimized stencil computations. Higher-order integration schemes can be defined using mathematical notations. Formura generates C code with MPI calls and performs autotuning. Hence its performance is portable to most distributed-memory computers. We show the scientific applicability of Formura by performing magnetohydrodynamics (MHD) and belowground biology simulations. Ability to reach bytes-per-flops ratio only attainable by temporal blocking is demonstrated. We also demonstrate scaling up to the full nodes of the K computer, with 1.184 Pflops, 11.62% floating-pointoperation efficiency, and 31.26% memory throughput efficiency.

Following the invention of the telegraph, electronic computer, and remote sensing, “big data” is bringing another revolution to weather prediction. As sensor and computer technologies advance, orders of magnitude bigger data are produced by new sensors and high-precision computer simulation or “big simulation.” Data assimilation (DA) is a key to numerical weather prediction (NWP) by integrating the real-world sensor data into simulation. However, the current DA and NWP systems are not designed to handle the “big data” from next-generation sensors and big simulation. Therefore, we propose “big data assimilation” (BDA) innovation to fully utilize the big data. Since October 2013, the Japan's BDA project has been exploring revolutionary NWP at 100-m mesh refreshed every 30 s, orders of magnitude finer and faster than the current typical NWP systems, by taking advantage of the fortunate combination of next-generation technologies: the 10-petaflops K computer, phased array weather radar, and geostationary satellite Himawari-8. So far, a BDA prototype system was developed and tested with real-world retrospective local rainstorm cases. This paper summarizes the activities and progress of the BDA project, and concludes with perspectives toward the post-petascale supercomputing era.

In this paper, we develop a rank-mapping algorithm for an icosahedral grid system on a massive parallel computer with the 3-D torus network topology, specifically on the K computer. Our aim is to improve the weak scaling performance of the point-to-point communications for exchanging grid-point values between adjacent grid regions on a sphere. We formulate a new rank-mapping algorithm to reduce the maximum number of hops for the point-to-point communications. We evaluate both the new algorithm and the standard ones on the K computer, using the communication kernel of the Nonhydrostatic Icosahedral Atmospheric Model (NICAM), a global atmospheric model with an icosahedral grid system. We confirm that, unlike the standard algorithms, the new one achieves almost perfect performance in the weak scaling on the K computer, even for 10,240 nodes. Results of additional experiments imply that the high scalability of the new rank-mapping algorithm on the K computer is achieved by reducing network congestion in the links between adjacent nodes.