Multi-GPU nodes are increasingly common in the rapidly evolving landscape of exascale supercomputers. On these systems, GPUs on the same node are connected through dedicated networks, with bandwidths up to a few terabits per second. However, gauging performance expectations and maximizing system efficiency is challenging due to different technologies, design options, and software layers. This paper comprehensively characterizes three supercomputers — Alps, Leonardo, and LUMI — each with a unique architecture and design. We focus on performance evaluation of intra-node and inter-node interconnects on up to 4,096 GPUs, using a mix of intra-node and inter-node benchmarks. By analyzing its limitations and opportunities, we aim to offer practical guidance to researchers, system architects, and software developers dealing with multi-GPU supercomputing. Our results show that there is untapped bandwidth, and there are still many opportunities for optimization, ranging from network to software optimization.

In April 2018, under the auspices of the POR-FESR 2014-2020 program of Italian Piedmont Region, the Turin's Centre on High-Performance Computing for Artificial Intelligence (HPC4AI) was funded with a capital investment of 4.5M€ and it began its deployment. HPC4AI aims to facilitate scientific research and engineering in the areas of Artificial Intelligence and Big Data Analytics. HPC4AI will specifically focus on methods for the on-demand provisioning of AI and BDA Cloud services to the regional and national industrial community, which includes the large regional ecosystem of Small-Medium Enterprises (SMEs) active in many different sectors such as automotive, aerospace, mechatronics, manufacturing, health and agrifood.

SVE (Scalable Vector Extension) is Arm's new vector instruction extension targeting high performance workloads. SVE offers many opportunities to optimise compute intensive workloads but, with the availability of SVE-enabled hardware still years away, we have to rely on simulation techniques in order to evaluate our implementations. Working with simulators can be tricky and it comes with many limitations but, used properly, a simulator like Gem5 is a valuable tool that can provide an opportunity to explore the possibilities opened by this new extension. As a use case, we focus our attention on the field of genomics, where the recent advent of high-throughput sequencing machines producing big amounts of genomic data has boosted the interest in efficient approximate string matching and alignment techniques. Genomics algorithms are the key computational building blocks for the downstream data analysis on resequencing projects where hundreds of GBytes of sequenced data are analysed against a reference genome in order to filter sequencing errors and detect potential genomic variation events. The computational requirements and sheer size of the input data used by these applications make them a challenging problem. Fortunately, they also exhibit a high degree of data parallelism, making them good candidates for vectorisation techniques. In this work we explore the unique opportunities that SVE provides in order to exploit the parallelism present in genomics algorithms. We discuss preliminary results, our simulation strategy, some of the obstacles and limitations we faced, and how to work around them in order to obtain meaningful results.

GPU computing has revolutionized HPC by bringing the performance of the supercomputer to the desktop. Attractive price, performance, and power characteristics allow multiple GPUs to be plugged into both desktop machines as well as supercomputer nodes for increased performance. Excellent performance and scalability can be achieved for some problems using hybrid combinations of multiple GPUs and CPU computing resources. This paper presents the acceleration of the open-source Quantum ESPRESSO package with the freely available phiGEMM library. Specifically, the parallel implementation and scaling of the phiGEMM matrix-matrix multiplication will be discussed. This library can be called from applications through all standard GEMM interfaces and it is able to perform matrix-matrix multiplications using one or more GPUs as well as the host multi-core processor. An 8.9-times speedup is reported in overall run-time of a representative AUSURF112 benchmark for a PWscf calculation. In addition, multi-GPU scaling and performance for 3D-FFTs are discussed.