High Performance Computing (HPC) systems generate a large amount of unstructured/alphanumeric log messages that capture the health state of their components. Due to their design complexity, HPC systems often undergo failures that halt applications (e.g., weather prediction, aerodynamics simulation) execution. However, existing failure prediction methods, which typically seek to extract some information theoretic features, fail to scale both in terms of accuracy and prediction speed, limiting their adoption in real-time production systems. In this paper, differently from existing work and inspired by current transformer-based neural networks which have revolutionized the sequential learning in the natural language processing (NLP) tasks, we propose a novel scalable log-based, self-supervised model (i.e., no need for manual labels), called Time Machine11A Time Machine allows us to travel into the future to observe the health state of HPC system and report back. Here, we travel into the log extension to report an upcoming failure., that predicts (i) forthcoming log events (ii) the upcoming failure and its location and (iii) the expected lead time to failure. Time Machine is designed by combining two stacks of transformer-decoders, each employing the self-attention mechanism. The first stack addresses the failure location by predicting the sequence of log events and then identifying if a failure event is part of that sequence. The lead time to predicted failure is addressed by the second stack. We evaluate Time Machine on four real-world HPC log datasets and compare it against three state-of-the-art failure prediction approaches. Results show that Time Machine significantly outperforms the related works on Bleu, Rouge, MCC, and F1-score in predicting forthcoming events, failure location, failure lead-time, with higher prediction speed.

In this paper we investigate the effects of wide vector instructions on modern processor caches. On the one hand, contemporary processors have large, highly associative caches, which greatly benefit applications that can exploit spatial or temporal data locality. On the other hand, vector instructions operate on wide lists of operands, and moving this data through the cache hierarchy can fill it up quickly. We use a selection of mini-apps representative of a range of scientific application classes to investigate the behaviour of caches in two state-of-the-art Arm-based processors, the Marvell ThunderX2 and the Fujitsu A64FX. We compile the application to target the Arm Scalable Vector Extension (SVE) and we model the caches of these two processors using a newly developed cache simulator. We then vary a number of cache parameters and show how these choice influence application behaviour at a range of SVE widths between 128 and 2048 bits. We observed a correlation between higher cache associativity and lower miss rate. For the first cache level, at higher line sizes an increase in associativity was necessary to decrease miss rate compared to a cache with the same total size but smaller line size; for the second level, higher associativity did not always result in better performance with long cache lines. As the SVE width was scaled, data was evicted from cache quicker, an effect which was more noticeable at smaller line sizes. Larger cache lines also allowed non-contiguous requests to be fulfilled with fewer loads, because each cache lines covers more memory space. We use a selection of mini-apps representative of a range of scientific application classes to investigate the behaviour of caches in two state-of-the-art Arm-based processors, the Marvell ThunderX2 and the Fujitsu A64FX. We compile the application to target the Arm Scalable Vector Extension (SVE) and we model the caches of these two processors using a newly developed cache simulator. We then vary a number of cache parameters and show how these choice influence application behaviour at a range of SVE widths between 128 and 2048 bits. We observed a correlation between higher cache associativity and lower miss rate. For the first cache level, at higher line sizes an increase in associativity was necessary to decrease miss rate compared to a cache with the same total size but smaller line size; for the second level, higher associativity did not always result in better performance with long cache lines. As the SVE width was scaled, data was evicted from cache quicker, an effect which was more noticeable at smaller line sizes. Larger cache lines also allowed non-contiguous requests to be fulfilled with fewer loads, because each cache lines covers more memory space. We observed a correlation between higher cache associativity and lower miss rate. For the first cache level, at higher line sizes an increase in associativity was necessary to decrease miss rate compared to a cache with the same total size but smaller line size; for the second level, higher associativity did not always result in better performance with long cache lines. As the SVE width was scaled, data was evicted from cache quicker, an effect which was more noticeable at smaller line sizes. Larger cache lines also allowed non-contiguous requests to be fulfilled with fewer loads, because each cache lines covers more memory space.

Abstract In this paper, we present scaling results from two production quality supercomputers that use the first generation of Arm‐based CPUs that have been optimized for scientific workloads. Both systems use Marvell ThunderX2 CPUs, which deliver high core counts and class‐leading memory bandwidth. The first system is Isambard, a Cray XC50 “Scout” system operated by the GW4 Alliance and the UK Met Office as a Tier‐2 national HPC service. The second system is one of three Arm‐based HPE Apollo 70 systems delivered as part of the Catalyst UK project, running at the University of Bristol. We compare scaling results from these two systems with three Cray XC50 systems based on Intel Skylake and Broadwell CPUs. We focus on a range of applications and mini‐apps that are important to the UK national HPC service, ARCHER, and to our project partners. We also compare the performance and maturity of the state‐of‐the‐art toolchains available on Arm‐based HPC systems.

Summary In this paper, we present performance results from Isambard, the first production supercomputer to be based on Arm CPUs that have been optimized specifically for HPC. Isambard is the first Cray XC50 “Scout” system, combining Cavium ThunderX2 Arm‐based CPUs with Cray's Aries interconnect. The full Isambard system will be delivered in the summer of 2018, when it will contain over 10 000 Arm cores. In this work, we present node‐level performance results from eight early‐access nodes that were upgraded to B0 beta silicon in March 2018. We present node‐level benchmark results comparing ThunderX2 with mainstream CPUs, including Intel Skylake and Broadwell, as well as Xeon Phi. We focus on a range of applications and mini‐apps important to the UK national HPC service, ARCHER, as well as to the Isambard project partners and the wider HPC community. We also compare performance across three major software toolchains available for Arm: Cray's CCE, Arm's version of Clang/Flang/LLVM, and GNU.

Solving the deterministic discrete ordinates neutral particle transport equation is a computationally expensive application. On an unstructured mesh, the discontinuous Galerkin finite element method is used for discretisation of the spatial domain. Additionally, an upwind dependency is applied forming wavefront sweeps across the spatial mesh for each iteration of the solve. We present a new mini-app, UnSNAP, which can be used to investigate the performance of arbitrarily high-order finite element unstructured transport on modern architectures. A new schedule appropriate for such architectures is presented. Finally, we show performance results for the mini-app on CPUs with high numbers of cores.

Fully binarised convolutional neural networks (CNNs) deliver very high inference performance using single-bit weights and activations, together with XNOR type operators for the kernel convolutions. Current research shows that full binarisation results in a degradation of accuracy and different approaches to tackle this issue are being investigated such as using more complex models as accuracy reduces. This paper proposes an alternative based on a multi-precision CNN frame-work that combines a binarised and a floating point CNN in a pipeline configuration deployed on heterogeneous hardware. The binarised CNN is mapped onto an FPGA device and used to perform inference over the whole input set while the floating point network is mapped onto a CPU device and performs re-inference only when the classification confidence level is low. A light-weight confidence mechanism enables a flexible trade-off between accuracy and throughput. To demonstrate the concept, we choose a Zynq 7020 device as the hardware target and show that the multi-precision network is able to increase the BNN accuracy from 78.5% to 82.5% and the CPU inference speed from 29.68 to 90.82 images/sec.

In this research we describe the development and optimisation of a new Monte Carlo neutral particle transport mini-app, neutral. In spite of the success of previous research efforts to load balance the algorithm at scale, it is not clear how to take advantage of the diverse architectures being installed in the newest supercomputers. We explore different algorithmic approaches, and perform extensive investigations into the performance of the application on modern hardware including Intel Xeon and Xeon Phi CPUs, POWER8 CPUs, and NVIDIA GPUs.When applied to particle transport the Monte Carlo method is not embarrassingly parallel, as might be expected, due to dependencies on the computational mesh that expose random memory access patterns. The algorithm requires the use of atomic operations, and exhibits load imbalance at the node-level due to the random branching of particle histories. The algorithmic characteristics make it challenging to exploit the high memory bandwidth and FLOPS of modern HPC architectures.Both of the parallelisation schemes discussed in this paper are dominated latency issues caused by poor data locality, and are restricted by the use of atomic operations for tallying calculations. We saw a significant improvement in performance through the use of hyperthreading on all CPUs and best performance on the NVIDIA P100 GPU. A key observation is that architectures that are tolerant to latencies may be able to hide the negative characteristics of the algorithms.

The arch project is a suite of mini-apps that have been developed with consistent coding practices, under a common infrastructural layer. Great emphasis has been placed on making the applications concise and easy to manipulate, while capturing the key performance characteristics of their proxied algorithmic classes. The suite is intended for traditional exploration of performance, portability and productivity on modern HPC architectures, but also introduces the potential for focussing on those characteristics of production application stacks that are not generally exposed with isolated mini-app developments. In this paper we discuss the implementation of each of the mini-apps, and present key findings from the development and optimisation process, alongside details of important future research directions.

Iterative sparse linear solvers are an important class of algorithm in high performance computing, and form a crucial component of many scientific codes. As intra and inter node parallelism continues to increase rapidly, the design of new, scalable solvers which can target next generation architectures becomes increasingly important. In this work we present TeaLeaf, a recent mini-app constructed to explore design space choices for highly scalable solvers. We then use TeaLeaf to compare the standard CG algorithm with a Chebyshev Polynomially Preconditioned Conjugate Gradient (CPPCG) iterative sparse linear solver. CPPCG is a communication-avoiding algorithm, requiring less global communication than previous approaches. TeaLeaf includes support for many-core processors, such as GPUs and Xeon Phi, and we include strong-scaling results across a range of world-leading Petascale supercomputers, including Titan and Piz Daint.

The continuous growth of high-performance computing (HPC) systems has lead to Fault Tolerance (FT) being identified as one of the major challenges for exascale computing, due to the expected decrease in Mean Time Between Failures (MTBF). One source of faults are soft errors, which can cause bit corruptions to the data held in memory. Current solutions for protection against these errors include hardware Error Correcting Codes (ECC), which incur overheads in power, memory bandwidth and storage, while also introducing more complexity to the hardware. In this paper we demonstrate Application-Based Fault Tolerance (ABFT) as an alternative method of protecting sparse matrices and dense vectors from data corruptions, requiring no additional dedicated memory storage. We use TeaLeaf, a heat conduction miniapp from the Mantevo Project, to demonstrate how these ABFT techniques can be adapted and applied to a sparse matrix solver-based application and its underlying data structures in order to improve reliability and performance.

Summary In this work, we evaluate several emerging parallel programming models: Kokkos, RAJA, OpenACC, and OpenMP 4.0, against the mature CUDA and OpenCL APIs. Each model has been used to port Tealeaf, a miniature proxy application, or mini app, that solves the heat conduction equation and belongs to the Mantevo Project. We find that the best performance is achieved with architecture‐specific implementations but that, in many cases, the performance portable models are able to solve the same problems to within a 5% to 30% performance penalty. While the models expose varying levels of complexity to the developer, they all achieve reasonable performance with this application. As such, if this small performance penalty is permissible for a problem domain, we believe that productivity and development complexity can be considered the major differentiators when choosing a modern parallel programming model to develop applications like Tealeaf.

Supercomputers offer new opportunities for scientific computing as they grow in size. However, their growth also poses new challenges. Resilience has been recognized as one of the most pressing issues to solve for extreme scale computing. Transistor scaling in the single-digit nanometer era and power constraints might dramatically increase the failure rate of next generation machines. DRAM errors have been analyzed in the past for different supercomputers but those studies are usually based on job scheduler logs and counters produced by hardware-level error correcting codes. Consequently, little is known about errors escaping hardware checks, which lead to silent data corruption. This work attempts to fill that gap by analyzing memory errors for over a year on a cluster with about 1000 nodes featuring low-power memory without error correction. The study gathered millions of events recording detailed information of thousands of memory errors, many of them corrupting multiple bits. Several factors are analyzed, such as temporal and spatial correlation between errors, but also the influence of temperature and even the position of the sun in the sky. The study showed that most multi-bit errors corrupted non-adjacent bits in the memory word and that most errors flipped memory bits from 1 to 0. In addition, we observed thousands of cases of multiple single-bit errors occurring simultaneously in different regions of the memory. These new observations would not be possible by simply analyzing error correction counters on classical systems. We propose several directions in which the findings of this study can help the design of more reliable systems in the future.

In this paper we demonstrate techniques for increasing the node-level parallelism of a deterministic discrete ordinates neutral particle transport algorithm on a structured mesh to exploit many-core technologies. Transport calculations form a large part of the computational workload of physical simulations and so good performance is vital for the simulations to complete in reasonable time. We will demonstrate our approach utilizing the SNAP mini-app, which gives a simplified implementation of the full transport algorithm but remains similar enough to the real algorithm to act as a useful proxy for research purposes. We present an OpenCL implementation of our improved algorithm which demonstrates a speedup of up to 2.5x the transport sweep performance on a many-core GPGPU device compared to a state-of-the-art multi-core node, the first time this scale of speedup has been achieved for algorithms of this class.

High-performance computing (HPC) systems continue to increase in size in the quest for ever higher performance. The resulting increased electronic component count, coupled with the decrease in feature sizes of the silicon manufacturing processes used to build these components, will result in future Exascale systems being more susceptible to soft errors caused by cosmic radiation than current HPC systems. Through the use of techniques such as hardware-based error-correcting codes (ECC) and checkpoint-restart, many of these faults can be mitigated, but at the cost of increased hardware overhead, run-time, and energy consumption that can be as much as 10 - 20%. For extreme scale systems, these overheads will represent megawatts of power consumption and millions of dollars of additional hardware cost, which could potentially be avoided with more sophisticated fault-tolerance techniques. In this paper we present a new software-based fault tolerance technique that can be applied to one of the most important classes of software in HPC: sparse matrix solvers. Our new technique enables us to exploit knowledge of the structure of sparse matrices in such a way as to improve the performance, energy efficiency and fault tolerance of the overall solution.

With the advent of heterogeneous computing systems consisting of multi-core central processing units (CPUs) and many-core graphics processing units (GPUs), robust methods are needed to facilitate fair benchmark comparisons between different systems. In this paper, we present a benchmarking methodology for measuring a number of performance metrics for heterogeneous systems. Methods for comparing performance and energy efficiency are included. Consideration is given to further metrics, such as associated running costs and carbon emissions. We give a case study for these metrics applied to Bristol University Docking engine, a molecular mechanics-based docking application that has been ported to open computing language at the University of Bristol. Results are included for both AMD and NVIDIA GPUs, and for a highly optimized code on the latest x86 CPUs.