The performance of AllReduce is crucial at scale. The recursive doubling with pairwise exchange algorithm theoretically achieves O (log 2 N ) scaling for short messages with N peers, but is limited by improvements in network latency. A multi-way exchange can be implemented using message pipelining, which is easier to improve than latency. Using our method, recursive multiplying, we show reductions in execution time of between 8% and 40% of AllReduce on a Cray XC30 over recursive doubling. Using a custom simulator we further explore the dynamics of recursive multiplying.

The performance of AllReduce is crucial at scale. The recursive doubling with pairwise exchange algorithm theoretically achieves O(log2 N) scaling for short messages with N peers, but is limited by improvements in network latency. A multi-way exchange can be implemented using message pipelining, which is easier to improve than latency. Using our method, recursive multiplying, we show reductions in execution time of between 8% and 40% of AllReduce on a Cray XC30 over recursive doubling.

This paper presents McMPI, an entirely new MPI library written in C# using only safe managed-code, and performance results from low-level benchmarks demonstrating ping-pong latency and bandwidth comparable with MS-MPI and MPICH2.

We have previously documented the on-going work in the EUFORIA project to parallelise and optimise European fusion simulation codes, see. This involves working with a wide range of codes to try and address any performance and scaling issues that these codes have. However, as no two simulation codes are exactly the same, it is very hard to apply exactly the same approach to optimising a disparate range of codes. Indeed, it can be seen from that the codes investigated range in terms of performance and ability from well-optimised, highly parallelised codes, to serial or poorly performing codes. After analysing, optimising, and parallelising a range of codes it is, actually, possible to discern a number of distinct optimisation techniques or approaches/strategies that can be used to improve the performance or scaling of a parallel simulation code. This paper outlines the distinct approaches that we have identified, highlighting their benefits and drawbacks, giving an overview of the type of work that is often attempted for fusion simulation code optimisation.

Abstract We introduce HPCx—the U.K.'s new National HPC Service—which aims to deliver a world‐class service for capability computing to the U.K. scientific community. HPCx is targeting an environment that will both result in world‐leading science and address the challenges involved in scaling existing codes to the capability levels required. Close working relationships with scientific consortia and user groups throughout the research process will be a central feature of the service. A significant number of key user applications have already been ported to the system. We present initial benchmark results from this process and discuss the optimization of the codes and the performance levels achieved on HPCx in comparison with other systems. We find a range of performance with some algorithms scaling far better than others. Copyright © 2005 John Wiley & Sons, Ltd.

This paper describes an implementation of generic MPI-2 single sided communications for SUN-MPI. Our implementation is layered on top of point-to-point MPI communications and therefore can be adapted to other MPI implementations. The code is designed to co-exist with other MPI-2 single sided implementations (for example direct use of shared memory) providing a generic fall-back implementation for those communication paths where an optimised single-sided implementation is not available. MPI-2 single sided communications require the transfer of data-type information as well as user data. We describe a type packing and caching mechanism used to optimise the transfer of data-type information. The performance of this implementation is measured in comparison to equivalent point to point operations and the shared memory implementation provided by SUN.