The performance of AllReduce is crucial at scale. The recursive doubling with pairwise exchange algorithm theoretically achieves O (log 2 N ) scaling for short messages with N peers, but is limited by improvements in network latency. A multi-way exchange can be implemented using message pipelining, which is easier to improve than latency. Using our method, recursive multiplying, we show reductions in execution time of between 8% and 40% of AllReduce on a Cray XC30 over recursive doubling. Using a custom simulator we further explore the dynamics of recursive multiplying.

The performance of AllReduce is crucial at scale. The recursive doubling with pairwise exchange algorithm theoretically achieves O(log2 N) scaling for short messages with N peers, but is limited by improvements in network latency. A multi-way exchange can be implemented using message pipelining, which is easier to improve than latency. Using our method, recursive multiplying, we show reductions in execution time of between 8% and 40% of AllReduce on a Cray XC30 over recursive doubling.

OpenMP has become a popular programming paradigm, as it allows programmers to easily write portable, parallel programs that exploit the multi-threading capabilities of modern computer architectures.This BOF provides a major forum for discussion between the OpenMP Architecture Review Board and the parallel programming community.This year, we will present new OpenMP features under consideration for OpenMP Specification 3.0. The new features extend the expressibility of OpenMP and make it applicable to a wider set of applications. One feature is the Task Queue construct, which provides a flexible mechanism for specifying units of work that are not pre-computed at the start of the construct.In addition, we will overview a proposed lightweight OpenMP tool information interface that supports data collection for performance profiling and tracing tools.In this BOF you will get a chance to learn about new OpenMP features and proposals, ask questions, and share your ideas about OpenMP.

Blue Gene was designed as a special purpose system, offering unprecedented computing performance coupled with very low power consumption and cost for a limited set of applications. Adoption has however become more widespread and Blue Gene is now being used in a relatively diverse range of scientific disciplines. Exploring the range and type of application that can make effective use of these systems, and the relevant architectural features influencing performance, is therefore highly topical.This tutorial will focus on the performance of a series of applications and consider the techniques required to achieve optimal performance and scaling on the Blue Gene architecture. We will discuss the factors and bottlenecks that influence performance and cover techniques for compiler, memory and communication optimization.Attendees will have the opportunity to gain hands on experience through a series of practical sessions on The University of Edinburgh's Blue Gene system.