In this paper, we describe a technique for detecting race conditions between direct memory access (DMA) operations and load/store instructions using a full system simulator. Our approach uses event monitoring features of a full system simulator to monitor DMA operations and the memory areas they access and detect conflicting accesses that could represent races conditions. Our race condition checker tracks DMA operations from the time they are issued until they are architecturally guaranteed to be complete, rather than simply tracking when they actually complete, and thus detects race conditions in programs even when the actual data accesses do not occur out of order. This feature is valuable because the mechanisms for ensuring ordering of asynchronous DMA operations are complex and often poorly understood by application programmers. These DMA operations may conflict with each other or with loads and stores performed by processor that initiated the operations, creating ample opportunity for race conditions to occur. We describe our race condition checker in detail and show how it can be used to easily detect race conditions in DMA operations initiated by special purpose cores.

A trend is developing in high performance computing in which commodity processors are coupled to various types of computational accelerators. Such systems are commonly called hybrid systems. In this paper, we describe our experience developing an implementation of the Linpack benchmark for a petascale hybrid system, the LANL Roadrunner cluster built by IBM for Los Alamos National Laboratory. This system combines traditional x86-64 host processors with IBM PowerXCell™ 8i1 accelerator processors. The implementation of Linpack we developed was the first to achieve a performance result in excess of 1.0 PFLOPS, and made Roadrunner the #1 system on the Top500 list in June 2008. We describe the design and implementation of hybrid Linpack, including the special optimizations we developed for this hybrid architecture. We then present actual results for single node and multi-node executions. From this work, we conclude that it is possible to achieve high performance for certain applications on hybrid architectures when careful attention is given to efficient use of memory bandwidth, scheduling of data movement between the host and accelerator memories, and proper distribution of work between the host and accelerator processors.

An instruction set level reference model was developed for the development of synergistic processing unit (SPU), which is one of the key components of the cell processor [Pham, 2005][Flachs, 2005]. This reference model was used for the simulators to define the instruction set architecture (ISA), for the random test case generator, for the reference in the verification environment and for the software development. Using the same reference model for multiple purposes made it easier to keep up with the architecture changes at the early stage of the microprocessor development. Also including the reference model in the simulation environment increased the robustness for the random test executions and made it possible to find bugs that are usually difficult to catch.