Exponential increases in complexity and scale make variability a growing threat to sustaining HPC performance at exascale. Performance variability in HPC I/O is common, acute, and formidable. We take the first step towards comprehensively studying linear and nonlinear approaches to modeling HPC I/O system variability in an effort to demonstrate that variability is often a predictable artifact of system design. Using over 8 months of data collection on 6 identical systems, we propose and validate a modeling and analysis approach (MOANA) that predicts HPC I/O variability for thousands of software and hardware configurations on highly parallel shared-memory systems. Our findings indicate nonlinear approaches to I/O variability prediction are an order of magnitude more accurate than linear regression techniques. We demonstrate the use of MOANA to accurately predict the confidence intervals of unmeasured I/O system configurations for a given number of repeat runs - enabling users to quantitatively balance experiment duration with statistical confidence.

Computer systems form the backbone of computing from very small, mobile devices to the huge datacenters that power the digital economy. These systems often exhibit large degrees of variability in their performance that is little understood, but such variability threatens to severely diminish the effectiveness of critical systems upon which society relies. Funded by a large NSF grant, the VarSys project at Virginia Tech researches the sources of variability in computer systems and develops methods to overcome it. We believe it is crucial to raise awareness of the phenomena surrounding variability in computer systems at the undergraduate level. Towards this end, we are connecting the research techniques developed as part of this NSF award to ongoing classroom projects in a core systems course. Our key insight is to expose students to the phenomenon as it occurs in the systems software modules (e.g. a memory allocator, a fork-join thread pool) they are themselves developing in the course. We have implemented a web-based system that allows students to submit their own systems-level code to a specialized cluster which then benchmarks it while systematically varying a number of ordinal and categorical variables. These variables reflect environmental factors that can influence the performance of complex systems. Students are then presented with a visual statistical analysis of the results and asked to interpret those results. We have successfully deployed this system in 2 semesters to over 250 students and collected student data about their experience with this system and are documenting our progress towards these important learning objectives.

Highly-parallel, high-performance scientific applications must maximize performance inside of a power envelope while maintaining scalability. Emergent parallel and distributed systems offer a growing number of operating modes that provide unprecedented control of processor speed, memory latency, and memory bandwidth. Optimizing these systems for performance and power requires an understanding of the combined effects of these modes and thread concurrency on execution time. In this paper, we describe how an analytical performance model that separates pure computation time (C) and pure stall time (S) from computation-memory overlap time (O) can accurately capture these combined effects. We apply the COS model to predict the performance of thread and power mode combinations to within 7% and 17% for parallel applications (e.g. LULESH) on Intel x86 and IBM BG/Q architectures, respectively. The key insight of the COS model is that the combined effects of processor and memory throttling and concurrency on overlap trend differently than the combined effects on pure computation and pure stall time. The COS model is novel in that it enables independent approximation of overlap which leads to capabilities and accuracies that are as good or better than the best available approaches.

Following an exhaustive set of experiments, we identify slowdowns in I/O performance that occur when processor power and frequency are increased. Our initial analyses indicate slowdowns are more likely to occur and more acute when the number of parallel I/O threads increases and the variability between runs is high. We use a micro benchmark-driven methodology to simplify isolation of the root causes of I/O performance loss. We classify the observed performance loss into two categories: file synchronization and file write delays. We introduce LUC, a runtime system to Limit the Unintended Consequences of power scaling and dynamically improve I/O performance. We demonstrate the effectiveness of the LUC system running on two platforms for two critical parallel transaction-oriented workloads including a mail server (vermeil) and online transaction processing (lotp).

Power states in power-scalable systems are managed to maximize performance and reduce energy waste. Power-scalable processor capabilities (e.g., Intel Turbo Boost) embrace a "faster is better" approach to power management. While these technologies can vastly improve performance and energy efficiency, there is a growing body of evidence that "faster is not always better". For example, in some I/O intensive benchmarks, we observe up to 47% performance loss when running codes at faster (higher power) frequencies versus slower (lower power) frequencies. To the best of our knowledge, this is the first work to systematically and accurately pinpoint the root cause of these types of slowdowns. The lack of such studies is likely due to three challenges we overcome in this work: 1) high runtime system variance, 2) bottleneck isolation across user- and system-space boundaries, and 3) non-determinism in parallel codes. Our analytical model-driven approach isolates resource contention as the root cause of slowdowns at higher processor speeds and suggests solutions we test empirically. We propose and evaluate the use of a power-aware that can increase performance more than 3-fold over the default Linux kernel while maintaining comparable reliability in the I/O subsystem. Our work motivates the need for more studies that potentially reconsider the "faster is better" design paradigm.

Data centers account for a growing percentage of US power consumption. Energy efficiency is now a first-class design constraint for the data centers that support cloud services. Service providers must distribute their data efficiently across multiple data centers. This includes creation of data replicas that provide multiple copies of data for efficient access. However, selecting replicas to maximize performance while minimizing energy waste is an open problem. State of the art replica selection approaches either do not address energy, lack scalability and/or are vulnerable to crashes due to use of a centralized coordinator. Therefore, we propose, develop and evaluate a simple cost-oriented decentralized replica selection system named EDR (Energy-Aware Distributed Running system), implemented with two distributed optimization algorithms. We demonstrate experimentally the cost differences in various replica selection scenarios and show that our novel approach is as fast as the best available decentralized approach DONAR, while additionally considering dynamic energy costs. We show that an average of 12% savings on total system energy costs can be achieved by using EDR for several data intensive applications.

With the increasing energy cost in data centers, an energy efficient approach to provide data intensive services in the cloud is highly in demand. This paper solves the energy cost reduction problem of data centers by formulating an energy-aware replica selection problem in order to guide the distribution of workload among data centers. The current popular centralized replica selection approaches address such problem but they lack scalability and are vulnerable to a crash of the central coordinator. Also, they do not take total data center energy cost as the primary optimization target. We propose a simple decentralized replica selection system implemented with two distributed optimization algorithms (consensus-based distributed projected subgradient method and Lagrangian dual decomposition method) to work with clients as a decentralized coordinator. We also compare our energy-aware replica selection approach with the replica selection where a round-robin algorithm is implemented. A prototype of the decentralized replica selection system is designed and developed to collect energy consumption information of data centers. The results show that the total energy cost can be effectively reduced by using our decentralized replica selection system comparing with a round-robin method. It also has low calculation and communication overhead and can be easily adapted to the real world cloud environment.