Traditionally, there have been two approaches to providing an operating environment for high performance computing (HPC). A Full-Weight Kernel(FWK) approach starts with a general-purpose operating system and strips it down to better scale up across more cores and out across larger clusters. A Light-Weight Kernel (LWK) approach starts with a new thin kernel code base and extends its functionality by adding more system services needed by applications. In both cases, the goal is to provide end-users with a scalable HPC operating environment with the functionality and services needed to reliably run their applications. To achieve this goal, we propose a new approach, called Fused OS, that combines the FWK and LWK approaches. Fused OS provides an infrastructure capable of partitioning the resources of a multicoreheterogeneous system and collaboratively running different operating environments on subsets of the cores and memory, without the use of a virtual machine monitor. With Fused OS, HPC applications can enjoy both the performance characteristics of an LWK and the rich functionality of an FWK through cross-core system service delegation. This paper presents the Fused OS architecture and a prototype implementation on Blue Gene/Q. The Fused OS prototype leverages Linux with small modifications as a FWK and implements a user-level LWK called Compute Library (CL) by leveraging CNK. We present CL performance results demonstrating low noise and show micro-benchmarks running with performance commensurate with that provided by CNK.

We introduce mainstream computing, a collaborative system that dynamically checks a program--via runtime assertion checks--to ensure that it is running according to expectation. Rather than enforcing strict, statically-defined assertions, our system allows users to run with a set of assertions that are statistically guaranteed to fail at a rate bounded by a user-defined probability, pfail. For example, a user can request a set of assertions that will fail at most 0.5% of the times the application is invoked. Users who believe their usage of an application is mainstream can use relatively large settings for pfail. Higher values of pfail provide stricter regulation of the application which likely enhances security, but will also inhibit some legitimate program behaviors; in contrast, program behavior is unregulated when pfail = 0, leaving the user vulnerable to attack. We show that our prototype is able to detect denial of service attacks, integer overflows, frees of uninitialized memory, boundary violations, and an injection attack. In addition we perform experiments with a mainstream computing system designed to protect against soft errors.

Supercomputers and clouds both strive to make a large number of computing cores available for computation. More recently, similar objectives such as low-power, manageability at scale, and low cost of ownership are driving a more converged hardware and software. Challenges remain, however, of which one is that current cloud infrastructure does not yield the performance sought by many scientific applications. A source of the performance loss comes from virtualization and virtualization of the network in particular. This paper provides an introduction and analysis of a hybrid supercomputer software infrastructure, which allows direct hardware access to the communication hardware for the necessary components while providing the standard elastic cloud infrastructure for other components.

XCPU2 is the evolution of our XCPU process management system which allows the users to compose the environment of the remote cluster nodes to match that of their desktop workstation. This creates the illusion of cluster computation resources being a seamless extension of their desktop interface, facilitating cluster acceleration of workflows which can be composed and visualized on the end-user desktop. XCPU2 allows programs running on the cluster to use the same versions of the libraries and tools the user installed locally on their desktop, and access support files such as configuration and data in the familiar places in which they are located on the end-user's workstation. XCPU2 builds on our earlier work with the XCPU process management system. Like XCPU, XCPU2's interface is represented as a set of files exported by a 9P file server. It supports heterogeneous clusters and multiple head nodes. Unlike XCPU, it uses a pull instead of push model for distributed applications, binaries, and data. In this paper we describe the XCPU2 clustering model, its operation and how the per-job filesystem configuration can be used to solve some of the common problems when running a cluster.

The complexity of modern microprocessor design involving billions of transistors at increasingly denser scales creates many challenges particularly in the area of design reliability and predictable yields. Researchers at IBM's Austin Research Lab have increasingly depended on software based simulation of various aspects of the design and manufacturing process to help address these challenges. The computational complexity and sheer scale of these simulations have lead to the exploration of the application of high-performance hybrid computing clusters to accelerate the design process. Currently, the hybrid clusters in use are composed primarily of commodity workstations and servers incorporating commodity NVIDIA-based GPU graphics cards and TESLA GPU computational accelerators. We have also been experimenting with blade clusters composed of both general purpose servers and PowerXcell accelerators leveraging the computational throughput of the Cell processor. In this paper we will detail our experiences with accelerating our workloads on these hybrid cluster platforms. We will discuss our initial approach of combining hybrid runtimes such as CUDA with MPI to address cluster computation. We will also describe a custom cluster hybrid infrastructure we are developing to deal with some of the perceived shortcomings of MPI and other traditional cluster tools when dealing with hybrid computing environments.

This paper explores the use of extended shell pipeline operators to establish distributed workflows and correlate results.

The complexity of today’s computer systems poses a challenge to system administrators. Current systems comprise a multitude of inter-related software components running on different servers. In this paper, we propose the use of the stackable storage mechanism as the foundation of centralized systems management. At the management level, we show how this mechanism can be used to implement an infrastructure that allows administrators to perform typical tasks fast and effortlessly. In particular, we find that our prototype could have avoided 40% of the human mistakes observed experimentally by previous research. At the storage level, we identify three key characteristics of stackable storage that allow the definition of different policies with distinct performance and scalability behaviors. We quantitatively compare five storage policies under different workloads and conclude that stackable storage is a viable approach.

If the operating system could be specialized for every application, many applications would run faster. For example, Java virtual machines (JVMs) provide their own threading model and memory protection, so general-purpose operating system implementations of these abstractions are redundant. However, traditional means of transforming existing systems into specialized systems are difficult to adopt because they require replacing the entire operating system. This paper describes Libra, an execution environment specialized for IBM's J9 JVM. Libra does not replace the entire operating system. Instead, Libra and J9 form a single statically-linked image that runs in a hypervisor partition. Libra provides the services necessary to achieve good performance for the Java workloads of interest but relies on an instance of Linux in another hypervisor partition to provide a networking stack, a filesystem, and other services. The expense of remote calls is offset by the fact that Libra's services can be customized for a particular workload; for example, on the Nutch search engine, we show that two simple customizations improve application throughput by a factor of 2.7.

Energy efficiency is becoming an increasingly important feature for both mobile and high-performance server systems. Most processors designed today include power management features that provide processor operating points which can be used in power management algorithms. However, existing power management algorithms implicitly assume that lower performance points are more energy efficient than higher performance points. Our empirical observations indicate that for many systems, this assumption is not valid.We introduce a new concept called critical power slope to explain and capture the power-performance characteristics of systems with power management features. We evaluate three systems - a clock throttled Pentium laptop, a frequency scaled PowerPC platform, and a voltage scaled system to demonstrate the benefits of our approach. Our evaluation is based on empirical measurements of the first two systems, and publicly available data for the third. Using critical power slope, we explain why on the Pentium-based system, it is energy efficient to run only at the highest frequency, while on the PowerPC-based system, it is energy efficient to run at the lowest frequency point. We confirm our results by measuring the behavior of a web serving benchmark. Furthermore, we extend the critical power slope concept to understand the benefits of voltage scaling when combined with frequency scaling. We show that in some cases, it may be energy efficient not to reduce voltage below a certain point.

This paper describes a mechanism allowing nodes to hand-off active connections by utilizing connection splicing at an edge-switch serving as a gateway to a server cluster. The mechanism is primarily intended to be used as part of a content aware request distribution strategy. Our approach uses an extended form of network address translation which maps inbound connection information (ie., address, port, and sequence number) to a separate outbound connection. A key difference in our approach is that while the switch performs network address translation and TCP splicing, the actual hand-off is triggered by the back-end nodes. This relieves the switch of performing any application layer responsibilities. Nodes may hand-off connections by first initiating a new connection to the destination and then sending a message to the gateway which splices the two connections together. The gateway modifies subsequent packet headers in order to create a transparent hand-off. This mechanism requires no modification to the operating system on the servers or the clients and supports HTTP/1.1 persistent connections and pipelined requests. To test our design, we implemented a soft-switch using Linux Netfilter which includes the extended network address translation. We provide some preliminary performance analysis and make recommendations for future work.