Recent advances in solid state technology have led to the introduction of solid state drives (SSDs). Today's SSDs store data persistently using NAND flash memory and support good random IO performance. Current work in exploiting flash in database systems has primarily focused on using its random IO capability for second level bufferpools below main memory. There has not been much emphasis on exploiting its persistence.

In this paper, we propose a new compression/decompression algorithm called LZB which belongs to a class of algorithms related to Lempel-Ziv (LZ). The distinguishing characteristic of LZB is that it allows decompression from arbitrary points in compressed data and decompressing a portion of compressed data does not require decompressing from the beginning. This is accomplished by setting a limit on how far back a reference in compressed data can directly or indirectly point to. This limit is defined as a sliding gate. In the well-known LZ77 algorithm, every match may be (directly or indirectly) referenced back to its first detected occurrence in the input. Hence, in order to decompress any part of the data we need to decompress from the beginning of compressed data. However, in LZB each match is only traced within the confinement of the designated gate size. This eliminates the need to decompress from the beginning. An example is illustrated in Figure 1. In order to take advantage of LZB and decompress data from any arbitrary location in the stream, the decoder needs to know where to start. During the compression, LZB stores a list of markers which specify distinct positions in the compressed version and their corresponding positions in the uncompressed version. LZB starts the decompression from the closest marker preceding the gate of the beginning of the data of interest.

Sharing resources such as caches and main memory bandwidth in multi-core systems requires a more sophisticated scheduling scheme. PAM is a low-overhead, user-level meta-scheduler which does not require any hardware or software changes. In particular, it operates by detecting resource congestions and providing guidelines to the standard system scheduler by limiting the assignment of processes to subsets of available cores. PAM contains a cache model that it uses to predict the impact of new schedules. PAM can be used to improve the system along three dimensions: performance, power, and energy consumption (and any combination of these three). On our prototype, we show individual benchmarks can improve by up to 33% and the overall system performance can be improved by as much as 14%.

Storage systems are the next frontier for providing protection against intrusion. Since storage systems see changes to persistent data, several types of intrusions can be detected by storage systems. Intrusion detection (ID) techniques can be deployed in various storage systems. In this paper, we study how intrusions can be detected at the block storage level and in SAN environments. We propose novel approaches for storage-based intrusion detection and discuss how features of state-of-the-art block storage systems can be used for intrusion detection and recovery of compromised data. In particular we present two prototype systems. First we present a real time intrusion detection system (IDS), which has been integrated within a storage management and virtualization system. In this system incoming requests for storage blocks are examined for signs of intrusions in real time. We then discuss how intrusion detection schemes can be deployed as an appliance loosely coupled with a SAN storage system. The major advantage of this approach is that it does not require any modification and enhancement to the storage system software. In this approach, we use the space and time efficient point-in-time copy operation provided by SAN storage devices. We also present performance results showing that the impact of ID on the overall storage system performance is negligible. Recovering data in compromised systems is also discussed.

In recent years, several user-level communication systems have been developed to eliminate the gap between the performance of networking technologies used in Network Based Computing platforms and that experienced by the applications. The Virtual Interface Architecture (VIA) specification has been recently developed to standardize these communication systems and to make their features available in commercial systems. In this paper, we take on a challenge of developing a communication substrate over VIA such that applications using the popular TreadMarks DSM package can take advantage of the enhanced communication performance of VIA. We discuss various design alternatives, derive the best set of these alternatives and implement them on two enhanced implementations of VIA (M-VIA and Berkeley VIA) on two different networking technologies, Gigabit Ethernet and Myrinet, respectively. We evaluate the performance of our implementation by using several microbenchmarks and applications. We show that the communication and wait times, and therefore the total execution times of different applications can be significantly reduced by using VIA. A reduction in the overall execution time up to 2.05 on an eight node system is demonstrated in comparison with the original UDP implementation. The new implementation also demonstrates better parallel speedup as the system size increases.

The Virtual Interface Architecture (VIA) has been recently proposed to standardize different existing user-level networking protocols for System Area Networks (SANs). Since the introduction of VIA, software and hardware implementations of VIA have become available. VIA has different components (such as doorbells completion queues, and virtual-to-physical address translation) and attributes (such as maximum transfer unit and reliability modes). Different implementations of VIA lead to different design strategies for efficiently implementing higher level communication layers/libraries (such as Message Passing Interface (MPI)). It also has implication on the performance of applications: Currently, there is no framework for evaluating different design choices and for obtaining insight about the design choices made in a particular implementation of VIA and their impact on the performance. In this paper, we address these issues by proposing a new microbenchmark suite called Virtual Interface Architecture Benchmark (VIBe) This suite consists of several microbenchmarks which are divided into three major categories: non-data transfer related micro-benchmarks, data transfer related micro-benchmarks, and programming model related micro-benchmarks. By using the new benchmark suite, the performance of VIA implementations can be evaluated under different communication scenarios and with respect to the implementation of different components and attributes of VIA. We demonstrate the use of VIBe to evaluate three implementations of VIA (M-VIA on Gigabit Ethernet, Berkeley VIA on Myrinet and cLAN VIA on Giganet). We show how the VIBe suite can provide insights to the implementation details of VIA and help software developers of programming model layers on top of VIA.

This paper considers the problem of efficient multicast on worm-hole routed irregular heterogeneous networks of workstations, using multiple unicast messages. The fundamental issues are the avoidance of link contention and effective use of the faster nodes in the system for distributing the multicast message. Previously proposed schemes have either considered optimization for heterogeneity or elimination of contention, but not both together. We present two algorithms that addresses both issues and demonstrate their superiority through simulation studies.

A new memory subsystem, called Memory Xpansion Technology (MXT), has been built for compressing main memory contents. MXT effectively doubles the physically available memory transparently to the CPUs, input/output devices, device drivers, and application software. An average compression ratio of two or greater has been observed for many applications. Since compressibility of memory contents varies dynamically, the size of the memory managed by the operating system is not fixed. In this paper, we describe operating system techniques that can deal with such dynamically changing memory sizes. We also demonstrate the performance impact of memory compression using the SPEC CPU2000 and SPECweb99 benchmarks. Results show that the hardware compression of memory has a negligible performance penalty compared to a standard memory for many applications. For memory starved applications and benchmarks such as SPECweb99, memory compression improves the performance significantly. Results also show that the memory contents of many applications can be compressed, usually by a factor of two to one.

The IBM RS/6000 SP system is one of the most cost-effective commercially available high performance machines. IBM RS/6000 SP systems support the Message Passing Interface standard (MPI) and LAPI. LAPI is a low level, reliable and efficient one-sided communication API library implemented on IBM RS/6000 SP systems. This paper explains how the high performance of the LAPI library has been exploited in order to implement the MPI standard more efficiently than the existing MPI. It describes how to avoid unnecessary data copies at both the sending and receiving sides for such an implementation. The resolution of problems arising from the mismatches between the requirements of the MPI standard and the features of LAPI is discussed. As a result of this exercise, certain enhancements to LAPI are identified to enable an efficient implementation of MPI on LAPI. The performance of the new implementation of MPI is compared with that of the underlying LAPI itself. The latency (in polling and interrupt modes) and bandwidth of our new implementation is compared with that of the native MPI implementation on RS/6000 SP systems. The results indicate that the MPI implementation on LAPI performs comparably to or better than the original MPI implementation in most cases. Improvements of up to 17.3 percent in polling mode latency, 35.8 percent in interrupt mode latency, and 20.9 percent in bandwidth are obtained for certain message sizes. The implementation of MPI on top of LAPI also outperforms the native MPI implementation for the NAS Parallel Benchmarks.

The IBM SP Switch-Connected NT cluster is one the newest clustering platforms available. In this paper, we discuss an experimental implementation of the Virtual Interface Architecture for this platform. We discuss different design issues involved in this implementation. In particular, we explain how the virtual-to-physical address translation can be implemented efficiently with a minimum Network Interface Card (NIC) memory requirement. We show how caching the VIA descriptors on the NIC can reduce the communication latency. We also present an efficient scheme for implementing the VIA door bells without any hardware support. A comprehensive performance evaluation study of the implementation is provided. The performance of the implemented VIA surpasses that of other existing software implementations of the VIA and is comparable to that of a hardware VIA implementation. The peak measured bandwidth for our system is observed to be 101.4 MBytes/s and the one-way latency for short messages is 18.2 microseconds. It is to be noted that the VIA implementation presented in this paper is not a part of any IBM product and no assumptions should be made regarding its availability as a product in the future.

The IBM RS/6000 SP is one of the most successful commercially available multicomputers. SP owes its success partially to the scalable, high bandwidth, low latency network. In this paper, we present the adaptive routing scheme used in the new SP network switch chip called the Switch2. We show that the adaptive routing methods outperform the oblivious routing methods on SP like multistage networks. It is shown that the adaptive routing increases the network throughout by up to 229% over oblivious routing in some cases. We also study the effect of output selection, functions on the network performance. We present six different output selection functions and study their performance for different system parameters and communication patterns through extensive simulation. The results show that three of these selection functions perform similar to each other and outperform the other selection functions consistently. Therefore, unlike previous research findings for meshes and tori, there is no need to use multiple (or hybrid) selection functions to obtain the best performance for the SP like bidirectional multistage interconnection networks. We also provide an analysis of the cost-effectiveness of different selection functions with respect to the complexity of their hardware implementations.

Networks of Workstations (NOW) have become an attractive alternative platform for high performance computing. Due to the commodity nature of workstations and interconnects and due to the multiplicity of vendors and platforms, the NOW environments are being gradually redefined as Heterogeneous Networks of Workstations (HNOW) environments. This paper presents a new framework for implementing collective communication operations (as defined by the Message Passing Interface (MPI) standard) efficiently for the emerging HNOW environments. We first classify different types of heterogeneity in HNOW and then focus on one important characteristic: communication capabilities of workstations. Taking this characteristic into account, we propose two new approaches Speed-Partitioned Ordered Chain (SPOC) and Fastest-Node First (FNF) to implement collective communication operations with reduced latency. We also investigate methods for deriving optimal trees for broadcast and multicast operations. Generating such trees is shown to be computationally intensive. It is shown that the FNF approach, in spite of its simplicity, can deliver performance within 1% of the performance of the optimal trees. Finally, these new approaches are compared with the approach used in the MPICH implementation on experimental as well as on simulated testbeds. On a 24-node existing HNOW environment with SGI workstations and ATM interconnection our approaches reduce the latency of broadcast and multicast operations by a factor of up to 3.5 compared to the approach used in the existing MPICH implementation. On a 64-node simulated testbed, our approaches can reduce the latency of broadcast and multicast operations by a factor of up to 4.5. Thus, these results demonstrate that there is significant potential for our approaches to be applied towards designing scalable collective communication libraries for current and future generation HNOW environments.