Many enterprises perform data-center transformation, consolidation, and migration in order to improve the efficiency of their IT infrastructures. These transformation projects begin with the discovery of the existing infrastructure, in particular the dependencies between applications. These dependencies are needed in planning, in order to determine how components influence one another, and in relinking, so that the component names and addresses can be updated. Typically, dependency discovery is done by network monitoring and middleware configuration analysis. These existing approaches will often fail to detect dependencies expressed in the application code. In this paper, we present the first method and tool for automatically identifying code-embedded external dependencies in Java Enterprise Edition applications. In addition, our tool can automatically alter the application code to update the dependencies, or externalize them to configuration files. We analyzed over 1000 Java EE applications from three enterprise environments. The results demonstrate the prevalence of code-embedded dependencies that would otherwise have to be identified manually, often causing failures during user-acceptance testing.

Discovering application dependency on data and storage is a key prerequisite for many storage optimization tasks such as data assignment to storage tiers, storage consolidation, virtualization, and handling unused data. However, in the real world these dependencies are rarely known, and discovering them is a challenge because of virtualization at various levels and the need for discovery methods to be non-intrusive. As a result, many optimization tasks are performed, if at all, without the full knowledge of application-to-storage dependencies.This paper presents a non-intrusive application-to-storage discovery method, and while it is built on our prior work, the storage discovery described here is entirely new. We used this discovery method in two production enterprise environments, consisting of about 323 servers, and we show how the discovered data enables three optimization tasks. First, we relate application criticality with storage tiers. Second, we find unused storage devices and we show how this information together with storage consolidation can be used to achieve power savings of up to two orders of magnitude. Third, we identify opportunities for database storage optimization.

Hard disks contain data - frequently an irreplaceable asset of high monetary and non-monetary value. At the same time, hard disks are mechanical devices that consume power, are noisy, and fragile when their platters are rotating.

Benchmarking is critical when evaluating performance, but is especially difficult for file and storage systems. Complex interactions between I/O devices, caches, kernel daemons, and other OS components result in behavior that is rather difficult to analyze. Moreover, systems have different features and optimizations, so no single benchmark is always suitable. The large variety of workloads that these systems experience in the real world also adds to this difficulty. In this article we survey 415 file system and storage benchmarks from 106 recent papers. We found that most popular benchmarks are flawed and many research papers do not provide a clear indication of true performance. We provide guidelines that we hope will improve future performance evaluations. To show how some widely used benchmarks can conceal or overemphasize overheads, we conducted a set of experiments. As a specific example, slowing down read operations on ext2 by a factor of 32 resulted in only a 2--5% wall-clock slowdown in a popular compile benchmark. Finally, we discuss future work to improve file system and storage benchmarking.

Developing file systems from scratch is difficult and error prone. Using layered, or stackable, file systems is a powerful technique to incrementally extend the functionality of existing file systems on commodity OSes at runtime. In this article, we analyze the evolution of layering from historical models to what is found in four different present day commodity OSes: Solaris, FreeBSD, Linux, and Microsoft Windows. We classify layered file systems into five types based on their functionality and identify the requirements that each class imposes on the OS. We then present five major design issues that we encountered during our experience of developing over twenty layered file systems on four OSes. We discuss how we have addressed each of these issues on current OSes, and present insights into useful OS and VFS features that would provide future developers more versatile solutions for incremental file system development.

We have designed a stackable file system called Redundant Array of Independent Filesystems (RAIF). It combines the data survivability properties and performance benefits of traditional RAIDs with the unprecedented flexibility of composition, improved security, and ease of development of stackable file systems. RAIF can be mounted on top of any combination of other file systems including network, distributed, disk-based, and memory-based file systems. Existing encryption, compression, antivirus, and consistency checking stackable file systems can be mounted above and below RAIF, to efficiently cope up with slow or unsecure branches. Individual files can be distributed across branches, replicated, stored with parity, or stored with erasure correction coding to recover from failures on multiple branches. Per-file incremental recovery, storage type migration, and load-balancing are especially well suited for grid storages. In this paper, we describe the current RAIF design, provide preliminary performance results and discuss current status and future directions.

Operating systems are complex and their behavior depends on many factors. Source code, if available, does not directly help understand the OS's behavior, as the behavior depends on actual workloads and external inputs. Runtime profiling is a key technique for understanding the behavior and mutual-influence of modern OS components. Such profiling is useful to prove new concepts, debug problems, and optimize the performance of existing OSs. Unfortunately, existing profiling methods lack in important areas: they do not provide much of the necessary information about the OS's behavior; they require OS modification and therefore are not portable; or they exact high overheads thus perturbing the profiled OS.

Deployment of asynchronous learning systems is still very limited mainly because of the use of expensive video to record and play back classroom lectures. The key advantage of the Lectern II approach is that all important lecture components can be effectively captured without video. Lectern II employs the touch-sensitive screen technology to build a "digital desk," which is shown to be able to effectively support and transparently capture the standard classroom lecturing activity. Recorded lectures can be edited and automatically uploaded to a Web server, and than viewed by students via a standard streaming player. As the total cost of a complete Lectern II system is under $4,000, Lectern II represents the first course lecture recording system that has the potential to be widely deployed in the classrooms of universities and K-12 schools.

Content replication and distribution is an effective technology to reduce the response time for Web accesses and has been proven quite popular among large Internet content providers. However, existing content distribution systems assume a store-and-forward delivery model and is mostly based on static content. This paper describes the design, implementation, and initial evaluation of a network resource management system for real-time Internet content distribution called Sago, which provides facilities to provision and allocate network resources so that multiple bandwidth-guaranteed and fault-tolerant multicast connections can be multiplexed on a single physical network. Sago includes a novel network resource mapping algorithm that takes into account both physical network topology and dynamic traffic demands, a network-wide fault tolerance mechanism that supports both node-level and link-level fault tolerance, and a hierarchical network link scheduler that provides performance protection among multicast connections sharing the same physical network link. Moreover, Sago does not require any IP multicasting support from underlying network routers because it performs application-level multicasting. The technologies underlying Sago are important building blocks for real-time content distribution networks, end-to-end quality of service guarantee over global corporate intranets, and application-specific adaptation of wide-area network services.