The complexity of configuring computing systems is a major impediment to the adoption of new information technology (IT) products and greatly increases the cost of IT services. This paper develops a model of configuration complexity and demonstrates its value for a change management system. The model represents systems as a set of nested containers with configuration controls. From this representation, we derive various metrics that indicate configuration complexity, including execution complexity, parameter complexity, and memory complexity. We apply this model to a J2EE-based enterprise application and its associated middleware stack to assess the complexity of the manual configuration process for this application. We then show how an automated change management system can greatly reduce configuration complexity.

In an effort to bring more structure to the task of automating systems management, we introduce an approach for incrementally introducing best-practice-based automation into IT service delivery and management. Our approach is process-centric, using best-practice processes to structure the automation, scope out high-value automation opportunities, identify interaction points between automation and its broader environment, and expose the overall impact of automation on the service delivery organization. We illustrate our approach by applying it to a specific example: reducing the cost of IT change management via a system that automates changes in the domain of software lifecycle management (such as installation/deployment and configuration)

The importance of benchmarking is increasing as every aspect of human life is relying on correct operation of computing systems. Although considerable efforts have been made, presently there are no widely accepted dependability benchmarks. The panelists, in the order shown above, address benchmarking of computer hardware, operating systems, applications, and systems.

The complexity of configuring computing systems is a major impediment to the adoption of new information technology (IT) products and greatly increases the cost of IT services. This paper develops a model of configuration complexity and demonstrates its value for a change management system. The model represents systems as a set of nested containers with configuration controls. From this representation, we derive various metrics that indicate configuration complexity, including execution complexity, parameter complexity, and memory complexity. We apply this model to a J2EE-based enterprise application and its associated middleware stack to assess the complexity of the manual configuration process for this application. We then show how an automated change management system can greatly reduce configuration complexity.

We describe an approach to quantitatively evaluating human-assisted failure-recovery tools and processes in the environment of modern Internetand enterprise-class server systems. Our approach can quantify the dependability impact of a single recovery system, and also enables comparisons between different recovery approaches. The approach combines aspects of dependability benchmarking with human user studies, incorporating human participants in the system evaluations yet still producing typical dependability-related metrics as results. We illustrate our methodology via a case study of a system-wide undo/redo recovery tool for e-mail services; our approach is able to expose the dependability benefits of the tool as well as point out areas where its behavior could use improvement.

We introduce the ROC-1 hardware platform, a large-scale cluster system designed to provide high availability for Internet service applications. The ROC-1 prototype embodies our philosophy of recovery-oriented computing (ROC) by emphasizing detection and recovery from the failures that inevitably occur in Internet service environments, rather than simple avoidance of such failures. ROC-1 promises greater availability than existing server systems by incorporating four techniques applied from the ground up to both hardware and software: redundancy and isolation, online self-testing and verification, support for problem diagnosis and concern for human interaction with the system.

We describe a methodology for identifying and characterizing dynamic dependencies between system components in distributed application environments such as e-commerce systems. The methodology relies on active perturbation of the system to identify dependencies and the use of statistical modeling to compute dependency strengths. Unlike more traditional passive techniques, our active approach requires little initial knowledge of the implementation details of the system and has the potential to provide greater coverage and more direct evidence of causality for the dependencies it identifies. We experimentally demonstrate the efficacy of our approach by applying it to a prototypical e-commerce system based on the TPC-W Web commerce benchmark, for which the active approach correctly identifies and characterizes 41 of 42 true dependencies out of a potential space of 140 dependencies. Finally, we consider how the dependencies computed by our approach can be used to simplify and guide the task of root-cause analysis, an important part of problem determination.

The goal of intelligent RAM (IRAM) is to design a cost-effective computer by designing a processor in a memory fabrication process, instead of in a conventional logic fabrication process, and include memory on-chip. To design a processor in a DRAM process one must learn about the business and culture of the DRAMs, which is quite different from microprocessors. The authors describe some of those differences and their current vision of IRAM applications, architectures, and implementations.

The lmbench suite of operating system microbenchmarks provides a set of portable programs for use in cross-platform comparisons. We have augmented the lmbench suite to increase its flexibility and precision, and to improve its methodological and statistical operation. This enables the detailed study of interactions between the operating system and the hardware architecture. We describe modifications to lmbench, and then use our new benchmark suite, hbench:OS, to examine how the performance of operating system primitives under NetBSD has scaled with the processor evolution of the Intel x86 architecture. Our analysis shows that off-chip memory system design continues to influence operating system performance in a significant way and that key design decisions (such as suboptimal choices of DRAM and cache technology, and memory-bus and cache coherency protocols) can essentially nullify the performance benefits of the aggressive execution core and sophisticated on-chip memory system of a modern processor such as the Intel Pentium Pro.