High Performance Computing applications and platforms have been typically designed without regard to power consumption. With increased awareness of energy cost, power management is now an issue even for compute-intensive server clusters. In this work, we investigate the use of power management techniques for high performance applications on modern power-efficient servers with virtualization support. We consider power management techniques such as dynamic consolidation and usage of dynamic power range enabled by low power states on servers. We identify application performance isolation and virtualization overhead with multiple virtual machines as the key bottlenecks for server consolidation. We perform a comprehensive experimental study to identify the scenarios where applications are isolated from each other. We also establish that the power consumed by HPC applications may be application dependent, non-linear and have a large dynamic range. We show that for HPC applications, working set size is a key parameter to take care of while placing applications on virtualized servers. We use the insights obtained from our experimental study to present a framework and methodology for power-aware application placement for HPC applications.

Large enterprise systems need continuous monitoring at infrastructure, application and business levels to detect and prevent problem situations. Traditionally, automated monitoring solutions are programmed once at setup based on a set of well-defined monitoring objectives and handed over to the operations team. Such solutions have underlying data models that are often complex and semantically rich but in stable environments, this complexity is generally hidden from the operations team, who only need to make minor configuration changes (e.g. setting thresholds) as and when required. However, the situation is now rapidly changing with enterprise data centers being subject to continuous transformations as new software, hardware and process components get deployed or updated. This puts an immense burden on monitoring activity because not only thousands of different parameters need to get monitored but the addition and modification of service level objectives (SLOs) may happen continuously. We describe a monitoring system architecture which simplifies the task of authoring and managing SLOs in such dynamic and heterogeneous environments. At the heart of our approach is a lightweight and extensible data model that is derived from more complex configuration models, so as to only expose data relevant for monitoring to the operations team. Simple string-tags derived from this model are then used to label SLOs and associated data streams. The approach localizes programming to the data-sensor layer and makes authoring simpler than the specification of objects in an alternate richer but complex object-oriented representation. We also describe a tag-driven real-time visualization tool that can organize data streams using their accompanying tags and ease user navigation through large volumes of monitoring data.

Renewed focus on virtualization technologies and increased awareness about management and power costs of running under-utilized servers has spurred interest in consolidating existing applications on fewer number of servers in the data center. The ability to migrate virtual machines dynamically between physical servers in real-time has also added a dynamic aspect to consolidation. However, there is a lack of planning tools that can analyze historical data collected from an existing environment and compute the potential benefits of server consolidation especially in the dynamic setting. In this paper we describe such a consolidation recommendation tool, called ReCon. Recon takes static and dynamic costs of given servers, the costs of VM migration, the historical resource consumption data from the existing environment and provides an optimal dynamic plan of VM to physical server mapping over time. We also present the results of applying the tool on historical data obtained from a large production environment.

Key areas in managing e-commerce systems are problem prediction, root cause analysis, and automated problem remediation. Anticipating SLO violations by proactive problem determination (PD) is particularly important since it can significantly lower the business impact of application performance problems. The main contribution of this paper is to investigate proactive PD based on two important concepts: dependency graphs and dynamic runtime performance characteristics of resources that comprise an I/T environment. The authors show how one can calculate and use the contribution of all supporting resources for a transaction to the end-to-end SLO for that transaction. Higher order moments of these components' contributions are further tracked for proactive alerting. An important aspect of this process is the classification of user transactions based on the profile of their resource usage, enabling one to set appropriate thresholds for the different classes only. Combined with the complete or semi-complete dependency information, our approach confines the scope of potential root causes to a small set of components, thus enabling efficient performance problem anticipation and quick remediation.

Projection of high dimensional data into a lower dimensional subspace is required for human understanding of the health of an IT infrastructure. Over the past several years, there have been a large number of dimensionality reduction techniques that have been proposed. Their direct application for visualizing system monitoring data is challenged by two factors. First, system monitoring data does not lie in a metric space. Second, system monitoring data is intrinsically iquestmulti-resolutioniquest in that an event may lead to cascaded events (a server going down impacts one or more applications running on the server; a network outage may impact several network dependent components). Lower dimensional representations which do not take into account the intrinsic multi-resolution nature of the monitoring data are thus limited in their utility and challenge human comprehension. In this paper, we exploit the multi-resolution nature of the monitoring data and a 1-of-n representation of event data to construct navigable multi-resolution topology preserving views for visualizing system monitoring data. We also demonstrate the efficacy of the proposed approach using data from a real data center.

Traditional software development involves reusing and building upon off-the-shelf software libraries. This paradigm provides compile time availability of both, component interfaces as well as corresponding implementations. It enables developers to write programs that build on existing functionality without worrying much about the runtime. Service oriented computing presents a different environment that involves actively running components in the form of services. Here, a search step becomes necessary to first discover the component(s) that satisfy the specified requirements. Moreover, services may come up or go down dynamically leading to expiry of searched results which makes dependent programs highly brittle. In this paper, we propose a paradigm shift from objects to services as first class entities. We present service matching techniques defined across design time and runtime. Using these techniques we define language level operations involving services as operands leading to a novel approach for services based software development.

Managing the service level objectives (SLO) in environments that implement e-commerce systems is a challenging task. It typically involves a clear understanding of how user transactions are supported by the I/T resources that comprise the e-commerce system. This paper investigates a subset of this important management problem. Using transaction to resource dependencies, the authors show how one can experimentally calculate the extent to which supporting resources for a transaction contribute to the end-to-end SLOs for that transaction. An important aspect of this process is the classification of user transactions, based on the profile of their resource usage, enabling one to set appropriate thresholds for different classes. This approach is then used to aid the detection and remediation of application performance bottlenecks.

The growing popularity of e-business has stimulated web sites to evolve from static content servers to complex multi-tier systems built from heterogeneous server platforms. E-businesses now spend a large fraction of their IT budgets maintaining, troubleshooting, and optimizing these web sites. It has been shown that such system management activities may be simplified or automated to various extents if a dynamic dependency graph of the system were available. Currently, all known solutions to the dynamic dependency graph extraction problem are intrusive in nature, i.e. require modifications at application or middleware level. In this paper, we describe non-intrusive techniques based on data mining, which process existing monitoring data generated by server platforms to automatically extract the system component dependency graphs in multi-tier e-business platforms, without any additional application or system modification.

The prevailing practice for testing protocol implementations is direct code instrumentation to trigger specific states in the code. This leaves very little scope for reuse of the test cases. In this paper, we present the design, implementation, and evaluation of VirtualWire, a network fault injection and analysis system designed to facilitate the process of testing network protocol implementations. VirtualWire injects user-specified network faults and matches network events against anticipated responses based on high-level specifications written in a declarative scripting language. With VirtualWire, testing requires no code instrumentation and fault specifications can be reused across versions of a protocol implementation. We illustrate the effectiveness of VirtualWire with examples drawn from testing Linux's TCP implementation and a real-time Ethernet protocol called Rether. In each case, 10 to 20 lines of script is sufficient to specify the test scenario. VirtualWire is completely transparent to the protocols under test, and additional overhead in protocol processing latency it introduces is below 10% of the normal.

Summary for only given. Conventional digital video playback systems provide only limited user interactivity, mostly in the form of VCR-like controls. In this model, the temporal ordering and the spatial viewpoints of the video streams being viewed are completely determined at authoring time. In contrast, we have defined a form of interactive video called active video (see http://www.ecsl.cs.sunysb.edu//spl sim/anindya/avs/avs.html, 2002), which supports hyper-linking among related video sequences and interpolation of video sequences with neighboring viewpoints, to offer end users the additional flexibility of choosing the sequencing and the viewing angle (even virtual ones) at playback time. However, active video has a substantially higher storage and transmission cost due to multiple time-synchronized video streams capturing the dynamic scene and the pixel-level correspondence maps encoding the spatial association among the frame-pairs of all adjacent streams. The maps are interpolated at run-time to generate virtual views. We describe and evaluate the following three compression techniques that alleviate the storage and network transmission costs of active video: spatial video compression; lossy map compression; lossless map compression.

Content replication and distribution is an effective technology to reduce the response time for Web accesses and has been proven quite popular among large Internet content providers. However, existing content distribution systems assume a store-and-forward delivery model and is mostly based on static content. This paper describes the design, implementation, and initial evaluation of a network resource management system for real-time Internet content distribution called Sago, which provides facilities to provision and allocate network resources so that multiple bandwidth-guaranteed and fault-tolerant multicast connections can be multiplexed on a single physical network. Sago includes a novel network resource mapping algorithm that takes into account both physical network topology and dynamic traffic demands, a network-wide fault tolerance mechanism that supports both node-level and link-level fault tolerance, and a hierarchical network link scheduler that provides performance protection among multicast connections sharing the same physical network link. Moreover, Sago does not require any IP multicasting support from underlying network routers because it performs application-level multicasting. The technologies underlying Sago are important building blocks for real-time content distribution networks, end-to-end quality of service guarantee over global corporate intranets, and application-specific adaptation of wide-area network services.

The advent of executable contents such as Java applets exposes World Wide Web (WWW) users to a new class of attacks that were not possible before. Despite an array of security checking, detection, and enforcement mechanisms built into the language model, the compiler, and the run-time system of Java, serious security breach incidents due to implementation bugs still arose repeatedly in the past several years. Without a provably correct implementation of Java's security architecture specification, it is difficult to make any conclusive statements about the security characteristic of current Java virtual machines. The Spout project takes an alternative approach to address Java's security problems. Rather than attempt a provable secure implementation, we aim to confine the damages of malicious Java applets to selective machines, thus, preventing the machines behind an organization's firewall from being attacked by malicious or buggy applets. More concretely, Spout is a distributed Java execution engine that transparently decouples the processing of an incoming applet's application logic from that of graphical user interface (GUI), such that the only part of an applet that is actually running on the requesting user's host is the harmless GUI code. A unique feature of the Spout architecture that does not exist in other similar systems, is that it is completely transparent to and does not require any modifications to WWW browsers or class libraries on the end hosts. This paper describes the detailed design, implementation, and performance measurements of the first Spout prototype, which also incorporates run-time resource monitoring mechanisms to counter denial-of-service attacks.

The advent of executable contents such as Java applets exposes WWW users to a new class of attacks that were not possible before. Serious security breach incidents due to implementation bugs arose repeatedly in the past several years. Without a provably correct implementation of Java's security architecture specification, it is difficult to make any conclusive statements about the security characteristic of current Java virtual machines. The Spout project takes an alternative approach to address Java's security problems. Rather than attempt a provably secure implementation, we aim to confine the damages of malicious Java applets to selective machines, thus protecting resources behind an organization's firewall from attacks by malicious or buggy applets. Spout is essentially a distributed Java execution engine that transparently decouples the processing of an incoming applet's application logic from that of the graphical user interface (GUI), such that the only part of an applet that is actually running on the requesting user's host is the harmless GUI code. A unique feature of the Spout architecture that does not exist in other similar systems, is that it is completely transparent to and does not require any modifications to WWW browsers or class libraries on the end hosts. This paper describes the design, implementation, and performance measurements of the first Spout prototype, which also incorporates run-time resource monitoring mechanisms to counter denial-of-service attacks.

Extensive research in TCP's congestion control mechanism has resulted in an effective algorithm that gives fairly precise estimates on the available bandwidth on a given network path (J. Postel, 1981; V. Jacobson, 1988). However, most past efforts focused on enhancing the accuracy and robustness of the path bandwidth estimation algorithms for individual TCP connections. Relatively fewer attempts have been made to further improve data transport efficiency by sharing path bandwidth information among concurrent TCP connections with the same sources and destinations. The paper proposes an aggregate TCP-based congestion control algorithm (ATCP) that allows individual TCP connections to reach their fair shares of the available network path bandwidth more quickly, while still observing TCP's congestion control semantics. In addition, the proposed algorithm is guaranteed to perform no worse than current TCP congestion control algorithm in all cases, and is designed to be implemented in a way that is completely transparent to both ends of a TCP connection. ATCP is particularly useful for TCP connections that are short-lived and yet have a long round-trip delay, such as Web page transfers using HTTP 1.0. Our trace-driven simulation study shows that the aggregate congestion control algorithm can reduce the normalized transaction latency by a factor of up to 2, compared to standard TCP.

Phoenix is a real-time network-attached storage device (NASD) that guarantees real-time data delivery to network clients even across single disk failure. The service interfaces that Phoenix provides are best-effort/real-time reads/writes based on unique object identifiers and block offsets. Data retrieval from Phoenix can be serviced in server push or client pull modes. Phoenix's real-time disk subsystem performance results from a standard cycle-based scan-order disk scheduling mechanism. However, the disk I/O cycle of Phoenix is either completely active or completely idle. This on-off disk scheduling model effectively reduces the power consumption of the disk subsystem, without increasing the buffer size requirement. Phoenix also exploits unused disk storage space and maintains additional redundancy beyond the generic RAID5-style parity. This extra redundancy, typically in the form of block replication, reduces the time to reconstruct the data on the failed disk. This paper describes the design, implementation, and evaluation of Phoenix, one of the first, if not the first, NASDs that support fault-tolerant, real-time, and low-power network storage service.

streams with other media types such as audio is an es-Fasy-io-use audio[video authoting took play a cmcial Ta2ein moving multimedia pTogTamsfrom TeseaTchcu-Tiosify to main-stTeam applications.This papeT desckbes the design and implementation of an inteTactitre video authoTing system called Zodiac, which employs an innovative edit histoTy abstraction to suppoTt Sevemt unique editing jeafuTes notfoun~in existing commercial and Te-seaTch video editing systems.Zo disc pTOVideSUSeTSa concepti~ally clean and semantically poweTfil bran&ing history model of edit operations to oTganize the authoTingpTocess, and to navigate among the design aL teTnatives.In addition, by analyzing the edit histo~, Zodiac is able to Teiiably detect a composed stTeam5 shot and scene boznda~es, ~hi& faci~itate intemctive ~~ideobTowsiny.Zodiac ako featuTes a video object annotation capability that allows meTs to associate annotations to moving objects in a video sequence.The annotations themselves could be text, image, audio, OT ~'ideo.Zodiac is built on top of hf3fFS, a file system specifica~~ydesigned foT inteTac~~ve mu~~imedia de- velaprnent environments, and implements an internal bufieT manage~that suppoTk transparent lossless com pT~ssion/decompression.Shot/scene detection, video object annotation, and bu~eT management all exploit the edit historr information foT peTfomance optimization.A complete digitd video authoring environment must support two fundamental functions: capturing and generation of raw video cfips, and temporal arrangement of video segments with special inter-segment transition eflects.b addition, the abiity to synchronize video Permissionto make digilzl or hard copies of all or part of thrs \trork for personzl or cl=