Pre-copy live migration is a popular virtual machine (VM) live migration mechanism that has been used in most hypervisors. In this mechanism, a key parameter, namely the maximum tolerable downtime, is required for each migration. However, defining an appropriate value for this parameter is nontrivial, especially for the migration of a virtual machine running CPU-intensive and memory-intensive applications. If an inappropriate value is used, the migration may suffer poor performance. Time-bound Live Migration (TLM) is another live migration mechanism that was proposed to solve this problem. The TLM can operate automatically without the need for the maximum tolerable downtime. Thus, the operational burden and performance penalty due to the configuration of the maximum tolerable downtime are eliminated. The contributions of this study are two folds. First, a novel compression framework for TLM, namely TLMZ, is proposed. Second, the live migration performances of TLM and TLMZ are evaluated against those of the pre-copy mechanism. In our comparisons, the pre-copy mechanism was configured to operate using a range of maximum tolerable downtime values and a number of compression options. An extensive number of experiments have been conducted. In our experiments, the TLM, TLMZ, and pre-copy mechanisms are used to migrate VMs running four OpenMP NAS parallel benchmarks. Experimental results show that TLM and TLMZ are practical solutions for the migrations of VMs running CPU-intensive and memory-intensive applications.

Live migration of virtual machines is the ability to move running virtual machines between two computers with minimal downtime. Although various migration mechanisms such as pre-copy, post-copy, and state compression have been proposed, they may suffer long migration times when the migrating virtual machines run large computation and memory intensive workloads. This paper presents the design and implementation of a novel Time-bound, thread-based Live Migration (TLM) mechanism, where additional threads are added to the pre-copy live migration algorithm to handle virtual machine state transfers within a bounded time period. In the time-bound principle, the upper-bound migration time of a virtual machine is proportional to the size of the virtual machine's memory. We propose a CPU over-committing mechanism to minimize migration downtime and avoid performance impacts to other virtual machines when the migration threads are in operation. We have implemented a prototype implementation of TLM on KVM, and conducted experiments by migrating virtual machines running a number of Class D OpenMP and MPI NAS parallel benchmarks. Experimental results showed the following: (i) TLM finished live migration in a bounded time period. Users are able to measure progress of migration operation. (ii) The CPU over-committing mechanism can be used to minimize live migration downtime. However, communication performance of virtual machines during live migration also declined as the number of over-committed CPUs reduced. The patterns of decline depended on execution behaviors of the applications on the virtual machines. (iii) The execution time increases of the OpenMP and MPI versions of the MG and IS benchmarks in our experiments were approximately equal to the migration times of TLM. (iv) We evaluated our CPU over-committing mechanism against the auto-convergence mechanism recently developed in kvm-1.6. We found that both mechanisms have their pros and cons, and their performance results are varied with application. Based on these results, we believe that the TLM design is practical for live migration of virtual machines running memory-intensive workloads, and the time-bound principle is an important new feature for pre-copy live migration optimization.

Virtual machine check pointing is the mechanism to save virtual machine state to a file for later recovery. Traditional check pointing mechanisms can suffer a long delay and cause a long disruption of services since they have to stop virtual machines to save state, which could be large. In this study, a novel Thread-based Live Check pointing (TLC) mechanism is proposed. This mechanism leverages the pre-copy live migration mechanism introducing a checkpoint thread, which is responsible for the majority of the check pointing activities. While the checkpoint thread is saving the virtual machine state to persistent storage, the virtual machine thread is allowed to progress with normal execution. However, the virtual machine thread will be periodically interrupted to incrementally copy dirty memory pages to a hash table. The interruptions will occur until the final stage of check pointing is reached. This approach is implemented in KVM and its performance evaluations are conducted using NAS parallel benchmarks. Experiments show that this approach can provide high levels of virtual machine responsiveness during check pointing. It can also reduce the check pointing overheads to as low as 0.53 times of that of the traditional approach, when operating on a virtual machine running memory intensive workloads.

Virtual machine, which typically consists of a guest operating system (OS) and its serial applications, can be checkpointed, migrated to another cluster node, and restarted later to its previous saved state. However, to date, it is nontrivial to provide checkpoint-restart mechanisms with the same level of transparency for distributed applications running on a cluster of virtual machines. To address this particular issue, we have created the Virtual Cluster CheckPointing (VCCP) system, a novel system for transparent coordinated checkpoint-restart of virtual machines and its distributed application on commodity clusters. In this paper, we detail the design and implementation of the VCCP system. Our VCCP prototype extends the open source QEMU system with kqemu module by implementing hypervisor-based Coordinated Checkpoint-Restart protocols. To verify and validate our prototype, we measured its performance using the NAS parallel benchmark. Our experimental results indicate that VCCP generates less than 1% of additional execution overhead for non-communication intensive parallel applications. Furthermore, our correctness analysis shows that VCCP does not cause message loss or reordering, which is a necessary property to ensure correctness of checkpoint-restart mechanism. Finally, we believe that VCCP is a promising checkpoint-restart alternative for legacy applications that have implemented traditional process-level checkpoint-restart.

In order to adopt high performance clusters and grid computing for mission critical applications, fault tolerance is a necessity. Common fault tolerance techniques in distributed systems are normally achieved with checkpoint-recovery and job replication on alternative resources, in cases of a system outage. The first approach depends on the system's MTTR while the latter approach depends on the availability of alternative sites to run replicas. There is a need for complementing these approaches by proactively handling failures at a job-site level, ensuring the system high availability with no loss of user submitted jobs. This paper discusses a novel fault tolerance technique that enables the job-site recovery in Beowulf cluster-based grid environments, whereas existing techniques give up a failed system by seeking alternative resources. Our results suggest sizable aggregate performance improvement during an implementation of our method in Globus-enabled HA-OSCAR. The technique called ''smart failover" provides a transparent and graceful recovery mechanism that saves job states in a local job-manager queue and transfers those states to the backup server periodically, and in critical system events. Thus whenever a failover occurs, the backup server is able to restart the jobs from their last saved state

Understanding the Earth's climate system and how it might be changing is a preeminent scientific challenge. Global climate models are used to simulate past, present, and future climates, and experiments are executed continuously on an array of distributed supercomputers. The resulting data archive, spread over several sites, currently contains upwards of 100 TB of simulation data and is growing rapidly. Looking toward mid-decade and beyond, we must anticipate and prepare for distributed climate research data holdings of many petabytes. The Earth System Grid (ESG) is a collaborative interdisciplinary project aimed at addressing the challenge of enabling management, discovery, access, and analysis of these critically important datasets in a distributed and heterogeneous computational environment. The problem is fundamentally a Grid problem. Building upon the Globus toolkit and a variety of other technologies, ESG is developing an environment that addresses authentication, authorization for data access, large-scale data transport and management, services and abstractions for high-performance remote data access, mechanisms for scalable data replication, cataloging with rich semantic and syntactic information, data discovery, distributed monitoring, and Web-based portals for using the system.

In a dynamic environment where a process can migrate from one host to another host, communication state transfer is a key issue of process coordination. We present a set of data communication and process migration protocols to support communication state transfer in a dynamic, distributed parallel environment. The protocols preserve the semantics of point-to-point communication; they guarantee message delivery, maintain message ordering, and do not introduce deadlock when blocking send or receive operations are performed during process migration. Analytical proofs and prototype implementation are conducted to confirm the correctness of the protocols. Analytical and experimental results show the proposed design is valid and has a true potential in network computing.

In the emerging world of Grid Computing, shared computational, data, other distributed resources are becoming available to enable scientific advancement through collaborative research and collaboratories. This paper describes the increasing role of ontologies in the context of Grid Computing for obtaining, comparing and analyzing data. We present ontology entities and a declarative model that provide the outline for an ontology of scientific information. Relationships between concepts are also given. The implementation of some concepts described in this ontology is discussed within the context of the Earth System Grid II (ESG)[1].

Mobility is a fundamental functionality of the next generation Internet computing. How to support mobility for legacy codes, however, is still an issue of research. The key to solve this outstanding issue is the support of heterogeneous process migration. During the last few years, we have successfully developed mechanisms to support heterogeneous process migration of legacy codes written in C, C++, and Fortran. We present in this paper the design of the high performance computing mobility (HPCM) middleware, the development and implementation of its key components, pre-compiler and its static libraries. Due to the similarity between process migration and checkpointing, the pre-compiler not only makes automatic process migration of legacy codes feasible, but also supports dynamic heterogeneous checkpointing. We perform a set of tests and compare experimental results with Porch, a well-known portable heterogeneous checkpointing system. The experimental results show that our methods are feasible, efficient and very promising.

Abstract This study presents a practical solution for data collection and restoration to migrate a process written in high‐level stack‐based languages such as C and Fortran over a network of heterogeneous computers. We first introduce a logical data model, namely the Memory Space Representation (MSR) model, to recognize complex data structures in process address space. Then, novel methods are developed to incorporate the MSR model into a process, and to collect and restore data efficiently. We have implemented prototype software and performed experiments on different programs. Experimental and analytical results show that: (1) a user‐level process can be migrated across different computing platforms; (2) semantic information of data structures in the process's memory space can be correctly collected and restored; (3) costs of data collection and restoration depend on the complexity of the MSR graph in the memory space and the amount of data involved; and (4) the implantation of the MSR model into the process is not a decisive factor of incurring execution overheads. With appropriate program analysis, we can practically achieve low overhead. Copyright © 2002 John Wiley & Sons, Ltd.

This paper presents the design of a communication state transfer protocol to support process migration in a dynamic, distributed computing environment. In our design, processes in distributed computation communicate one another via message passing and are migration-enabled. Due to mobility, mechanisms to maintain reliability and correctness of data communication are needed. Following an event-based approach, Such mechanisms are derived to handle various communication situations when a process migrates. These mechanisms collectively preserve the semantics of the communication and support efficient communication state transfer.

In a dynamic environment, where a process can be migrated from one host to another host, communication state transfer is a key issue of process coordination. This paper presents algorithms for data communication and migration protocols to support communication state transfer in a dynamic, distributed parallel environment. These algorithms collectively presence the semantics of the communication and are practical for large-scale distributed systems. The assumptions and validity of our solution are discussed. Based on our early results in process migration, we implement a prototype system for process state transfer. Experimental results confirm our design is valid and has a true potential in practice.

This study presents a practical solution for data collection and restoration to migrate a process written in high level stack-based languages such as C and Fortran over a network of heterogeneous computers. We study a logical data model which recognizes complex data structures in process address space. Then, novel methods are developed to incorporate the model into a process and to collect and restore data efficiently. We have implemented a prototype software and performed experiments on different programs. Experimental and analytical results show that (I) a user-level process can be migrated across different computing platforms, (2) semantic information of data structures in the process's memory space can be correctly collected and restored, (3) the costs of data collection and restoration depend on the complexity of the logical model representing the process's data structures and the amount of data involved and (4) the implantation of the data collection and restoration mechanisms into the process is not a decisive factor of incurring execution overheads; with appropriate program analysis, we can achieve practically low overhead.