Cloud computing users are most concerned about the application turnaround time and the monetary cost involved. For lower monetary costs, less expensive services, like spot instances offered by Amazon, are often made available, albeit to their relatively frequent resource unavailability that leads to on-going execution being evicted, thereby undercutting execution performance. Meanwhile, multithreaded applications may take advantage of elastic resource availability and cost fluctuation inherent to the systems. However, their potential gains on utilizing spot instances would be contingent upon how they handle resource unavailability, calling for an effective checkpointing. This work presents design and implementation of our enhanced adaptive incremental checkpointing (EAIC) for multithreaded applications on the RaaS clouds under spot instance pricing. EAIC model takes into account spot instance revocation events, besides hardware failures, for fast and accurately predicting the desirable points of time to take checkpoints so as to markedly reduce the expected job turnaround time and the monetary cost. The experimental results from our established test bed on PARSEC benchmarks under real spot instance price traces from Amazon EC2 show that EAIC lowers both the application turnaround time and the monetary cost markedly (by up to 58% and 59%, respectively) in comparison to its recent checkpointing counterpart.

Multithreaded applications are common in high performance cloud computing systems, able to take advantage of elastic resource availability and cost fluctuation inherent to the systems. When applications involve many threads over more cores leased from the RaaS (Resource-as-a-Service) cloud under spot instance pricing for faster execution, resource unavailability are more likely to occur, undercutting execution performance gains potentially offered by those more cores. As a result, checkpointing is required to lower the adverse impact of resource unavailability on execution performance of such multithreaded applications. Given checkpointing often incurs expensive I/O to remote storage, this work presents design and implementation of our adaptive incremental checkpointing (AIC) for multithreaded applications on the RaaS clouds. AIC utilizes the idle cores for adaptive delta compression and remote checkpointing, significantly reducing the expected job turnaround time and the aggregated file size at remote storage. To ensure high compatibility and portability for AIC, we exploit techniques to avoid using kernel-specific data structures. AIC has been evaluated using PARSEC benchmarks on our established testbed, which resembles a multicore system acquired from the RaaS cloud. The results show that AIC noticeably reduces the expected turnaround time (by up to 37%) and the aggregated file size (by up to 8.3×) when compared to a recent multi-level checkpointing scheme with fixed checkpoint intervals.

Checkpointing has been widely adopted in support of fault-tolerance and job migration, with checkpoint files preferably kept also at remote storage to withstand unavailability/failures of local nodes in networked systems. Lately, I/O bandwidth to remote storage becomes the bottleneck for checkpointing on a large-scale system. This paper proposes an adaptive incremental checkpointing (AIC), aiming to reduce the checkpointing file size considerably so that its involved overhead is lowered and thus the expected job turnaround time drops. Given production multicore systems are observed to have unused cores often available, we design AIC to make use of separate cores for carrying out multi-level checkpointing with delta compression at desirable points of time adaptively. We develop a new Markov model for predicting the performance of such multi-level concurrent checkpointing, with AIC performance evaluated using six SPEC benchmarks under various system sizes. AIC is observed to lower the normalized expected turnaround time substantially (by up to 47%) when compared to its static counterpart and a recent multi-level checkpointing scheme with fixed checkpoint intervals.

The discovery and management of energy resources, especially at locations in the Gulf of Mexico, requires an economic but technically enhanced infrastructure. Research teams from Louisiana State University, University of Louisiana at Lafayette, and Southern University Baton Rouge are engaged in a collaborative effort to create a ubiquitous computing and monitoring system (UCoMS) for the discovery and management of energy resources. The UCoMS team has sucessfully addressed two difficult issues in this research: (1) the computational challenges faced by compute-intensive simulations for reservoir uncertainty analysis that requires thousands of simulations and deals with terabytes, and even petabytes, of data, (2) the development of a prototype wireless sensor network (WSN) infrastructure to collect and process realtime data from production locations. While the former requires the intensive computational power of the UCoMS grid resources, the latter requires efficient interfacing between WSN & grid. A unified workflow analysis has been performed to ensure smooth operation of both efforts and a unified portal has been created. This paper integrates the above two workflows and portals into a single platform. It illustrates the need for such integration for users with similar (but not same) goals and describes how to partition users among different groups with different access rights to ensure security within subgroups. Such a system can easily integrate future UCoMS sub-projects into a unified whole. Hence, our portal prototype serves as a good example of the benefit that may accrue from integrated workflows.