As the security issue of cloud-offloaded Deep Learning (DL) inference is drawing increasing attention, running DL inference in Trusted Execution Environments (TEEs) has become a common practice. Latency prediction of TEE-hosted DL model inference is essential for many scenarios, such as DNN model architecture searching with a latency constraint or layer scheduling in model-parallelism inference. However, existing solutions fail to address the memory over-commitment issue in resource-constrained environments inside TEEs.This paper presents Sectum, an accurate latency predictor for DL inference inside TEE enclaves. We first perform a synthetic empirical study to analyze the relationship between inference latency and memory occupation. Sectum predicts inference latency following a two-stage design based on some critical observations. First, Sectum uses a Graph Neural Network (GNN)-based model to detect whether a given model would trigger memory over-commitment in TEEs. Then, combining operator-level latency modeling with linear regression, Sectum could predict the latency of a model. To evaluate Sectum, we design a large dataset that contains the latency information of over 6k CNN models. Our experiments demonstrate that Sectum could achieve over 85% ±10% accuracy of latency prediction. To our knowledge, Sectum is the first method to predict TEE-hosted DL inference latency accurately.

Cloud computing has been widely adopted by personal developers and enterprises because of its on-demand and elastic resource usage paradigm. Currently most cloud applications are running on one single cloud. Cloud vendors provide users a bewildering choice of VMs and the choice can have a big impact on performance and cost. Therefore, it is difficult to choose the appropriate VMs to process jobs efficiently and economically. Besides, more and more applications are demanding to run across several clouds with heterogeneous resources to satisfy the requirements like best cost efficiency, avoidance of vender lock-in, and geolocation sensitive service. Existing systems lack a mechanism to unify the management of heterogeneous computing resources across clouds. In this paper, we present Docklet to build a virtual private cloud in JointCloud environments, managing heterogeneous and distributed cloud resources and selecting suitable resources according to the needs of users. At the same time, it provides computing framework support for scientific computing workloads, having the customized software stacks, configurations readily available. With Docklet, users only need one browser to do their jobs online. In the end, we evaluate the performance of running scientific workloads in Docklet.

Hyper-parameter tuning (HPT) is crucial for many machine learning (ML) algorithms. But due to the large searching space, HPT is usually time-consuming and resource-intensive. Nowadays, many researchers use public cloud resources to train machine learning models, convenient yet expensive. How to speed up the HPT process while at the same time reduce cost is very important for cloud ML users. In this paper, we propose SpotTune, an approach that exploits transient revocable resources in the public cloud with some tailored strategies to do HPT in a parallel and cost-efficient manner. Orchestrating the HPT process upon transient servers, SpotTune uses two main techniques, fine-grained cost-aware resource provisioning, and ML training trend predicting, to reduce the monetary cost and runtime of HPT processes. Our evaluations show that SpotTune can reduce the cost by up to 90% and achieve a 16.61x performance-cost rate improvement.

EMR (Elastic Map Reduce) is a service provided by mainstream cloud vendors for data processing users to directly obtain well-managed Hadoop YARN clusters on the cloud. Preemptible instance is a kind of cloud server that is cheap but is likely to be reclaimed by cloud vendors suddenly. Running EMR clusters on preemptible instances relies on YARN's own fault-tolerance, which is limited. In this paper, we present PrTaurus as an availability-enhanced EMR service on preemptible instances. PrTaurus integrates a system-level checkpoint capability based on Docker into YARN to further improve its fault-tolerance. In addition, PrTaurus's scheduling strategy takes advantage of Alibaba Cloud's one-hour protection policy. Furthermore, a new method that comprehensively considers cost-efficiency, preemption risk and overhead is proposed to select cluster instances. We evaluated PrTaurus through simulations on real-world workload and instance price traces. Experimental results show that compared with the existing EMR clusters running on preemptible instances, PrTaurus significantly reduces cost (13.0%-74.6%), instance preemptions (60.3%-88.9%), and task preemptions (86.0 % - 98.6 %).

Nowadays, as more and more developers choose to deploy their work on the cloud, diverse workloads are being run in the cloud. Recognizing the diversity, cloud providers offer a rich selection of VMs with a large configuration space. The choice of cloud service can have a big impact on performance and cost. However, there are few guidelines given by the cloud providers. Choosing the right cloud service is difficult for users only with configuration information. Besides, there lacks a comparison between different cloud providers. To aid in the selection of VMs, we present CloudMeter, a tool to continuously run benchmarks on VMs of different cloud providers and grade each VM. With the grade, we recommend appropriate cloud service according to the requirement of cloud users. We also use CloudMeter to draw significant insights into cloud services among different cloud providers.

Modern cloud vendors provide thousands of types of virtual machines, presenting the user a bewildering choice. Many previous works try to train accurate models for a workload to predict its performance on a given configuration, which could be applicable to a concrete class of workloads (e.g., Spark-SVM jobs). But it is difficult to build a universal prediction model for all the workloads, mostly because different workloads have different runtime metrics distribution. We argue that if the runtime metrics data could be transferred before being fed to the prediction model, the knowledge learned in one workload may be reused to other workloads. In this paper, we synthetically investigate the similarity and difference between a set of workloads and the possibility to transfer between them. We use an LSTM auto-encoder to encode the temporal runtime metrics into a fixed-size vector, which is fed into a linear regression model. In our experiments, the metrics transferring approach could help reduce the prediction error of the linear regression model.

The increasing popularity of cloud storage is leading many organizations to move their data into the cloud. However, putting all data in one cloud causes problems such as vendor lock-in, increased service costs, and data availability. In this paper, we introduce DCStore, a Cloud-of-Clouds storage service designed for an organization to outsource their data into the clouds. To achieve the goal of cost-efficient and high-available, we combine three key techniques. First, DCStore eliminates the redundant data at client-side to save storage cost via application-aware chunking method. Second, DCStore uses an inner-chunk based erasure coding scheme to distribute unique chunks across multiple clouds for high availability. Finally, a container-based share management strategy is used for performance optimization. Our experimental evaluations show that DCStore can improve the performance and cost efficiency significantly, compared with existing Cloud-of-Clouds storage systems.

The size of data sets being collected and analyzed in data science field is growing rapidly, making traditional big data processing solution prohibitively expensive. Especially when the data sets are too large, distributed techniques are inevitable even for simple embarrassing parallel jobs. However, distributed computing is still inaccessible to a large number of users. For example, many average users are still struggling with complex cluster management and configuration tools[24] even just for summing up a group of numbers in a large data file. In this paper, we present BDViewer, a web-based big data processing and visualizing tool. BDViewer uses JavaScript plugins to enable users to view, process and visualize their large data files just through a web browser. By just clicking a button, users can open a large data file online and view the file contents immediately no matter how large the file is. In the back-end, BDViewer is built on a virtual private cloud system. Users' operations in a web browser are converted into map-reduce jobs and MPI tasks that are executed on the cloud. At the end of this paper, some experiments are carried out, which demonstrate BDViewer's effectiveness and ease of use.

The emergence of cloud computing makes it easier for enterprises and individual users to obtain distributed computing resources. At the same time, developing distributed systems is still complex and difficult. In order to reduce the complexity in distributed application development, a series of techniques such as RPC, message-oriented middleware have been proposed and widely used. These techniques hide the underlying communicating details but require developers to take control of coordination among different machines for complex distributed tasks. This paper proposes a lightweight event-driven framework called DistGear to simplify the development of complex task processing in distributed applications. DistGear is a layer of software built on top of communication facilities for providing developers with a programming abstraction to easily handle distributed tasks and control machine coordination in tasks. We implement DistGear based on coroutines and message communication. Experience with DistGear in a simplified real-world system suggests it is effective. Performance evaluation also demonstrates DistGear's good concurrency and low resources consumption.

Service Level Agreements (SLAs) play an important role in service-based systems. However, traditional approaches to establish SLAs are mostly manual and predefined which is not suitable for the highly dynamic and unpredictable service-oriented environment. In this paper, we propose a policy-based framework for supporting dynamic and automated SLA negotiations for Web services. In our framework, we extend the WS-Policy framework to provide a domain-independent policy language for specifying QoS constraints over the QoS attributes that are to be negotiated. Negotiation agents are dynamically created to perform SLA negotiations on behalf of each negotiating party in a P2P way using standard web services invocations. Decision making models of negotiation agents are also defined in a declarative way and can be reconfigured easily. We have implemented a prototype of our framework and demonstrated our approach through a case study.

Determinism and throughput are two important performance measures for Java-based real-time applications, but they often conflict. Therefore, it is significant to improve throughput for Java-based real-time applications while guaranteeing its execution time determinism. In this paper, we propose an automatic configuration approach to assign real-time thread priorities to solve the above-mentioned problem. In this approach, we propose an innovative representation of determinism related with real-time thread priorities using stochastic process. Java-based real-time application's throughput is quantified with thread priorities as parameters. The algorithm of integer programming is used to optimize throughput with boundary conditions of the level of determinism. Finally, the Sweet Factory application is tested to evaluate the effect of our approach. Experiment results show that throughput for Java-based real-time applications could be efficiently improved while keeping the execution time determinism with our approach.

Service Level Agreements (SLAs) play an important role in guaranteeing successful collaborations among autonomous entities in Internet-based Virtual Computing Environment (iVCE). However, traditional static and predefined SLAs are not suitable for the open and dynamic iVCE. This paper proposes a policy-based framework for supporting dynamic and automated SLA negotiations between autonomous entities in iVCE. Requirements or constraints on SLA attributes as well as negotiation strategies of negotiating parties are both specified in policy specifications using a simple but expressive policy language that extends the WS-Policy framework. According to these policies, a negotiation agent will be dynamically created for each negotiation party, which is responsible for SLA negotiations on behalf of its owner party. We have implemented a prototype of our framework and demonstrated our approach through a case study.

In order to support EJB3.0, many open-source application server vendors have implemented a new container which is independent with the existing EJB2.x one. However, the functional redundancy between both of the containers increases the maintenance cost when the software are modified because of problems or new requirements. This paper proposes a Dual-container to support EJB2.x and EJB3.0 in a unified architecture. Software engineering perspective is applied, three principles including greatest-common-divisor, lease-common-multiple, and transparent intrusion, are presented to guide the construction of the architecture. We implemented the Dual-container in an open-source J2EE application server. The evaluation results show that the Dual-container is effective in reducing the functional redundancy and increasing the maintainability. We demonstrate that in a comparative analysis.

This paper describes a service oriented programming platform for Internet-based virtual computing environment (iVCE), which is targetted for enabling collaboration among autonomous entities over Internet. This platform provides mechanisms and facilities to help iVCE programmers write and run applications in a relatively stable and trusted environment. In this platform, autonomous entities first voluntarily join in virtual community attracted by common interests. Then entities will selectively play specific roles predefined in collaboration process which is specified in a declarative language named APEL. Capability requirements and environment specifications are also provided to enable negotiation and adaptation while process is executed. All business interfaces and negotiation interfaces are published using standard Web services to maximize reuse and interoperation.