Partially sharded data parallel (PSDP) reduces GPU memory footprint in data-parallel training without incurring additional communication overhead. Existing work on PSDP has primarily focused on minimizing memory consumption, such as by offloading model states to CPU, while not enough attention has been paid to poor communication performance. We observe that the imbalanced distribution of model states across GPUs and inter-layer data dependencies lead to severe underutilization of network bandwidth and poor scalability of the current PSDP system. To address these issues, we present Chimera, a system that enhances the communication efficiency of PSDP in both offloading and non-offloading settings. First, Chimera introduces a fine-grained state-sharding strategy to balance both model state and communication load across GPUs, while leveraging tensor fusion to amortize communication launch overhead. Second, we design a multi-stage, fine-grained communication pipelining mechanism that decouples inter-layer dependencies, thereby improving bandwidth utilization. Finally, we develop a practical auto-tuning tensor fusion algorithm to adaptively optimize fusion decisions for end-to-end training performance. We evaluate Chimera on cloud GPU clusters using representative large language models. Experimental results demonstrate that Chimera improves training throughput by$1.1 \times$to$1.8 \times$over state-of-theart PSDP systems.

Existing Data Parallel (DP) trainings for deep neural networks (DNNs) often experience limited scalability in speedup due to substantial communication overheads. While Overlapping technique can mitigate such problem by paralleling communication and computation in DP, its effectiveness is constrained by the high communication-to-computation ratios (CCR) of DP training tasks. Gradient compression (GC) is a promising technique to obtain lower CCR by reducing communication volume directly. However, it is challenging to obtain real performance improvement by applying GC into Overlapping because of (1) severe performance penalties in traditional GCs caused by high compression overhead and (2) decline of Overlapping benefit owing to the possible data dependency in GC schemes. In this paper, we propose COVAP, a novel GC scheme designing a new coarse-grained filter, makes the compression overhead close to zero. COVAP ensures an almost complete overlap of communication and computation by employing adaptive compression ratios and tensor sharding tailored to specific training tasks. COVAP also adopts an improved error feedback mechanism to maintain training accuracy. Experiments are conducted on Alibaba Cloud ECS instances with different DNNs of real-world applications. The results illustrate that COVAP outperforms existent GC schemes in time-to-solution by 1.92x-15.39x and exhibits near-linear scaling. Furthermore, COVAP achieves best scalability under experiments on four different cluster sizes.

Many distributed stream computing engines have emerged to handle big data, and they can be deployed in cloud environments consisting of native networks or container networks. Most of the benchmark research on stream computing engines are carried out under the native network, and the research on the impact on container network on stream computing engines is currently inadequate. However, the use of container network will inevitably lead to performance degradation, which is the disadvantage of all virtual networks. In this work, we build Apache Storm and Apache Flink, which are Streaming Computation Engines in container network and native network environments and conduct performance measurements through experiments processing textual data to verify how much performance decreases in container network. Experiments show that the throughput in a container network environment is 1%-5% lower and CPU utilization is 11%-18% lower than in a local network environment.

Real-time monitoring, providing the real-time status information of servers, is indispensable for the management of distributed systems, e.g. failure detection and resource scheduling. The scalability of fine-grained monitoring faces more and more severe challenges with scaling up distributed systems. The real-time compression which suppresses remote information update to reduce continuous monitoring cost is a promising approach to address the scalability problem. In this paper, we present the Linear Compression Algorithm (LCA) which is the application of the linear filter to real-time monitoring. To our best knowledge, existing work and LCA only explores the correlations of values of each single metric at various times. We present a novel lightweight REal-time Compression Algorithm (ReCA) which employs discovery methods of the correlation among metrics to suppress remote information update in distributed monitoring. The compression algorithms mentioned above have limited compression power because they only explore either the correlations of values of each single metric at various times or that among metrics. Therefore, we propose the Mixed Compression Algorithm (MiCA) which explores both of the correlations to achieve higher compression ratio. We implement our algorithms and an existing compression algorithm denoted by CCA in a distributed monitoring system Ganglia and conduct extensive experiments. The experimental results show that LCA and ReCA have comparable compression ratios with CCA, that MiCA achieves up to 38.2%, 27% and 44.5% higher compression ratios than CCA, LCA and ReCA with negligible overhead, respectively, and that LCA, and ReCA can both increase the scalability of Ganglia about 1.5 times and MiCA can increase about 2.33 times under a mixed-load circumstance.

Speculative and clone execution are existing techniques to overcome the problems of task stragglers and performance degradation in heterogeneous clusters for big data processing. In this paper, we propose an alternative approach to solving the problems based on analysis results of profiling and the relations of the system parameters. Our approach adjusts the amount of task slots of nodes dynamically to match the processing power of the nodes, according to current task progress rate and resource utilization. It contrasts with the existing techniques by attempting to prevent task stragglers from occurring in the first place through maintaining a balance between resource supply and demand. We have implemented this method in the Hadoop MapReduce platform, and the TPC-H benchmark results show that it achieves 20–30% performance improvement and 35–88% less stragglers than existing techniques.

With the fusion of cloud computing and virtualization technology, system security under virtualization becomes a key point in recent research. As a foundational technology to construct a secure system, virtual machine introspection receives more attention than ever. Almost all of the existing virtual machine monitors take the privileged virtual machine (Domain-0) as the monitoring machine, which ignore the threats brought by Domain-0 because of its huge code base of user-level tools. Besides, para-virtualized machines cannot provide the basic support for popular security applications of Windows operating system. This paper proposes a secure monitoring framework based on hardware virtualization. We use Windows operating system to build a monitoring virtual machine in hardware virtual machine domain, and set up monitoring mechanism in it. In addition, the security of the Windows monitoring machine itself is ensured all through its lifetime-bootstrap and runtime. The experiments show our secure monitoring system performs well in the secure monitoring process. The performance overhead it brings is considered to be acceptable.

In a shared virtual computing environment, dynamic load changes as well as different quality requirements of applications in their lifetime give rise to dynamic and various capacity demands, which results in lower resource utilization and application quality using the existing static resource allocation. Furthermore, the total required capacities of all the hosted applications in current enterprise data centers, for example, Google, may surpass the capacities of the platform. In this paper, we argue that the existing techniques by turning on or off servers with the help of virtual machine (VM) migration is not enough. Instead, finding an optimized dynamic resource allocation method to solve the problem of on-demand resource provision for VMs is the key to improve the efficiency of data centers. However, the existing dynamic resource allocation methods only focus on either the local optimization within a server or central global optimization, limiting the efficiency of data centers. We propose a two-tiered on-demand resource allocation mechanism consisting of the local and global resource allocation with feedback to provide on-demand capacities to the concurrent applications. We model the on-demand resource allocation using optimization theory. Based on the proposed dynamic resource allocation mechanism and model, we propose a set of on-demand resource allocation algorithms. Our algorithms preferentially ensure performance of critical applications named by the data center manager when resource competition arises according to the time-varying capacity demands and the quality of applications. Using Rainbow, a Xen-based prototype we implemented, we evaluate the VM-based shared platform as well as the two-tiered on-demand resource allocation mechanism and algorithms. The experimental results show that Rainbow without dynamic resource allocation (Rainbow-NDA) provides 26 to 324 percent improvements in the application performance, as well as 26 percent higher average CPU utilization than traditional service computing framework, in which applications use exclusive servers. The two-tiered on-demand resource allocation further improves performance by 9 to 16 percent for those critical applications, 75 percent of the maximum performance improvement, introducing up to 5 percent performance degradations to others, with 1 to 5 percent improvements in the resource utilization in comparison with Rainbow-NDA.

Virtualization can provide significant benefits in virtualized data centers by enabling efficient and effective live migration to ensure service level agreement(SLA). Most of existing studies make decision on which bad virtual machines (VMs) should be migrated to which appropriate physical machines (PMs) in terms of resource utilizations. However, migration actions may degrade migrated application performance due to extra CPU and bandwidth consumptions. Furthermore, negative performance interferences amongst applications scheduled to the same PM may arise given the poor performance isolations of VMs on a PM. We design and implement a VM migration selection system with less migration costs and application performance interferences, called LVMCI (Live Virtual machine Migration with less Costs and application Interference). We propose a migration cost evaluation model to analyze quantitatively the aspects (i.e. throughput and response latency) of application performance degradation. Dirty rate and frequent dirty rate are two key factors that affect iteration time and downtime. We implement a tool that measures these parameters before VMs are migrated. We distinguish the performance degradation of migrated applications caused by memory iteration phase and stop-and-copy phase, which helps to select VM migrated. Besides that, we propose a performance interference model which helps to select the destination PM. The experimental results show that our system can estimate memory iteration time and downtime with high accuracy, and ensures a high level of SLAs by minimizing performance degradation during migration process and performance interference among co-located VMs at the destination PM.

Virtualization has been widely adopted in data centers for improving efficiency and flexibility. Multiple applications are co-hosted in virtualized data centers. In order to meet the Service Level Agreements (SLA), how to allocate resources for multiple applications is an important and challenging task, especially when dealing with fluctuating workloads and complex server applications. Virtual Machine Monitor provides fine-grained resource allocation and live migration. In this paper, we develop RTCOIN-Qclouds, a response time-aware, cost-aware and interference-aware control framework that tunes resource allocation, which ensures a high level of meeting the SLAs. Every physical machine's resources are assigned to multiple virtual machines which run on it based on application's response time, rather than traditional methods based on resource utilization. Virtual machine migration allows data centers to rebalance workloads across physical machines. However, migration actions may lead to performance impact during the migration process. Current virtualization techniques do not provide effective performance isolation between virtual machines (VMs). Specially, hidden contention for physical resources impacts performance differently in different virtual machines. As to the problem of selecting which virtual machines to be migrated, we consider migration cost. As to the problem of selecting which physical machine to be placed, we consider performance interference. Furthermore, we experimentally validate the effectiveness of response time-aware resource allocation in our framework using microbenchmarks.

Live migration of virtual machines(VMs) is widely used for system management in virtualized servers. When the loads increase and SLAs of some applications are violated, dynamic migration of virtual machines across physical machines (PMs) has the potential to ensure a high level of meeting the SLAs. Because of consuming extra CPU and bandwidth, application performance may be degraded during the migration process. However, different applications have different performance degradation. We design and implement a VM migration selection method that decides which VMs should be migrated. It can not only eliminate resouce competition on the PM, but also have less performance degradation during the migration process. We propose a performance degration-aware model to analyze applications' performance degradation which is directly sensitive to users. We analyze migration source code and find that memory size, dirty rate and frequent dirty rate are key factors that affect iteration time and downtime. We implement a tool that measures dirty rate and frequent dirty rate before VMs are migrated. we make a distinction between memory iteration phrase and stop-and-copy phrase owing to different performance degradation. The experimental results show that our method is effective.

A cloud computing provider can dynamically allocate virtual machines (VM) based on the needs of the customers, while maintaining the privileged access to the Management Virtual Machine that directly manages the hardware and supports the guest VMs. The customers must trust the cloud providers to protect the confidentiality and integrity of their applications and data. However, as the VMs from different customers are running on the same host, an attack to the management virtual machine will easily lead to the compromise of the guest VMs. Therefore, it is critical for a cloud computing system to ensure the trustworthiness of management VMs. To this end, we propose VMGuard, an integrity monitoring and detecting system for management virtual machines in a distributed environment. VMGuard utilizes a special VM, Guard Domain, which runs on each physical node to monitor the co-resident management VMs. The integrity measurements collected by the Guard Domains are sent to the VMGuard server for safe store and independent analysis. The experimental evaluation of a Xen-based prototype shows that VMGuard can quickly detect the root kit attacks while the performance overhead is low.

The trend of using virtualization for server consolidation is more and more popular in enterprise data center. However, on-demand resource allocation among the concurrent hosted services in such a virtualized environment is still a challenge. In order to optimize resource allocation among services in data center, this paper proposes a multi-tiered resource scheduling scheme which automatically provides on-demand capacities to the hosted services via resources flowing among VMs. We model the resource flowing using optimization theory. Based on this model, we present a global re-source flowing algorithm in the multi-tiered resource scheduling scheme. This algorithm preferentially ensures performance of some critical services by degrading of others to some extent when resource competition arises. Using our RAINBOW prototype, we evaluate the multi-tiered resource scheduling scheme with the performance improvements for the most critical services up to 9%~16%, which are 75% of the maximum improvement margin, while performance degradation of others is up to 2%, and leads to 1%~5% improvements in resource utilization than RAINBOW without resource flowing. Compared with the existent scheme, our work leads to 9% less improvements for critical services, while introduces 39% less degradation to low priority services.

Server consolidation based on virtualization technology will simplify system administration, reduce the cost of power and physical infrastructure, and improve utilization in today's Internet-service-oriented enterprise data centers. How much power and how many servers for the underlying physical infrastructure are saved via server consolidation in VM-based data centers is of great interest to administrators and designers of those data centers. Various workload consolidations differ in saving power and physical servers for the infrastructure. The impacts caused by virtualization to those concurrent services are fluctuating considerably which may have a great effect on server consolidation. This paper proposes a utility analytic model for Internet-oriented server consolidation in VM-based data centers, modelling the interaction between server arrival requests with several QoS requirements, and capability flowing amongst concurrent services, based on the queuing theory. According to features of those services' workloads, this model can provide the upper bound of consolidated physical servers needed to guarantee QoS with the same loss probability of requests as in dedicated servers. At the same time, it can also evaluate the server consolidation in terms of power and utility of physical servers. Finally, we verify the model via a case study comprised of one e-book database service and one e-commerce Web service, simulated respectively by TPC-W and SPECweb2005 benchmarks. Our experiments show that the model is simple but accurate enough. The VM-based server consolidation saves up to 50% physical infrastructure, up to 53% power, and improves 1.7 times in CPU resource utilization, without any degradation of concurrent services' performance, running on Rainbow - our virtual computing platform.

Virtual machine technology has played an important role in data center. Distributed services deployed in multiple virtual machines, may reside on one physical machine. This situation requires an efficient inter-domain communication channel with transparency and security principles ensured. Although current inter-domain mechanism has gained a much better performance compared to traditional inter-domain path offered by hypervisor, shared data channel size limitation and additional copy are still two restrains against a higher performance efficiency. In this paper, we give an analysis on these limitations, overcome these shortcomings, and achieve a higher efficient inter-domain communication channel. By applying virtual address protection mechanism during channel bootstrap, we enlarge the maximum size of shared data channel. By pointing network packet structure to the buffer in the shared data channel, we avoid an extra data copy on the receiver VM side. In our evaluation using a number of standard benchmarks, we have reduced the latency by nearly 40%, increased the throughput by approximately 45% and cut down more than 3500 CPU cycles per packet.

Grid technologies promise to bring the grid users high performance. Consequently, scheduling is being becoming a crucial problem. Herd behavior is a common phenomenon, which causes the severe performance decrease in grid environment with respect to bad scheduling behaviors. In this paper, on the basis of the theoretical results of the homogeneous balls and bins model, we proposed a novel stochastic algorithm to avoid herd behavior. Our experiments address that the multi-choice strategy, combined with the advantages of DHT, can decrease herd behavior in large-scale sharing environment, at the same time, providing better schedule performance while burdening much less scheduling overhead than greedy algorithms. In the case of 1000 resources, the simulations show that, for the heavy load(i.e. system utilization rate 0.5), the multi-choice algorithm reduces the number of incurred herds by a factor of 36, the average job waiting time by a factor of 8, and the average job turn-around time by 12% compared to the greedy algorithms

In this paper, we propose four general queueing models based on input and server distributions, to analyze a special grid system, VEGA grid system version 1.1 (VEGA1.1). The mean queue lengths and mean waiting times of these models are deduced. The two classic applications, the computing-oriented application (blast computing) and online transaction processing application (air booking service) are employed to analyze and evaluate the grid system and the models. Meanwhile, we validate VEGA's four prominent characteristics, versatile services, enabling intelligence, global uniformity and autonomous control. Finally, we analyze the experiment results, predict the behavior of VEGA1.1 in equilibrium and point out the bottlenecks of the system

Resource discovery is crucial to efficient deployment of a grid system whose dynamic, heterogeneous characteristics make it difficult. In this paper, Vega Infrastructure for Resource Discovery (VIRD) is developed, then augmented with new features (i.e., some new algorithms) to build a C/S (client/server) and P2P (peer-to-peer) hybrid resource discovery framework. The three layered architecture of the VIRD is developed to make advantage of the physical and logical topologies of the Internet to facilitate resource discovery. With our simulations and theoretical analysis, it is proved that VIRD is of good scalability with respect to the sizes of the underlying backbone. Even when the resource density is low and the max TTL (time-to-live) is small, VIRD still achieves high search success rates in a small amount of hops. Compared with flooding and random walk algorithms via the same search success rates, VIRD outperforms them in both network traffic and response time.

Summary form only given. Here we present two efficient coherence protocols for data grid applications. The data intensive grid applications often adopt the replica strategy to solve the data access bottleneck. It is important to keep coherence among all replicas of a copy of large amount of data. The first strategy for coherence is called lazy copy coherence protocol and the second aggressive copy coherence. Meanwhile, we consider in which situation we should make a replica for a copy of large amount of data.

In this paper, we propose an efficient barrier synchronization scheme on networks with arbitrary topologies. We first present a distributed method in building a barrier routing tree. The barrier messages can be delivered adaptively according to the hierarchy of the established barrier tree to void congestion and faulty nodes in the network. We then propose a new technique, called bandwidth-preempting technique, for a blocked barrier message to preempt a channel occupied by a data message so that the latency of a barrier message can be controlled without affecting much of the overall system performance. We also propose an analytical performance model and present simulation results for the performance evaluation of the proposed scheme. Performance evaluations show that the proposed scheme outperforms the existing algorithms for barrier synchronization.

In this paper, we introduce a family of scalable interconnection network topologies, named Recursive Cube of Rings (RCR), which are recursively constructed by adding ring edges to a cube. RCRs possess many desirable topological properties in building scalable parallel machines, such as fixed degree, small diameter, wide bisection width, symmetry, fault tolerance, etc. We first examine the topological properties of RCRs. We then present and analyze a general deadlock-free routing algorithm for RCRs. Using a complete binary tree embedded into an RCR with expansion-cost approximating to one, an efficient broadcast routing algorithm on RCRs is proposed. The upper bound of the number of message passing steps in one broadcast operation on a general RCR is also derived.

With respect to scalability and arbitrary topologies of the underlying networks in multiprogramming and multithread environments, fault tolerance in acknowledged ATAB and concurrent communications become a challenge to reliable general wormhole routing multicomputers with arbitrary topologies. In this paper, the virtual ring tree (VRT) is proposed to deal with the challenge. A single startup is needed in the two proposed algorithms by a simple virtual node space, which also reduces the complexity of routing at intermediate steps of ATAB algorithms and re-beginning an ATAB, by cacheable virtual channels. The proposed algorithm can automatically handle static faults in networks.