Adaptive batching is a promising technique to reduce the communication and synchronization overhead for training Deep Neural Network (DNN) models. In this paper, we study how to speed up the training of a DNN model using adaptive batching, without degrading the convergence performance in a heterogeneous GPU cluster. We propose a novel DNN training system, called JABAS (Joint Adaptive Batching and Automatic Scaling). In JABAS, a DNN training job is executed on a DNN training framework called IIDP, which provides the same theoretical convergence rate of distributed SGD in a heterogeneous GPU cluster. To maximize the performance of the job with adaptive batching, JABAS employs adaptive batching and automatic resource scaling jointly. JABAS changes a global batch size every p iterations in a fine-grained manner within an epoch, while auto-scaling to the best GPU allocation for the next epoch in a coarse-grained manner. Using three heterogeneous GPU clusters, we evaluate JABAS for seven DNN models including large language models. Our experimental results demonstrate that JABAS provides 33.3% shorter training time and 54.2% lower training cost than the state-of-the-art adaptive training techniques, on average, without any accuracy loss.

Transformer-based Large Language Models (LLMs) demand significant computational and memory resources due to the autoregressive token generation in decoder blocks. In particular, the attention layer in LLM models has low arithmetic intensity but high memory traffic, thus requiring frequent updates to the KV matrices with each decoder iteration. As a result, LLM inference becomes memory bound, leading to increased latency. To address this, we introduce PAISE, a framework leveraging Processing-In-Memory (PIM) technology to offload memory-intensive tasks. PAISE employs GPU-PIM heterogeneous computing resources to optimize inference operations in transformer-based LLMs. The framework comprises (i) a scheduling algorithm that decides which operations to offload to PIM based on model configuration and PIM hardware specifications and (ii) an enhanced PIM kernel that performs transaction-wise interleave-batched GEMM (General Matrix Multiplication) operations, maximizing data throughput via data layout adjustments. We implemented PAISE on the GPT-2 and Llama2-7B models using an AMD MI100 GPU with HBM-PIM devices. Our evaluations show that offloading the attention layer to PIM reduces execution time by up to 48.3% compared to GPU-only inference, demonstrating PAISE’s significant potential to enhance the efficiency of LLM inference, which could lead to faster and more efficient AI applications.

This paper presents the economic impacts of spot instance service on the cloud service providers (CSPs) and the customers when the CSPs offer it along with the on-demand instance service to the customers. We model the interaction between CSPs and customers as a non-cooperative two-stage dynamic game. Our equilibrium analysis reveals (i) the techno-economic interrelationship between the customers' heterogeneity, resource availability, and CSPs' pricing policy, and (ii) the impacts of the customers' service selection (spot vs. on-demand) and the CSPs' pricing decision on the CSPs' market share and revenue, as well as the customers' utility. The key technical challenges lie in, first, how we capture the strategic interactions between CSPs and customers, and second, how we consider the various practical aspects of cloud services, such as heterogeneity of customers' willingness to pay for the quality of service (QoS) and the fluctuating resource availability. The main contribution of this paper is to provide CSPs and customers with a better understanding of the economic impact caused by a certain price policy for the spot service when the equilibrium price, which from our two-stage dynamic game analysis, is able to set as the baseline price for their spot service.

Remote Direct Memory Access (RDMA) characteristics, such as high bandwidth, low latency, and low CPU utilization, have positioned RDMA as mainstream for interconnect of cloud-based High-Performance Computing (HPC) services. However, existing RDMA technologies, including InfiniBand and RoCEv2, have limitations in terms of compatibility with legacy networks, scalability in large-scale deployment, and cost-inefficient. In order to address these challenges, we propose Cloud-optimized RDMA Networking (CORN). It features cloud-optimized congestion control, which considers the Bandwidth Delay Product (BDP) and the inflight packets to determine the amount of traffic to be transmitted. This congestion control scheme significantly reduces the likelihood of packet loss due to overflowing buffers on the network switches. CORN leverages the traditional Selective ACK (SACK) to deal with packet drops caused by network congestion or H/W fault. Consequently, CORN can support lossy RDMA networks on Ethernet. In addition, the two features of CORN are designed to operate without any modifications or configurations of the network switches. CORN functions as a shim layer between UDP and RDMA, operating solely within the end host. This design ensures the seamless deployment of CORN. The implementation using ns3 shows that CORN is feasible and more efficient than congestion control schemes like DCQCN, TIMELY, and HPCC.

End user experiences on mobile devices with their rich sets of sensors are constrained by limited device battery lives and restricted form factors, as well as by the `scope' of the data available locally. The `Personal Cloud' distributed software abstractions address these issues by enhancing the capabilities of a mobile device via seamless use of both nearby and remote cloud resources. In contrast to vendor-specific, middleware-based cloud solutions, Personal Cloud instances are created at hypervisor-level, to create for each end user the federation of networked resources best suited for the current environment and use. Specifically, the Cirrostratus extensions of the Xen hypervisor can federate a user's networked resources to establish a personal execution environment, governed by policies that go beyond evaluating network connectivity to also consider device ownership and access rights, the latter managed in a secure fashion via standard Social Network Services. Experimental evaluations with both Linux- and Android-based devices, and using Facebook as the SNS, show the approach capable of substantially augmenting a device's innate capabilities, improving application performance and the effective functionality seen by end users.

There is an increasing number of network-enabled computing devices in homes and offices, driven by continued improvements in device capabilities and network connectivity. By exploiting the virtualization technologies that have begun to pervade even the mobile domain, these devices -- hardware components, such as displays, input devices, disks, or processors, can be decoupled from the physical platforms on which they reside to form a resource pool or device cloud. By drawing on the composite resources of device clouds, applications can leverage the heterogeneity present in the cloud to exploit hardware/device differences in terms of power consumption, computational speeds, display sizes, or the presence of certain accelerators, and can take advantage of software diversity in terms of the different operating environments and applications that efficiently operate on individual devices. This paper implements and evaluates the concept of device clouds, in which virtual execution platforms dynamically composed from sets of devices are built for applications, using automated methods that are based on simple policies. Experimental results identify the basic overheads associated with device clouds and their use, and demonstrate the advantages of dynamically constructed virtual platforms rather than individual machines, both in terms of improvements in system properties like power usage and improvements in user experiences for media delivery and play out.

Today, people usually find it difficult to work seamlessly on PCs in different places. It is mainly because the PCs have different computing environments in terms of versions of applications and desktop settings like fonts or languages. The Software Mobility technology enables people to use their own personalized computing environment on any PC ensuring seamless work We designed our Software Mobility solution with an visualization layer. Because this layer decouples applications from the operating system, applications can be moved and run on any PC without disruption. We will demonstrate our Software Mobility implementation that shows people can use Windows XP applications and user preference desktop settings on any PC without any installation or configuration.