In cloud environments, GPU-based deep neural network (DNN) inference servers are required to meet the Service Level Objective (SLO) latency for each workload under a specified request rate, while also minimizing GPU resource consumption. However, previous studies have not fully achieved this objective. In this paper, we propose ParvaGPU, a technology that facilitates spatial GPU sharing for large-scale DNN inference in cloud computing. ParvaGPU integrates NVIDIA’s Multi-Instance GPU (MIG) and Multi-Process Service (MPS) technologies to enhance GPU utilization, with the goal of meeting the diverse SLOs of each workload and reducing overall GPU usage. Specifically, ParvaGPU addresses the challenges of minimizing underutilization within allocated GPU space partitions and external fragmentation in combined MIG and MPS environments. We conducted our assessment on multiple A100 GPUs, evaluating 11 diverse DNN workloads with varying SLOs. Our evaluation revealed no SLO violations and a significant reduction in GPU usage compared to state-of-the-art frameworks.

Industrial Internet of Things (IIoT) applications can greatly benefit from leveraging edge computing. For instance, applications relying on deep neural network (DNN) models can be sliced and distributed across IIoT devices and the network edge to reduce inference latency. However, low network performance between IIoT devices and the edge often becomes a bottleneck. In this study, we propose ScissionLite, a holistic framework designed to accelerate distributed DNN inference using lightweight data compression. Our compression method features a novel lightweight down/upsampling network tailored for performance-limited IIoT devices, which is inserted at the slicing point of a DNN model to reduce outbound network traffic without causing a significant drop in accuracy. In addition, we have developed a benchmarking tool to accurately identify the optimal slicing point of the DNN for the best inference latency. ScissionLite improves inference latency by up to 15.7× with minimal accuracy degradation.

This paper presents a virtualization-based computing and storage resource provisioning manager, termed RPM, for edge devices. Given a request from an edge device, RPM can perform dynamic volume provisioning and accelerated computation on augmented computing resources. Also, RPM can migrate application execution conducted by edge devices online. We show that RPM is a promising approach through our experiments.

The design of QUEST (software QUality Evaluation System and Technology) tools, and especially of its repository, is discussed. QUEST tools are composed of a software process assessment tool (QUEST-A), a software process improvement tool (QUEST-I) and a repository for software process information (QUEST-R). We place the emphasis of this paper on the design of QUEST-R, which acts as a process asset library (PAL) and a meta-process support tool. The meta-process knowledge can be accumulated from several cases of process improvement, which can be initiated by software process assessment and process metrics. The results or outputs of QUEST-A, QUEST-I or the other tools act as a driver of knowledge inference. To build QUEST-R, a knowledge-based software process model (KB-SPM), which incorporates meta-process objects and their relationships, is proposed. Several factors, including environmental changes and various process improvement strategies, are explicitly modeled as frames. QUEST-R (1) makes it possible to generate and customize project process models in various application domains; (2) makes it possible to reuse the software process objects; and (3) makes it easy to accumulate and enhance meta-process knowledge. Besides supporting the meta-process phase, to increase the effectiveness of the meta-process, the following facilities are proposed: (1) rules for control knowledge; (2) more affecting factors for the generation of software processes; (3) a dependency structure, for describing improvement items and their interdependence; and (4) a mechanism to automatically update the knowledge.