Extracting information from large-scale high-dimensional data is a fundamentally important task in high performance computing, where the hierarchical Tucker (HT) tensor learning approach (learning a tensor-tree structure) has been widely used in many applications. However, HT tensor learning algorithms are compute-intensive due to the “curse of dimensionality,” i.e., the time complexity grows exponentially with the order of the data tensor. The computation of HT tensor learning algorithms boils down to tensor primitives, which are amenable to computing on GPU tensor cores. Existing work does not support HT tensor learning using GPU tensor cores. There are three main challenges to address: 1) to accelerate tensor learning primitives using GPU tensor cores; 2) to implement the tensor learning algorithms using GPU tensor cores and multiple GPUs; 3) to support large-scale data tensors exceeding the GPU memory capacity. In this paper, we present efficient HT tensor learning primitives using GPU tensor cores and demonstrate three applications. First, we utilize GPU tensor cores to optimize HT tensor learning primitives, including tensor contractions, tensor matricizations and tensor singular value decomposition (SVD). We employ the optimized primitives to optimize HT tensor decomposition algorithms for Big Data analysis. Second, we propose a novel HT tensor layer for deep neural networks, whose training process only involves a forward pass without back propagation. The forward pass consists of tensor operations, thus further exploiting the computing power of GPU tensor cores. Third, we apply the optimized primitives to develop a tensor-tree structured quantum machine learning algorithmtree-tensor network (TTN). Compared with TensorLy and TensorNetwork on NVIDIA A100 GPUs, our third-order HT tensor decomposition algorithm achieves up to$8.92 \times$and$6.42 \times$speedups, respectively, and our high-order case achieves up to$32.67 \times$and$23.97 \times$speedups, respectively. Our HT tensor layer for a fully connected neural network achieves$49.2 \times$compression at the cost of 0.5% drops in accuracy and$1.42 \times$speedup compared with the implementation on CUDA cores; for the AlexNet, our HT tensor layer achieves$9.45 \times$compression at the cost of 0.8% drops in accuracy and$1.87 \times$speedup compared with the implementation on CUDA cores. Our TTN algorithm achieves up to$11.17\times$speedup compared with TensorNetwork, indicating the potential of optimized tensor learning primitives for the classical simulation of quantum machine learning algorithms.

Researchers have proposed many novel methods to detect abnormal taxi trajectories. However, most of the existing methods usually adopt a counting-based strategy, which may cause high false positives due to imprecisely identifying diverse trajectories as anomalies and therefore, they need the support of large-scale historical trajectories to work properly. To improve detection precision and efficiency, in this article, we propose STR, an online abnormal taxi trajectory detection method based on spatio-temporal relations. The basic principle behind STR is that given the displacement from the source point to a testing point, if the driving time and driving distance are not within the normal ranges, the point is identified as anomalous. To learn the two normal ranges for driving time and driving distance, STR defines two spatio-temporal models which characterize the relationship between displacement and driving distance/driving time. To improve detection efficiency, STR reduces the number of models that need to be learned by making full use of the similarity of transportation modes in different time periods and neighboring areas. The effectiveness and performance of STR are evaluated on real-world taxi trajectories. The experiment results show that compared with counting-based methods, STR achieves greater precision by reducing false positives. Furthermore, STR is more efficient than its counterparts and is suitable for online detection.

Data completion is a problem of filling missing or unobserved elements of partially observed datasets. Data completion algorithms have received wide attention and achievements in diverse domains including data mining, signal processing, and computer vision. We observe a ubiquitous tubal-sampling pattern in big data and Internet of Things (IoT) applications, which is introduced by many reasons such as high data acquisition cost, downsampling for data compression, sensor node failures, and packet losses in low-power wireless transmissions. To meet the time and accuracy requirements of applications, data completion methods are expected to be accurate as well as fast. However, the existing methods for data completion with the tubal-sampling pattern are either accurate or fast, but not both. In this article, we propose high-performance graphics processing unit (GPU) tensor completion for data completion with the tubal-sampling pattern. First, by exploiting the convolution theorem, we split a tensor least-squares minimization problem into multiple least-squares sub-problems in the frequency domain. In this way, massive parallelisms are exposed for many-core GPU architectures while still preserving high recovery accuracy. Second, we propose computing slice-level and tube-level tasks in batches to improve GPU utilization. Third, we reduce the data transfer cost by eliminating the accesses to the CPU memory inside algorithm loop structures. The experimental results show that the proposed tensor completion is both fast and accurate. Using synthetic data of varying sizes, the proposed GPU tensor completion achieves maximum 248.18×, 7, 403.27×, and 33.27× speedups over the CPU MATLAB implementation, GPU element-sampling tensor completion in the cuTensor-tubal library, and GPU high-performance matrix completion, respectively. With a 50 percent sampling rate, the proposed GPU tensor completion achieves a recovery error of 1.40e-5, which is comparable with that of the GPU element-sampling tensor completion and three orders of magnitude better than that of the GPU high-performance matrix completion. To utilize multiple GPUs in servers, we design a multi-GPU scheme for tubal-sampling tensor completion. The multi-GPU tensor completion achieves maximum 1.89× speedup on two GPUs versus on a single GPU for medium or big tensors. We further evaluate the performance of the proposed GPU tensor completion in three real applications, namely, video transmission in wireless camera networks, RF fingerprint-based indoor localization, and seismic data completion, and it achieves maximum speedups of 448.68×, 24.63×, and 311.54×, respectively. We integrate this high-performance GPU tensor completion implementation into the cuTensor-tubal library to support various applications.

Tensors are the cornerstone data structures in high-performance computing, big data analysis and machine learning. However, tensor computations are compute-intensive and the running time increases rapidly with the tensor size. Therefore, designing high-performance primitives on parallel architectures such as GPUs is critical for the efficiency of ever growing data processing demands. Existing GPU basic linear algebra subroutines (BLAS) libraries (e.g., NVIDIA cuBLAS) do not provide tensor primitives. Researchers have to implement and optimize their own tensor algorithms in a case-by-case manner, which is inefficient and error-prone. In this paper, we develop the cuTensor-tubal library of seven key primitives for the tubal-rank tensor model on GPUs: t-FFT, inverse t-FFT, t-product, t-SVD, t-QR, t-inverse, and t-normalization. cuTensor-tubal adopts a frequency domain computation scheme to expose the separability in the frequency domain, then maps the tube-wise and slice-wise parallelisms onto the single instruction multiple thread (SIMT) GPU architecture. To achieve good performance, we optimize the data transfer, memory accesses, and design the batched and streamed parallelization schemes for tensor operations with data-independent and data-dependent computation patterns, respectively. In the evaluations oft-product, t-SVD, t-QR, t-inverse and t-normalization, cuTensor-tubal achieves maximum 16.91x, 27.03x, 38.97x, 22.36x,15.43x speedups respectively over the CPU implementations running on dual 10-core Xeon CPUs. Two applications, namely, t-SVD-based video compression and low-tubal-rank tensor completion, are tested using our library and achieve maximum 9.80x and 269.26x speedups over multi-core CPU implementations.

In this paper, we optimize the computations of third-order low-tubal-rank tensor operations on many-core GPUs. Tensor operations are compute-intensive and existing studies optimize such operations in a case-by-case manner, which can be inefficient and error-prone. We develop and optimize a BLAS-like library for the low-tubal-rank tensor model called cuTensor-tubal, which includes efficient GPU primitives for tensor operations and key processes. We compute tensor operations in the frequency domain and fully exploit tube-wise and slice-wise parallelisms. We design, implement, and optimize four key tensor operations namely t-FFT, inverse t-FFT, t-product, and t-SVD. For t-product and t-SVD, cuTensor-tubal demonstrates significant speedups: maximum 29.16 ×, 6.72× speedups over the non-optimized GPU counterparts, and maximum 16.91× and 27.03× speedups over the CPU implementations running on dual 10-core Xeon CPUs.

The GPU pass-through technology, under the virtual machine(VM), is always used in CUDA programming. The data will transfer between the VM main memories and GPU memory during the processing of CUDA programs. It is known that the transmission speed of the pinned memory is faster than the pageable when a large amount of data appears. However, the cost of pinned memory will occupy a lot of physical memory and lead to a lack of memory for the VM. So it can also affect the memory usage of other applications in the VM. In this paper, we propose a method to improve the performance of CUDA in virtual environment. In our experiment, the virtual machine manager is chosen as Kernel-based Virtual Machine(KVM), and the GPU pass-through is used for distributing the GPU for the VMs. We also defined a new module as CUDA Memory Management(CMM) which is added into the virtual environment. The result shows that the approach has high efficiency in CUDA program. And the efficiency of data transmission for using the CMM module is more than 16% than the pageable memory only using mode.

In a typical GPGPU, the on-chip storage is critical to the massive parallelism and is desired to be large. However, the fast increasing size of the on-chip storage based on traditional SRAM cells, such as register file (RF), shared memory and first level data (L1D) cache, makes the area cost and energy consumption unsustainable for future GPGPUs. In this paper, we first propose to use the embedded-DRAM (eDRAM) as an alternative for the on-chip storage. Compared to the conventional SRAM, eDRAM enables higher density and lower leakage power, but suffers from limited data retention time. Periodic refresh operation is a viable approach to maintain data integrity but aggravates the performance and energy consumption with the scaling of eDRAM cells into deep sub-micron technology nodes. To recover the performance loss, we exploit the features in the GPGPU architecture and propose various novel refresh schemes to mitigate the refresh penalty. To improve the energy efficiency, we apply lightweight compiler techniques and runtime monitoring for selective refreshing that intelligently eliminate the unnecessary refreshes. The evaluation on our proposed refresh schemes demonstrates that, comparing to the conventional SRAM-based designs, our eDRAM-based on-chip storage exhibits comparable performance but less energy consumption and smaller silicon area, enabling the sustainable on-chip storage scaling for even higher parallelism in future GPGPUs.

Recently, tapping into renewable energy sources has shown great promise in alleviating server energy poverty and reducing IT carbon footprint. Due to the limited, time-varying green power generation, matching server power demand to runtime power budget is often crucial in green data centers. However, existing studies mainly focus on the temporal variability of the power supply and demand, while largely ignore the spatial variation issue in server hardware. With more complex computing units integrated and the technology scaling, the performance/power variation among nodes and the conservative supply voltage margin of each core can greatly compromise the power matching effectiveness that a green datacenter can achieve. This paper explores green datacenter design that takes into account non-uniform hardware power characteristics. We propose is cope, a novel power management framework that can (1) expose architecture variability to the datacenter facility-level scheduler for efficient power matching, and (2) balance the energy usage and lifetime of compute nodes in the highly dynamic green computing environment. Using realistic hardware profiling data and renewable energy data, we show that is cope can reduce the energy cost up to 54%, while maintaining fairly balanced processor utilization rate and negligible profiling overhead.

A modern general-purpose graphics processing unit (GPGPU) usually consists of multiple streaming multiprocessors (SMs), each having a pipeline that incorporates a group of threads executing a common instruction flow. Although SMs are designed to work independently, we observe that they tend to exhibit very similar behavior for many workloads. If multiple SMs can be grouped and work in the lock-step manner, it is possible to save energy by sharing the front-end units among multiple SMs, including the instruction fetch, decode, and schedule components. However, such sharing brings architectural challenges and sometime causes performance degradation. In this article, we show our design, implementation, and evaluation for such an architecture, which we call Buddy SM . Specifically, multiple SMs can be opportunistically grouped into a buddy cluster. One SM becomes the master, and the rest become the slaves. The front-end unit of the master works actively for itself as well as for the slaves, whereas the front-end logics of the slaves are power gated. For efficient flow control and program correctness, the proposed architecture can identify unfavorable conditions and ungroup the buddy cluster when necessary. We analyze various techniques to improve the performance and energy efficiency of Buddy SM. Detailed experiments manifest that 37.2% front-end and 7.5% total GPU energy reduction can be achieved.

A modern GPU processor consumes several times power of a multi-core CPU and delivers a much higher processing throughput. Researchers propose various architectural innovations to improve its energy efficiency. We observe that different streaming processors (SMs) in a GPU tend to exhibit very similar behavior for many GPU workloads. If multiple SMs can be grouped together and work in synchronous manner, it is possible to save energy by sharing the front-end in the SM pipeline including the instruction fetch, decode and schedule units. For efficient flow control and program correctness, the proposed architecture can identify unfavorable conditions and ungroup the SMs when necessary. However, sharing pipeline front-end between multiple SMs brings architectural challenges. In this paper, we show our design, implementation and evaluation for such an architecture. Detailed experiment results manifest 33.7% front-end and 6.8% total GPU energy reduction can be achieved.