The Sparse Matrix-Vector Multiplication (SpMV) kernel is used in a broad class of linear algebra computations. SpMV computations result in a performance bottleneck in many high performance applications, so optimizing SpMV performance is paramount. While implementing this kernel on a GPU can potentially boost performance significantly, current GPU libraries either provide modest performance gains or are burdened with high sparse format conversion overhead. In this paper we introduce the Vertical Compressed Sparse Row (VCSR) format, a novel memory-aware format that out-performs previous proposed formats on a GPU. We first motivate the design of our baseline VCSR format and then step through a series of enhancements that further improve VCSR's memory efficiency (VCSR-MEM) and performance (VCSR-INTRLV), while also considering conversion overhead. VCSR attempts to produce a high degree of thread-level parallelism and memory utilization by exploiting knowledge of GPU memory microarchitecture. VCSR can reduce the number of global memory transactions significantly, an issue not addressed by most other sparse formats. In addition, VCSR provides a novel reordering mechanism. It minimizes the size of the compressed matrix, handles both regular/irregular sparse matrices, and can be customized based on matrix size. VCSR also minimizes conversion overhead, as compared to full or partial row reordering. Our methodology is highly configurable and can be optimized for any sparse matrix. We have evaluated the VCSR format for the SpMV kernel when run on two different NVIDIA GPUs, the Kepler K40 and the Volta V100. We compare VCSR with NVIDIA's cuSPARSE library (the HYB format), a state-of-the-art sparse library. We also compare against other state-of-the-art CSR-based formats, including CSR5, merge-base SpMV and HOLA. We evaluate the benefits of VCSR over the entire University of Florida's SuiteSparse dataset collection. The VCSR-baseline format achieves an average speedup ranging from$1.10\times$to$1.39\times$when compared to the performance of the four state-of-the-art formats on an NVIDIA V100. While the VCSR-MEM format can save a significant amount of memory space, it is a bit slower than our VCSR-baseline. VCSR-INTRLV performs much better than the VCSR-baseline, and even when including the conversion overhead, achieves an average speedup of$1.08\times$as compared to HOLA (the best performing format among the prior schemes).

Graph Neural Networks (GNNs) have emerged as a promising class of Machine Learning algorithms to train on non-euclidean data. GNNs are widely used in recommender systems, drug discovery, text understanding, and traffic forecasting. Due to the energy efficiency and high-performance capabilities of GPUs, GPUs are a natural choice for accelerating the training of GNNs. Thus, we want to better understand the architectural and system-level implications of training GNNs on GPUs. Presently, there is no benchmark suite available designed to study GNN training workloads. In this work, we address this need by presenting GNNMark, a feature-rich benchmark suite that covers the diversity present in GNN training workloads, datasets, and GNN frameworks. Our benchmark suite consists of GNN workloads that utilize a variety of different graph-based data structures, including homogeneous graphs, dynamic graphs, and heterogeneous graphs commonly used in a number of application domains that we mentioned above. We use this benchmark suite to explore and characterize GNN training behavior on GPUs. We study a variety of aspects of GNN execution, including both compute and memory behavior, highlighting major bottlenecks observed during GNN training. At the system level, we study various aspects, including the scalability of training GNNs across a multi-GPU system, as well as the sparsity of data, encountered during training. The insights derived from our work can be leveraged by both hardware and software developers to improve both the hardware and software performance of GNN training on GPUs.

Deep Neural Networks (DNNs) have emerged as an important class of machine learning algorithms, providing accurate solutions to a broad range of applications. Sparsity in activation maps in DNN training presents an opportunity to reduce computations. However, exploiting activation sparsity presents two major challenges: i) profiling activation sparsity during training comes with significant overhead due to computing the degree of sparsity and the data movement; ii) the dynamic nature of activation maps requires dynamic dense-to-sparse conversion during training, leading to significant overhead. In this article, we present Spartan, a lightweight hardware/software framework to accelerate DNN training on a GPU. Spartan provides a cost-effective and programmer-transparent microarchitectural solution to exploit activation sparsity detected during training. Spartan provides an efficient sparsity monitor, a tile-based sparse GEMM algorithm, and a novel compaction engine designed for GPU workloads. Spartan can reduce sparsity profiling overhead by 52.5× on average. For the most compute-intensive layers, i.e., convolutional layers, we can speedup AlexNet by 3.4×, VGGNet-16 by 2.14×, and ResNet-18 by 2.02×, when training on the ImageNet dataset.

In this paper, we propose Ensemble Learning models to identify factors contributing to preterm birth. Our work leverages a rich dataset collected by a NIEHS P42 Center that is trying to identify the dominant factors responsible for the high rate of premature births in northern Puerto Rico. We investigate analytical models addressing two major challenges present in the dataset: 1) the significant amount of incomplete data in the dataset, and 2) class imbalance in the dataset. First, we leverage and compare two types of missing data imputation methods: 1) mean-based and 2) similarity-based, increasing the completeness of this dataset. Second, we propose a feature selection and evaluation model based on using undersampling with Ensemble Learning to address class imbalance present in the dataset. We leverage and compare multiple Ensemble Feature selection methods, including Complete Linear Aggregation (CLA), Weighted Mean Aggregation (WMA), Feature Occurrence Frequency (OFA) and Classification Accuracy Based Aggregation (CAA). To further address missing data present in each feature, we propose two novel methods: 1) Missing Data Rate and Accuracy Based Aggregation (MAA), and 2) Entropy and Accuracy Based Aggregation (EAA). Both proposed models balance the degree of data variance introduced by the missing data handling during the feature selection process, while maintaining model performance. Our results show a 42% improvement in sensitivity versus fallout over previous state-of-the-art methods.

Recently there has been a rapidly growing demand for faster machine learning (ML) processing in data centers and migration of ML inference applications to edge devices. These developments have prompted both industry and academia to explore custom accelerators to optimize ML executions for performance and power. However, identifying which accelerator is best equipped for performing a particular ML task is challenging, especially given the growing range of ML tasks, the number of target environments, and the limited number of integrated modeling tools. To tackle this issue, it is of paramount importance to provide the computer architecture research community with a common framework capable of performing a comprehensive, uniform, and fair comparison across different accelerator designs targeting a particular ML task. To this aim, we propose a new framework named TFLITE-SOC (System On Chip) that integrates a lightweight system modeling library (SystemC) for fast design space exploration of custom ML accelerators into the build/execution environment of Tensorflow Lite (TFLite), a highly popular ML framework for ML inference. Using this approach, we are able to model and evaluate new accelerators developed in SystemC by leveraging the language's hierarchical design capabilities, resulting in faster design prototyping. Furthermore, any accelerator designed using TFLITE-SOC can be benchmarked for inference with any DNN model compatible with TFLite, which enables end-to-end DNN processing and detailed (i.e., per DNN layer) performance analysis. In addition to providing rapid prototyping, integrated benchmarking, and a range of platform configurations, TFLITE-SOC offers comprehensive performance analysis of accelerator occupancy and execution time breakdown as well as a rich set of modules that can be used by new accelerators to implement scaling up studies and optimized memory transfer protocols. We present our framework and demonstrate its utility by considering the design space of a TPU-like systolic array and describing possible directions for optimization. Using a compression technique, we implement an optimization targeting reducing the memory traffic between DRAM and on-device buffers. Compared to the baseline accelerator, our optimized design shows up to 1.26× speedup on accelerated operations and up to 1.19× speedup on end-to-end DNN execution.

The rapidly growing popularity and scale of data-parallel workloads demand a corresponding increase in raw computational power of Graphics Processing Units (GPUs). As single-GPU platforms struggle to satisfy these performance demands, multi-GPU platforms have started to dominate the high-performance computing world. The advent of such systems raises a number of design challenges, including the GPU microarchitecture, multi-GPU interconnect fabric, runtime libraries, and associated programming models. The research community currently lacks a publicly available and comprehensive multi-GPU simulation framework to evaluate next-generation multi-GPU system designs.

Machine learning has seen tremendous growth in recent years thanks to two key advances in technology: massive data generation and highly-parallel accelerator architectures. The rate that data is being generated is exploding across multiple domains, including medical research, environmental science, web-search, and e-commerce. Many of these advances have benefited from emergent web-based applications, and improvements in data storage and sensing technologies. Innovations in parallel accelerator hardware, such as GPUs, has made it possible to process massive amounts of data in a timely fashion. Given these advanced data acquisition technology and hardware, machine learning researchers are equipped to generate and sift through much larger and complex datasets quickly. In this work, we focus on accelerating Kernel Dimension Alternative Clustering algorithms using GPUs. We conduct a thorough performance analysis by using both synthetic and real-world datasets, while also modifying both the structure of the data, and the size of the datasets. Our GPU implementation reduces execution time from minutes to seconds, which enables us to develop a web-based application for users to, interactively, view alternative clustering solutions.

In recent years, the availability of data-driven analytics has become a key tool in discovery in public health and environmental science research. As a result, these communities have looked to leverage recent advances in machine learning algorithms. This class of algorithms are able to find hidden patterns and develop new knowledge in complex data, accelerating the rate of discovery in multiple research domains. In this paper, we present our methodology of applying machine learning algorithms to health outcomes, chemical exposures, and social behavior data from expectant mothers, as part of the NIEHS-supported PROTECT Center. The ultimate goal is to determine the dominant factors/features potentially responsible for the high rate of premature births in Puerto Rico.Many commonly-used machine learning algorithms can be used for feature selection. However, given the imbalance in our birth outcome data, with many more term (i.e., 37 weeks or longer) versus preterm pregnancies (i.e., less than 37 weeks), analysis of the PROTECT dataset presents many unique challenges. In addition to outcome imbalance, our database contains both quantitative and categorical data variables, adding some complexity to the analytical methods used. Applying straightforward correlation or regression analysis would be insufficient. Our datasets also contain a significant amount of missing data (incomplete records), providing noisy input to our algorithms. A further challenge is that we are working with a relatively limited set of complex data (only 2000 participants to date), so our models must be able to be built with a relatively small number of data samples.To overcome these challenges, we have implemented a cus-tomized end-to-end analytical toolchain which forms a pre-processing pipeline. Our framework performs general data filtering and handles missing data fields using a similarity-based approach. Next, we apply one of a number of different machine learning algorithms, including Linear Correlation, Normalized Mutual Information, Logistic Regression, and Decision Trees. We use these during both feature selection and model performance evaluation. Finally, we present top-ranked features produced by our model as potential key contributors of high preterm birth rates in Puerto Rico, and discuss results across these algorithms.

In this poster paper, we present an efficient data management framework used to support ongoing research in the NIEHS-supported Puerto Rico Testsite for Exploring Contamination Threats (PROTECT) Center. Our framework provides an efficient database subsystem and a series of associated workflows, supporting the tasks of data import, data cleaning, and secure transmission of privacy-sensitive PROTECT data, while enabling online data inquiry, visualization, and data processing to support data analytics.

An emerging class of applications attempt to make use of both the CPU and GPU in a heterogeneous system. The peak performance for these applications is achieved when both the CPU and GPU are used collaboratively. However, along with this increased gain in performance, power and energy management is a larger challenge. In this paper we address the issue of executing applications that utilize both the CPU and GPU in an energy efficient way. Towards this end, we propose a power management framework named Airavat that tunes the CPU, GPU and memory frequencies, synergestically, in order to improve the energy efficiency of collaborative CPU-GPU applications. Airavat uses machine learning-based prediction models, combined with feedback based Dynamic Voltage and Frequency Scaling to improve the energy efficiency of such applications. We demonstrate our framework on the NVIDIA Jetson TX1 and observe an improvement in terms of Energy Delay Product (EDP) by 24% with negligible performance loss.

GPUs have been shown to deliver impressive computing performance, while also providing high energy efficiency, across a wide range of high-performance and embedded system workloads. However, limited support for efficient communication and synchronization between the CPU and the GPU impacts our ability to fully exploit the benefits of heterogeneous systems. Recently, the Heterogeneous System Architecture (HSA) was introduced to address these issues with synchronization and communication, but given the low-level nature of HSA, it was not easily adopted by the broader programming community. In 2016, AMD described the Radeon Open Compute (ROC) platform that brings high-level programming frameworks such as OpenCL, HC++, and HIP to end users. These high-level programming frameworks offer a simpler programming experience by wrapping complex HSA APIs, while still delivering the power of HSA. To date, there has been little evaluation of the potential performance benefits and trade-offs of leveraging the ROC platform. In this work, we evaluate the performance of the ROC platform using the Hetero-Mark and DNNMark benchmark suites. Equipped with Hetero-Mark, we compare the performance of different programming frameworks, including OpenCL, HC++, and HIP on both integrated APUs and discrete GPUs. We also present three new CPU-GPU collaborative patterns and employ three new benchmarks to evaluate system-level atomics. With DNNMark and a new DNN Face Detection benchmark, we evaluate the performance of ROC libraries including rocBLAS and MIOpen. We also provide guidance on best practices to programmers when developing applications leveraging the ROC platform.

GPUs have become a very popular platform for accelerating the processing involved in deep learning applications. One class of popular variants, Convolutional Neural Networks (CNNs), have been widely deployed to run on GPUs. In many application settings, a GPU has sufficient computing power and memory space to accommodate the dense matrix operations performed during CNN training. However, few characterization studies have considered how CNNs can impact microarchitectural structures in a GPU. In this paper, we perform a characterization of one selected CNN workload as run on two different NVIDIA GPUs from distinct microarchitecture families, highlighting the impact that microarchitecture plays on this important class of workload. First, we analyze the performance implications of a CNN model using microarchitectural details on a layer-by-layer basis, and characterize the memory access behavior in the context of a typical GPU memory hierarchy, considering hardware resource utilization associated with each primitive in the CNN model. We identify major bottlenecks by considering the potential limits of using a single GPU. Additionally, we evaluate a number of optimization approaches, such as L1 cache bypassing and kernel fusion. L1 cache bypassing can achieve up to a 6.2% speedup for a single layer, but manipulating L1 cache provides very limited benefits in terms of application speedup, while kernel fusion provides an overall application speedup of 4.0%, on average.