Graphs-based neural networks are powerful tools for analyzing intricate non-euclidean data structures to solve complex real-world problems bounded by latency and throughput constraints. However, the computation structures operate on ultra-sparse and unstructured matrices resulting in load balancing and data locality challenges for the sparse matrixmatrix multiplication (SpMM) in massively parallel processors. To overcome these challenges, researchers have turned to model pruning techniques that effectively compress the hidden dimensions in the dense matrices without compromising model accuracy. However, exploiting sparsity for performance becomes challenging when the compressed dimensions cannot exploit the full potential of vector-level parallelism in single instruction multiple thread (SIMT) processors. We propose to virtualize the vector unit into virtual SIMT slots to map the low-dimensional SpMM computations, and improve parallel hardware utilization. The key challenge is to find parallel work to occupy the exposed virtualized slots. A static slot mapping strategy exposes parallelism by trading off load-balancing, memory coalescing, and data locality when distributing work among the slots. The key insight is that work efficiency and parallelism are highly input graph dependent, and static schemes cannot optimally exploit performance. An input-aware slot mapping strategy is proposed that adapts to the input graph's characteristics at SIMT granularity to expose maximum parallelism at runtime. The evaluation with an NVIDIA H100 GPU shows improved performance scaling using the proposed input-aware slot mapping heuristic compared to the static baseline.

Secure processors with hardware-enforced isolation are crucial for secure cloud computation. However, commercial secure processors have underestimated the capabilities of attackers and failed to provide secure execution environments capable of protecting sensitive information against side-channel attacks. Remote Attestation protocols based on traditional signature schemes are not secure under side-channel attacks anymore since their secret keys can be leaked. Previously, Key-Insulated Schemes (KIS) have been introduced to mitigate the damage caused by secret key exposure in cryptosystems by breaking the lifetime of secret keys into independent sessions. KIS protect the security of all other sessions if any session keys are compromised, however, provide no security guarantees for a compromised session. We introduce a new cryptographic primitive called One-Time Signature with Secret Key Exposure (OTS-SKE), which ensures no one can forge a valid signature of a new message or nonce even if all secret session keys are leaked. OTS-SKE enables us to sign attestation reports securely under a powerful adversary who can observe all digital states in secure enclaves through side-channel attacks. We also minimize the trusted computing base by introducing a secure co-processor that is only responsible for key generation into the system. Our experiments show that the signing of OTS-SKE is faster than KIS as well as Elliptic Curve Digital Signature Algorithm (ECDSA) used in Intel SGX.

Performing training and inference for Graph Neural Networks (GNNs) under tight latency constraints has become increasingly difficult as real-world input graphs continue to grow. Compared to traditional DNNs, GNNs present unique computational challenges due to their massive, unstructured, and sparse input graphs. Prior works have applied irregular and structured model pruning techniques to reduce the complexity of GNNs to accelerate GNN performance. However, irregular pruning techniques presented in the literature use floating point operations to estimate G NN performance, which does not reveal the true performance implications of model sparsity caused by the diminished parallelism of sparse matrix multiplication kernels. This paper quantitatively shows that irregular sparsity in G NN models is unable to be exploited to improve performance in parallel architectures that employ highly vectorized hardware. While structured pruning can overcome these issues, the existing structured pruning work for GNNs introduces performance scalability challenges as low-dimensional mapping of the pruned model is unable to exploit the full parallelism potential of the GPU's vectorized hardware. We propose PruneGNN, an optimized algorithm-architecture framework for structured GNN pruning. At the algorithm level, a dimension-pruning-aware sparse training method is proposed that achieves high sparsity while maintaining accuracy. At the architecture level, novel SIMD-aware kernels are proposed that exploit matrix-operator-level parallelism and unlock performance gains with reduced-dimension GNN models. The efficacy of the proposed framework is evaluated for end-to-end inference as well as training performance using real-world dynamic and static graphs on representative GNN models. Experimental results using an NVIDIA A100 GPU show that PruneGNN achieves an average of 2 x speedup over the prior structured pruning work for state-of-the-art GNN models.

Graph neural networks have seen tremendous adoption to perform complex predictive analytics on massive and unstructured real-world graphs. The trend in hardware accelerator designs has identified significant challenges with harnessing graph locality and workload imbalance due to ultra-sparse and irregular matrix computations at a massively parallel scale. This paper addresses the load imbalance challenge and identifies that state-of-the-art either introduces complex specialized hardware to auto-tune for load-balanced execution at runtime or relies on software-only approaches that exploit parallelism. We propose a novel software-only load-balancing sparse matrix-matrix (SpMM) algorithm that unlocks fine-grain parallelism while maintaining controlled need-based targeted synchronizations to achieve robust performance scaling. The MergePath-SpMM algorithm achieves superior performance using commercial offthe-shelf GPU processors when compared to state-of-the-art hardware accelerators and software-only implementations.

Network pruning is a widely used technique to reduce computation cost and model size for deep neural networks. However, the typical three-stage pipeline (i.e., training, pruning, and retraining (fine-tuning)) significantly increases the overall training time. In this article, we develop a systematic weight-pruning optimization approach based on surrogate Lagrangian relaxation (SLR), which is tailored to overcome difficulties caused by the discrete nature of the weight-pruning problem. We further prove that our method ensures fast convergence of the model compression problem, and the convergence of the SLR is accelerated by using quadratic penalties. Model parameters obtained by SLR during the training phase are much closer to their optimal values as compared to those obtained by other state-of-the-art methods. We evaluate our method on image classification tasks using CIFAR-10 and ImageNet with state-of-the-art multi-layer perceptron based networks such as MLP-Mixer; attention-based networks such as Swin Transformer; and convolutional neural network based models such as VGG-16, ResNet-18, ResNet-50, ResNet-110, and MobileNetV2. We also evaluate object detection and segmentation tasks on COCO, the KITTI benchmark, and the TuSimple lane detection dataset using a variety of models. Experimental results demonstrate that our SLR-based weight-pruning optimization approach achieves a higher compression rate than state-of-the-art methods under the same accuracy requirement and also can achieve higher accuracy under the same compression rate requirement. Under classification tasks, our SLR approach converges to the desired accuracy × faster on both of the datasets. Under object detection and segmentation tasks, SLR also converges 2× faster to the desired accuracy. Further, our SLR achieves high model accuracy even at the hardpruning stage without retraining, which reduces the traditional three-stage pruning into a two-stage process. Given a limited budget of retraining epochs, our approach quickly recovers the model’s accuracy.

In recent years, graph representation learning has gained significant popularity, which aims to generate node embeddings that capture features of graphs. One of the methods to achieve this is employing a technique called random walks that captures node sequences in a graph and then learns embeddings for each node using a natural language processing technique called Word2Vec. These embeddings are then used for deep learning on graph data for classification tasks, such as link prediction or node classification. Prior work operates on pre-collected temporal graph data and is not designed to handle updates on a graph in real-time. Real world graphs change dynamically and their entire temporal updates are not available upfront. In this paper, we propose an end-to-end graph learning pipeline that performs temporal graph construction, creates low-dimensional node embeddings, and trains multi-layer neural network models in an online setting. The training of the neural network models is identified as the main performance bottleneck as it performs repeated matrix operations on many sequentially connected low-dimensional kernels. We propose to unlock finegrain parallelism in these low-dimensional kernels to boost performance of model training.

As real-world graphs expand in size, larger GNN models with billions of parameters are deployed. High parameter count in such models makes training and inference on graphs expensive and challenging. To reduce the computational and memory costs of GNNs, optimization methods such as pruning the redundant nodes and edges in input graphs have been commonly adopted. However, model compression, which directly targets the sparsification of model layers, has been mostly limited to traditional Deep Neural Networks (DNNs) used for tasks such as image classification and object detection. In this paper, we utilize two state-of-the-art model compression methods (1) train and prune and (2) sparse training for the sparsification of weight layers in GNNs. We evaluate and compare the efficiency of both methods in terms of accuracy, training sparsity, and training FLOPs on real-world graphs. Our experimental results show that on the ia-email, wiki-talk, and stackoverflow datasets for link prediction, sparse training with much lower training FLOPs achieves a comparable accuracy with the train and prune method. On the brain dataset for node classification, sparse training uses a lower number FLOPs Oess than 1/7 FLOPs of train and prune method) and preserves a much better accuracy performance under extreme model sparsity. Our model sparsification code is publicly available on GitHubl1.

Network pruning is a widely used technique to reduce computation cost and model size for deep neural networks. However, the typical three-stage pipeline, i.e., training, pruning and retraining (fine-tuning) significantly increases the overall training trails. In this paper, we develop a systematic weight-pruning optimization approach based on Surrogate Lagrangian relaxation (SLR), which is tailored to overcome difficulties caused by the discrete nature of the weight-pruning problem while ensuring fast convergence. We further accelerate the convergence of the SLR by using quadratic penalties. Model parameters obtained by SLR during the training phase are much closer to their optimal values as compared to those obtained by other state-of-the-art methods. We evaluate the proposed method on image classification tasks using CIFAR-10 and ImageNet, as well as object detection tasks using COCO 2014 and Ultra-Fast-Lane-Detection using TuSimple lane detection dataset. Experimental results demonstrate that our SLR-based weight-pruning optimization approach achieves higher compression rate than state-of-the-arts under the same accuracy requirement. It also achieves a high model accuracy even at the hard-pruning stage without retraining (reduces the traditional three-stage pruning to two-stage). Given a limited budget of retraining epochs, our approach quickly recovers the model accuracy.