Network-on-chip (NoC) is a critical shared resource in scalable multicore processors; however, it is well-known that shared resources can lead to side-channel attacks. In this work, we demonstrate how contention for on-chip bandwidth in GPUs can lead to fine-grain information leakage and enable side-channel attacks. As a case study, we demonstrate how RSA key bit information can be leaked on a real GPU. We also describe how interconnect characteristics from the side-channel or an interconnect-gram can be used to fingerprint kernels executing on the GPU. To defend against such fine-grain side-channel attack, we propose secure arbitration that prevents information leakage while minimizing performance impact during normal execution. In particular, we present a novel ghost arbitration that prevents interconnect contention from being leveraged to leak information by keeping track of “ghost” requests or requests when other nodes receive free arbitration to enable least-recently-used priority. However, if the attacker reverse engineers the arbitration, a naive implementation of ghost arbitration can still lead to information leakage. Thus, we propose a weighted ghost arbitration that exploits “malicious” communication patterns to prevent information leakage with minimal loss in performance. Compared to previously proposed arbitration that is secure (e.g., strict time-division multiplexing), ghost arbitration is able to improve performance by up to$4\times$•

A critical component of high-throughput processors such as GPUs is the network-on-chip (NoC) that interconnects the large number of cores and the memory partitions together. In this work, we provide a detailed analysis, in terms of latency and bandwidth, of real GPU NoC across several generations of modern NVIDIA GPUs. Our analysis identifies how non-uniform latency exists between the cores and the memory partitions based on their physical location in the GPU. The non-uniformity can result in up to approximately 70 % difference in on-chip latency. In comparison, the bandwidth provided from the cores to the memory partitions is approximately uniform. However, recent GPUs that consist of multiple GPU “partitions” present different on-chip latency and bandwidth characteristics when communicating between the partitions. Based on our analysis of real GPU interconnect, we discuss potential implications including its impact on timing used in side-channel attacks as well as NoC microarchitectures. We show how the non-uniform latency can be exploited in a timing side-channel attack within a GPU as the core location impacts performance (or timing). In addition, proper understanding (and proper assumptions) of GPU NoC is critical to ensure a network that does not bottleneck the overall system performance.

As GPUs are becoming widely deployed in the cloud infrastructure to support different application domains, the security concerns of GPUs are becoming increasingly important. In particular, the support for multiprogramming in modern GPUs has led to new vulnerabilities since multiple kernels in a GPU can be executed at the same time. In this work, we propose a new microarchitectural timing covert channel for GPUs that can be established based on the shared, on-chip interconnect channels. We first reverse-engineer the organization of the on-chip networks in modern GPUs to understand the core placements throughout the GPU. The hierarchical organization of the GPU results in the sharing of interconnect bandwidth between neighboring cores. Based on this understanding, we identify how contention for the interconnect bandwidth can be exploited for a novel covert channel attack. We propose two types of interconnect-based covert channels that exploit the on-chip network hierarchy. Unlike cache-based covert channels, no states of the on-chip network need to be modified for communication in our interconnect-based covert channel and the impact of contention is very predictable. By exploiting the parallelism of GPUs, our proposed covert channel results in very high bandwidth – achieving approximately 24 Mbps of bandwidth on NVIDIA Volta GPUs and results in one of the highest known microarchitectural covert channel bandwidth.