Secure Two-party Computation (2PC) allows two parties to compute any function on their private inputs without revealing their inputs to each other. In the offline/on- line model for 2PC, correlated randomness that is independent of all inputs to the computation, is generated in a preprocessing (offline) phase and this randomness is then utilized in the online phase once the inputs to the parties become available. Most 2PC works focus on optimizing the online time as this overhead lies on the critical path. A recent paradigm for obtaining efficient 2PC protocols with low online cost is based on the cryptographic technique of function secret sharing (FSS).We build an end-to-end system Orca to accelerate the computation of FSS-based 2PC protocols with GPUs. Next, we observe that the main performance bottleneck in such accelerated protocols is in storage (due to the large amount of correlated randomness), and we design new FSS-based 2PC protocols for several key functionalities in ML which reduce storage by up to 5×. Compared to prior state-of-the-art on secure training accelerated with GPUs in the same computation model (PIRANHA, Usenix Security 2022), we show that Orca has 4% higher accuracy, 98 × lesser communication, and is 22 × faster on CIFAR-10. For secure ImageNet inference, Orca achieves sub-second latency for VGG-16 and ResNet-50 and outperforms the state-of-the-art by 8 — 103 ×.

Secure 2-party computation (2PC) enables secure inference that offers protection for both proprietary machine learning (ML) models and sensitive inputs to them. However, the existing secure inference solutions suffer from high latency and communication overheads, particularly for transformers. Function secret sharing (FSS) is a recent paradigm for obtaining efficient 2PC protocols with a preprocessing phase. We provide Sigma, the first end-to-end system for secure transformer inference based on FSS. By constructing new FSS-based protocols for complex machine learning functionalities, such as Softmax, GeLU and SiLU, and also accelerating their computation on GPUs, Sigma improves the latency of secure inference of transformers by 11 - 19x over the state-of-the-art that uses preprocessing and GPUs. We present the first secure inference of generative pre-trained transformer (GPT) models. In particular, Sigma executes Meta's Llama2 (available on HuggingFace) with 13 billion parameters in 44 seconds and GPT2 in 1.6 seconds

Multi-Chip Module (MCM) designs have emerged as a key technique to scale up a GPU’s compute capabilities in the face of slowing transistor technology. However, the disaggregated nature of MCM GPUs with many chiplets connected via in-package interconnects leads to non-uniformity. We explore the implications of MCM’s non-uniformity on the GPU’s virtual memory. We quantitatively demonstrate that an MCM-aware virtual memory system should aim to 1 leverage aggregate TLB capacity across chiplets while limiting accesses to L2 TLB on remote chiplets, 2 reduce accesses to page table entries resident on a remote chiplet’s memory during page walks. We propose MCM-aware GPU virtual memory (MGvm) that leverages static analysis techniques, previously used for thread and data placement, to map virtual addresses to chiplets and to place the page tables. At runtime, MGvm balances its objective of limiting the number of remote L2 TLB lookups with that of reducing the number of remote page table accesses to achieve good speedups (52%, on average) across diverse application behaviors.

GPU (Graphics Processing Unit) architecture has evolved to accelerate parts of a single application at a time. Consequently, several aspects of its architecture, particularly the virtual memory, have embraced a shared-mostly design. This implicitly assumes that a single application and, thus, one address space is resident in the GPU at a time. However, recent trends, e.g., deployment of GPUs in the cloud, necessitate efficient multi-tenancy. Multi-tenancy is needed for sharing the physical resources of a large server-class GPU across multiple concurrent tenants (applications) for resource consolidation while ensuring fairness among the tenants.We first quantify how different components of GPU's virtual memory can impede multi-tenancy. We show that shared page walkers are a key bottleneck under multi-tenancy. We, therefore, propose dynamic page walk stealing that enables soft partitioning of the shared pool of walkers- reducing destructive interference between the tenants while also aggregating resources where possible. Over today's design, we improve throughput by 37%, and weighted IPC by 15%, on average, over 45 workloads.