Since the increasing complexity of deep learning models, tensor decomposition is one of the promising solutions that reduce computational complexity in deep learning models. By decomposing a convolution layer with a large weight tensor into multiple layers with smaller weight tensors, tensor decomposition can reduce the number of operations and weight memory spaces. However, existing tensor decomposition schemes face difficulties in reducing peak memory usage of the entire inference. The decomposed layers produce the reduced-sized tensors during inference, but the reduced tensors should be restored to their original sizes due to skip connections and non-decomposed activation layers between the decomposed layers. To reduce the peak memory usage of the end-to-end inference of the decomposed models, this work proposes a new tensor memory optimization scheme and its prototype compiler, called TeMCO. TeMCO replaces the original internal tensors used in the skip connections with reduced internal tensors derived by the decomposed layers. In addition, TeMCO fuses the decomposed layers and the non-decomposed activation layer and thus keeps the reduced internal tensors produced without restoring them. Thanks to the optimizations, this work reduces memory usage of internal tensors by 75.7% for 10 models of 5 deep learning architectures.

Fully homomorphic encryption (FHE) offers a promising solution for privacy-preserving cloud computing by allowing cloud servers to compute on encrypted data without decryption. However, its applicability is limited by the programming burden of ciphertext management and considerable operational latency. Recently proposed FHE compilers automate ciphertext management, but they transform all data into ciphertexts without filtering private data, thus unnecessarily increasing FHE ciphertexts and the overall latency. This work introduces a new privacy-authority type, called privacy-set (PSet), that allows programmers to annotate authorized devices for each unit of private data. Moreover, this work proposes a new privacy authority-aware compiler that automatically transforms a PSet-annotated plain program into an FHE-enabled edge-cloud cooperative program with operation authority- and latency-aware partitioning. This work evaluates the PSet compiler with six machine learning and deep learning applications, and demonstrates that the PSet compiler performs 4.92 times faster than the existing FHE compilers that do not support edge-cloud partitioning.

This work proposes Occamy, a new memory-efficient DNN compiler that reduces the memory usage of a DNN model without affecting its accuracy. For each DNN operation, Occamy analyzes the dimensions of input and output tensors, and their liveness within the operation. Across all the operations, Occamy analyzes liveness of all the tensors, generates a memory pool after calculating the maximum required memory size, and schedules when and where to place each tensor in the memory pool. Compared to PyTorch, on an integrated embedded GPU for six DNNs, Occamy reduces the memory usage by 34.6% and achieves a geometric mean speedup of 1.25×.

Only with a right schedule and a right topology layout, a graph algorithm can be efficiently processed on GPUs. Existing GPU graph processing frameworks try to find an optimal schedule and topology layout for an algorithm via iterative search, but they fail to find the optimal configuration because their schedules and topology layouts are tightly coupled in their processing models. Moreover, their tightly coupled schedules and topology layouts make it difficult for developers to extend the tuning space. To easily enlarge the tuning space of GPU graph processing, this work proposes a new GPU graph processing abstraction scheme that fully decouples schedules, topology layouts, and algorithms from each other with abstraction interfaces. Moreover, this work proposes GRAssembler, a new GPU graph processing framework that efficiently integrates the decoupled schedule, topology layout, and algorithm without abstraction overhead. Thanks to the efficient decoupling and integration, GRAssembler increases the tuning space from 336 to 4,480 and achieves 30.4% higher performance on geomean average, compared to the state-of-the-art GPU graph processing framework.

Serverless computing like Function-as-a-Service (FaaS) is attractive for IoT service providers, liberating the providers from server maintenance. Since a data processing function is executed on the cloud instead of a dedicated server in the FaaS platform, the service users send their private data in their IoT devices to the third-party cloud, taking privacy leakage risks. Homomorphic encryption (HE) can preserve the privacy by enabling encrypted data processing on the cloud, but using HE for every data item incurs large computation and communication overheads. This work proposes SelectiveCrypt, a compiler-assisted semantic-aware encryption scheme that applies different cryptographic primitives depending on the operations on each data item. SelectiveCrypt homomorphically encrypts data items if arithmetic operations are applied to the data, while SelectiveCrypt encrypts data items with a symmetric key if the data are stored in the cloud without any arithmetic operation. The SelectiveCrypt framework consists of a compiler and its runtime system. The SelectiveCrypt compiler statically analyzes the data processing, determines an appropriate cryptographic primitive for each data item, and automatically transforms arithmetic operations into the homomorphic computation. The SelectiveCrypt runtime encrypts and decrypts the data items according to the static analysis result. This work evaluates the prototype SelectiveCrypt framework with five benchmarks that reflect real-world IoT scenarios. The evaluation results show that the SelectiveCrypt framework successfully reduces response time and communication overhead by 1.59 times and 9.61 times, respectively, compared with a HE scheme.