How to preserve the data privacy during the training of deep neural network (DNN) is a key security concern in the artificial intelligence era. However, most existing solutions based on homomorphic encryption and Trusted Execution Environment (TEE) are incompatible with heterogeneous Neural Network Accelerators (NNAs), leading to significant performance loss. We propose a novel method based on vector decomposition to allocate operators across different NNAs, ensuring both throughput and privacy simultaneously. Furthermore, based on this approach, we have designed a compiler that automatically converts front-end model descriptions into backend encrypted computation graphs, which is running securely over trusted and untrusted hardware. This compiler heuristically determines the allocation scheme based on hardware affinity and cross-hardware communication costs, significantly reducing additional overhead. Experimental results demonstrate that our method does not incur extra accuracy costs and achieves a throughput significantly higher than existing methods. Deploying our approach at scale on a platform with a billion users, we have verified its negligible impact on real-world operations while ensuring the privacy protection capability for cross-domain data.

Recommendation systems (RS) play a vital role in various domains. However, under recent data regulations like General Data Protection Regulation (GDPR), traditional RS that rely on collecting user's interaction data centrally face significant challenges. Federated learning (FL) enables collaborative model training among users while keeping their private data locally. Yet, the constrained resources of devices often limit the size of the learned model, resulting in suboptimal recommendation performance. To overcome the dilemma of data accessibility and model size, we propose DualRec, a novel collaborative training framework for device and cloud recommendation models. In DualRec, users train lightweight models on devices to harness their local private data, while a larger model is simultaneously trained on the cloud server to exploit its substantial resources. Devices and the cloud server collaboratively train their models, compensating for individual limitations of model size and data availability, enabling mutual empowerment and benefits. Specifically, we introduce an efficient aggregation mechanism for recommendation models to boost the collaborative training performance of device models. With the learned device models, we propose to generate pseudo user interaction data to train the server model. To enhance the training performance of the server model, we design an automated denoising mechanism to mitigate the negative impact of noisy samples in the generated pseudo dataset. Finally, the learned knowledge of the server model is distilled to device models for enhanced on-device recommendation performance. Extensive experiments demonstrate the superior performance of DualRec compared to state-of-the-art baselines.

The embedded software may migrate the collected data to the server for DNN computation acceleration, which may compromise privacy. We propose a DNN computation framework that combines TEE and NNA to address the privacy leakage problem. We design an NNA-friendly encryption method that enables NNA to correctly compute the encrypted linear input. Facing the overhead of TEE-NNA interaction, we design a pipeline-based prefetch mechanism that can reduce the TEE interaction overhead. Experimentally, our approach proves to be compatible with a wide range of NPUs and TPUs, and improves the performance by 8-19 times over the TEE scheme.

Today’s intelligent services are built on well-trained deep neural network (DNN) models, which usually require large private datasets along with a high cost for model training. It consequently makes the model providers cherish the pre-trained DNN models and only distribute them to authorized users. However, malicious users can steal these valuable models for abuse, illegal copy and redistribution. Attackers can also extract private features from even authorized models to leak partial training datasets. They both violate privacy. Existing techniques from secure community attempt to avoid parameter leakage during model authorization but yet cannot solve privacy issues sufficiently. In this paper, we propose a privacy-preserving model authorization approach, AgAuth, to resist the aforementioned privacy threats. We devise a novel scheme called Information-Agnostic Conversion (IAC) for forwarding procedure to eliminate residual features in model parameters. Based on it, we then propose Inference-on-Ciphertext (CiFer) mechanism for DNN reasoning, which includes three stages in each forwarding. The Encrypt phase first converts the proprietary model parameters to demonstrate uniform distribution. The Forward stage per-forms forwarding function without decryption at authorized side. Specifically, this stage just computes over ciphertext. The Decrypt phase finally recovers the information-agnostic outputs to informative output tensor for real-world services. In addition, we implement a prototype and conduct extensive experiments to evaluate its performance. The qualitative and quantitative results demonstrate that our solution AgAuth is privacy-preserving to defend against model theft and feature leakage, without accuracy loss or notable performance decrease.

Time-efficient artificial intelligence (AI) service has recently witnessed increasing interest from academia and industry due to the urgent needs in massive smart applications such as self-driving cars, virtual reality, high-resolution video streaming, etc. Existing solutions to reduce AI latency, like edge computing and heterogeneous neural-network accelerators (NNAs), face high risk of privacy leakage. To achieve both low-latency and privacy-preserving purposes on edge servers (e.g., NNAs), this paper proposes ENIGMA that can exploit the trusted execution environment (TEE) and heterogeneous NNAs of edge servers for edge inference. The low-latency is supported by a new ahead-of-time analysis framework for analyzing the linearity of multilayer neural networks, which automatically slices forward-graph and assigns sub-graphs to TEE or NNA. To avoid privacy leakage issue, we then introduce a pre-forwarded cipher generation (PFCG) scheme for computing linear sub-forward-graphs on NNA. The input data is encrypted to ciphertext that can be computed directly by linear sub-graphs, and the output can be decrypted to obtain the correct output. To enable non-linear computation of sub-graphs on TEE, we use ring-cache and automatic vectorization optimization to address the memory limitation of TEE. Qualitative analysis and quantitative experiments on GPU, NPU and TPU demonstrate that ENIGMA is not only compatible with heterogeneous NNAs, but also can avoid leakages of private features with latency as low as 50-milliseconds.

While tremendous growing mobile apps offer users rich services and functionalities, they also bring significant performance and energy issues. Code sharing is promising to address these issues, but existing application-level code sharing is rather restrictive. This paper develops the a transparent machine code sharing for mobile devices, and presents its design, implementation, and deployment. SnapCode enables machine code sharing across a wide variety of commercial off-the-shelf Android devices. By sharing and running machine code, SnapCode can offer significant speed-ups: an average speed-up of 9.9X for one-time trial apps, and up to 120X in apps’ regular uses. In addition, it can save more than 80 percent energy consumption.

In cloud and edge networks, federated learning involves training statistical models over decentralized data, where servers aggregate models through intermediate updates trained from clients. By utilizing private and local data it improves quality of personalized services and reduces user's concern for privacy. However, federated learning still leaks multimedia features through trained intermediate updates and thereby is not privacy-preserving for multimedia. Existing techniques applied from secure community attempt to avoid multimedia features leakages for federated learning but yet cannot address issues of privacy. In this paper, we propose a privacy-preserving solution that avoids multimedia privacy leakages in federated learning. Firstly, we devise a novel encryption scheme called Non-Informative Transformation (NIT) for federated aggregation to eliminates residual multimedia features in intermediate updates. Based on the scheme, we then propose Just-Learn-over-Ciphertext (JLoC) mechanism for federated learning, which includes three stages in each model iteration. The Encrypt stage encrypts intermediate updates and makes it non-informative distribution at clients. The Aggregate stage performs model aggregation without decryption at servers. Specifically, this stage just computes over ciphertext, and its output of aggregation also keeps non-informative. The Decrypt stage converts non-informative outputs of aggregation to available parameters for the next iteration at clients. Moreover, we implement a prototype and conduct experiments to evaluate its privacy and performance on real devices. The experimental results demonstrate that our methods can defend against potential attacks for multimedia privacy leakages without accuracy loss in commercial off-the-shelf products.

Deep learning provides new opportunities for mobile applications to achieve higher performance than before. Rather, the deep learning implementation on mobile device today is largely demanding on expensive resource overheads, imposes a significant burden on the battery life and limited memory space. Existing methods either utilize cloud or edge infrastructure that require to upload user data, however, resulting in a risk of privacy leakage and large data transfers; or adopt compressed deep models, nevertheless, downgrading the algorithm accuracy. This paper provides DeepShark, a platform to enable mobile devices with the ability of flexible resource allocation in using commercial-off-the-shelf (COTS) deep learning systems. Compared to existing approaches, DeepShark seeks a balanced point between time and memory efficiency by user requirements, breaks down sophisticated deep model into code block stream and incrementally executes such blocks on system-on-chip (SoC). Thus, DeepShark requires significantly less memory space on mobile device and achieves the default accuracy. In addition, all referred user data of model processing is handled locally, thus to avoid unnecessary data transfer and network latency. DeepShark is now developed on two COTS deep learning systems, i.e., Caffe and TensorFlow. The experimental evaluations demonstrate its effectiveness in the aspects of memory space and energy cost.

Videos over HTTP adaptive streaming have been the most popular vehicle for delivering media content on mobile platform. Rather, today's mobile video streaming are excessively tailored for visual quality, imposing a heavy burden on user's data budget. In this paper, we aim to optimize mobile video streaming of low bitrate efficiency with considering human visual acuity, i.e., preferably avoid sacrificing viewing quality. First, we conduct to in-depth analysis of mobile HTTP adaptive video streaming with a focus not only on how it works, but also on the significance of bitrate saving. Second, we identify a novel research problem on excessive visual quality which leads to bitrate-inefficient video streaming, and propose a flexible system called EyeTube to address it. Specifically, we apply dynamic resolution scaling on mobile video streaming to trade off the bitrate efficiency and user viewing experience Third, we derive general principles for achieving bitrate-efficient mobile video streaming, and employ the principles to an open source web browser, i.e., Chromium, to verify its applicability. An end-to-end EyeTube system is implemented on Samsung smartphones, and the efficiency are evaluated against 10 popular YouTube videos. Experimental results show that all the bitrates of the 10 videos can be reduced by at least 54.2% on average and up to 90.9% at most when the resolution is quartered. A user study with 40 respondents has indicated that our system can achieve good performance on both bitrate saving and high viewing quality.