Privacy-preserving Machine Learning as a Service (MLaaS) enables the powerful cloud server to run its well-trained neural model upon the input from resource-limited client, with both of server's model parameters and client's input data protected. While computation efficiency is critical for the practical implementation of privacy-preserving MLaaS and it is inspiring to witness recent advances towards efficiency improvement, there still exists a significant performance gap to real-world applications. In general, state-of-the-art frameworks perform function-wise efficiency optimization based on specific cryptographic primitives. Although it is logical, such independent optimization for each function makes noticeable amount of expensive operations unremovable and misses the opportunity to further accelerate the performance by jointly considering privacy-preserving computation among adjacent functions. As such, we propose COIN: Conjunctive Optimization with Interleaved Nexus, which remodels mainstream computation for each function to conjunctive counterpart for composite function, with a series of united optimization strategies. Specifically, COIN jointly computes a pair of consecutive nonlinear-linear functions in the neural model by reconstructing the intermediates throughout the whole procedure, which not only eliminates the most expensive crypto operations without invoking extra encryption enabler, but also makes the online crypto complexity independent of filter size. Experimentally, COIN demonstrates 11.2x to 29.6x speedup over various function dimensions from modern networks, and 6.4x to 12x speedup on the total computation time when applied in networks with model input from small-scale CIFAR10 to large-scale ImageNet.

Machine Learning as a Service (MLaaS) is an innovative framework that enables a broad range of users to capitalize on the powerful Artificial Intelligence (AI) technologies. Nevertheless, MLaaS raises a privacy concern for both the client data and server model. To address this issue, several Secure Inference (SI) frameworks for MLaaS have been proposed in the literature that take advantage of Homomorphic Encryption (HE) operations. However, the computation cost of these frameworks is still high, especially for real-time applications. In this paper, we propose a novel system called input structure optimization for neural networks (TILE) to accelerate SI. The goal of TILE is to reduce both linear and non-linear computation costs, as well as non-linear communication costs in MLaaS, while maintaining the model accuracy. TILE defines two novel HE-friendly input structures: Internal Tile and External Tile Structures, aimed at reducing the HE operations for SI. We also develop a search mechanism to identify optimal application locations for these input structures. We apply TILE to widely used models such as VGG and ResNet, and datasets including Cifar10 and Tiny-ImageNet. The experimental results demonstrate that TILE effectively reduces the computation time, with up to 51.57% reduction for a state-of-the-art SI framework. Furthermore, TILE can also be applied to models that have already been pruned to significantly reduce the computation time, to further reduce the overall computation time by 25.90%.

To enable common users to capitalize on the power of deep learning, Machine Learning as a Service (MLaaS) has been proposed in the literature, which opens powerful deep learning models of service providers to the public. To protect the data privacy of end users, as well as the model privacy of the server, several state-of-the-art privacy-preserving MLaaS frameworks have also been proposed. Nevertheless, despite the exquisite design of these frameworks to enhance computation efficiency, the computational cost remains expensive for practical applications. To improve the computation efficiency of deep learning (DL) models, model pruning has been adopted as a strategic approach to remarkably compress DL models. However, for practical deep neural networks, a problem called pruning structure inflation significantly limits the pruning efficiency, as it can seriously hurt the model accuracy. In this paper, we propose MOSAIC, a highly flexible pruning framework, to address this critical challenge. By first pruning the network with the carefully selected basic pruning units, then assembling the pruned units into suitable HE Pruning Structures through smart channel transformations, MOSAIC achieves a high pruning ratio while avoiding accuracy reduction, eliminating the problem plagued by the pruning structure inflation. We apply MOSAIC to popular DL models such as VGG and ResNet series on classic datasets such as CIFAR-10 and Tiny ImageNet. Experimental results demonstrate that MOSAIC effectively and flexibly conducts pruning on those models, significantly reducing the Perm, Mult, and Add operations to achieve the global cost reduction without any loss in accuracy. For instance, in VGG-16 on Tiny ImageNet, the total cost is reduced to 21.14% and 29.49% under the MLaaS frameworks GAZELLE and CrypTFlow2, respectively.

Machine Learning as a Service (MLaaS) has paved the way for numerous applications for resource-limited clients, such as IoT/mobile users. However, it raises a great challenge for privacy, including both the data privacy of clients and model privacy of the server. While there have been extensive studies on privacy-preserving MLaaS, a direct adoption of current frameworks leads to intractable efficiency bottleneck for MLaaS with resource constrained clients. In this paper, we focus on MLaaS with resource constrained clients and propose a novel privacy-preserving framework called SPOT to address a unique challenge, the memory constraint of such clients, such as IoT /mobile devices, which results in significant computation stalls at the server in privacy-preserving MLaaS. We develop 1) a novel structure patching scheme to enable independent computations for sequential inputs at the server to eliminate the computation stall, and 2) a patch overlap tweaking scheme to minimize overlapped data between adjacent patches and thus enable more efficient computation with flexible cryptographic parameters. SPOT demonstrates significant improvement on computation efficiency for MLaaS with IoT /mobile clients. Compared with the state-of-the-art framework for privacy-preserving MLaaS, SPOT achieves up to 2 × memory utilization boost and a speedup up to 3 × on computation time for modern neural networks such as ResNet and VGG.

Privacy-preserving Machine Learning as a Service (MLaaS) enables the resource-limited client to cost-efficiently obtain inference output of a well-trained neural model that is possessed by the cloud server, with both client’s input and server’s model parameters protected. While efficiency plays a core role for practical implementation of privacy-preserving MLaaS and it is encouraging to witness recent advances towards efficiency improvement, there still exists a significant performance gap to real-world applications. The basic logic in state-of-the-art frameworks involves an individual computation for each function of the neural model, based on specific cryptographic primitives. While it is definitely logical, we look back to the necessity of this function-wise methodology and initiate the comprehensive exploration towards allied optimization for efficient privacy-preserving MLaaS. Under such fresh perspective, we remodel the computation process that is always from input to output of the same function in mainstream works, to the allied counterpart that is from one function’s input associated with the start of expensive overhead to another function’s output enabling effective circumvention of unnecessary cost within the procedure. As such we propose FIT (Function Input Tuning) which features by a computation module for composite function with a series of joint optimization strategies. Theoretically, FIT not only eliminates the most expensive crypto operations without invoking extra encryption enabler, but also makes the running-time crypto complexity independent of filter size. Experimentally, FIT demonstrates tens of times speedup over various function dimensions from modern models, and 4.5× to 35.5× speedup on the total computation time when plugged in neural networks with data from small-scale MNIST to large-scale ImageNet.

There is an increasing tendency to fine-tune large-scale pre-trained language models (LMs) using small private datasets to improve their capability for downstream applications. In this paper, we systematically analyze the pre-train and then fine-tune the process of generative LMs and show that the fine-tuned LMs would leak sensitive keywords of the private datasets even without any prior knowledge of the downstream tasks. Specifically, we propose a novel and efficient keyword inference attack framework to accurately and maximally recover sensitive keywords. Owing to the fine-tuning process, pre-trained and fine-tuned models might respond differently to identical input prefixes. To identify potential sensitive sentences for training the fine-tuend LM, we introduce a contrast difference score that assesses the response variations between a pre-trained LM and its corresponding fine-tuned LM. Following this, we iteratively fine-tune the pre-trained model using these sensitive sentences to minimize the disparity between the target model and the pre-trained model, thereby maximizing the number of inferred sensitive keywords. We implement two types of keyword inference attacks (i.e., domain and private) according to our framework and conduct comprehensive experiments on three downstream applications to evaluate the performance. The experimental results demonstrate that our domain keyword inference attack achieves a precision of 85%, while our private keyword inference attack can extract highly sensitive personal information for a significant number of individuals (approximately 0.3% of all customers in the private fine-tuning dataset, which contains 40,000 pieces of personal information).

The pandemics, such as COVID-19 are worldwide health risks and result in catastrophic impacts on the global economy. To prevent the spread of pandemics, it is critical to trace the contacts between people to identify the infection chain. Nevertheless, the privacy concern is a great challenge to contact tracing. Moreover, existing contact tracing apps cannot obtain the macro-level infection risk information, e.g., the hotspots where the infection occurs, which, however, is critical to optimize healthcare planning to better control and prevent the outbreak of pandemics. In this article, we develop a novel privacy-preserved pandemic tracing system, privacy-preserved pandemic infection risk computation (PRISC), to compute the infection risk through cellular-enabled IoT devices. In the PRISC system, there are three parties: 1) a mobile network operator (MNO); 2) a social network provider; and 3) the department of health. The physical contact records between users are obtained by the MNO from the users’ cellular-enabled IoT devices. The social contacts are obtained by the social network provider, while the health department has the records of pandemic patients. The three parties work together to compute a heatmap of pandemic infection risk in a region, while fully protecting the data privacy of each other. The heatmap provides both macro and micro-level infection risk information to help control pandemics. The experiment results indicate that PRISC can compute an infection risk score within a couple of seconds and a few mega-bytes (MBs) communication cost, for data sets with 100000 users.

In order to protect user privacy in Machine Learning as a Service (MLaaS), a series of ingeniously designed privacy-preserving frameworks have been proposed. The state-of-the-art approaches adopt Homomorphic Encryption (HE) for linear function and Garbled Circuits (GC)/Oblivious Transfer (OT) for nonlinear operation to improve computation efficiency. Despite the encouraging progress, the computation cost is still too high for practical applications. This work represents the first step to effectively prune privacy-preserving deep learning models to reduce computation complexity. Although model pruning has been discussed extensively in the machine learning community, directly applying the plaintext model pruning schemes offers little help to reduce the computation in privacy-preserving models. In this paper we propose Hunter, a structured pruning method that identifies three novel HE-friendly structures, i.e., internal structure, external structure, and weight diagonal to guide the pruning process. Hunter outputs a pruned model that, without any loss in model accuracy, achieves a significant reduction in HE operations (and thus the overall computation cost) in the privacy-preserving MLaaS. We apply Hunter in various deep learning models, e.g., AlexNet, VGG and ResNet over classic datasets including MNIST, CIFAR-10 and ImageNet. The experimental results demonstrate that, without accuracy loss, Hunter efficiently prunes the original networks to reduce the HE Perm, Mult, and Add operations. For example, in the state-of-the-art VGG-16 on ImageNet with 10 chosen classes, the total number of Perm is reduced to as low as 2% of the original network, and at the same time, Mult and Add are reduced to only 14%, enabling a significantly more computation-efficient privacy-preserving MLaaS.

In a ubiquitous environment enclosing cooperative Internet-of-Things (IoT) devices, individuals, and entities, digital identity management (DIM) becomes critical and challenging. DIM pertains to device identities authentication and verification to enable trustworthy service exchange, data collection, and decision making. DIM is the supporting pillar for all online services and the foundation for security and authentication mechanisms. Due to the extreme heterogeneity, scale, and configuration complexity of such environments, enabling trustworthy DIM is crucial and seriously challenging. In an IoT context, devices use local digital identities stored within a tamper-proof unit and verified by a centralized authority for authentication. The recent attacks on IoT systems showed how vulnerable such a design is. It is also an inherent problem that influences humans. From that, self-sovereign identity (SSI) has emerged as a decentralized DIM approach embracing the concept of portable self-possession identity. SSI was presented to couple the digital identity from the owner to enable large-scale cooperation. However, digital identity storage and verification still occur on the device and in a centralized manner. Utilizing a local single-point-of-failure storage memory for verifiable credentials is one of the considerable drawbacks in contemporary SSI. In this regard, this article introduces decentralized trustworthy-self-sovereign identity management (DT-SSIM), a novel decentralized trustworthy SSI management framework. DT-SSIM integrates the secret share scheme with the blockchain-based smart contracts technologies to provide transparent and trustworthy SSI-based DIM services for IoT. Storing IoT identity credentials outside the devices’ local storage preserves the identity credentials from being tampered with or misused. Evaluations and discussions show the resiliency assessment of the system and the cost and estimated running times for verification processes in DT-SSIM.

Deep learning (DL) has demonstrated superior success in various of applications, such as image classification, speech recognition, and anomalous detection. The unprecedented performance gain of DL largely depends on tremendous training data, high-performance computation resources, and well-designed model structures. However, privacy concerns raise from such necessities. First, as the training data are usually distributed among multiple parties, directly exposing and collecting such large amount of data could violate the laws especially for private information, such as personal identities, medical records, and financial profiles. Second, locally deploying advantageous computation resources is costly for individual party having partial data. Third, direct release of well-trained model parameters threatens the information about training data or the intellectual property of model owners. Therefore, individual party prefers outsourcing computation (data) in a secure way to powerful cloud servers such as Microsoft Azure, and how to enable the cloud servers to perform DL algorithms without revealing data owners’ private information and model owners’ valuable parameters is emerging as an urgent task, which is termed as privacy-preserving (outsourcing) DL. In this article, we review the state-of-the-art researches in privacy-preserving DL based on multiparty secure computation with data encryption and summarize these techniques in both training phase and inference phase. Specifically, we categorize the techniques with respect to the linear and nonlinear computations, which are the two basic building blocks in DL. Following a comprehensive overview of each research scheme, we present primary technical hurdles needed to be addressed and discuss several promising directions for future research.

The rapid development of ubiquitous and high-speed wireless communication technology has driven the increasingly serious demand for the Location-based Services (LBSs). In this circumstance, we propose a new crowd-sourced calibration-free and inertial sensor- independent indoor pedestrian localization approach, namely Mann-Whitney rank-sum testing via segments labeling (Marvel). In concrete terms, first of all, the motion paths are modeled by using the A* algorithm with the floor plan provided by the merchant, and then each motion path is segmented according to the preset expected localization accuracy. Second, by setting the signal similarity threshold, the Received Signal Strength (RSS) sequences which are collected by the human subjects following their daily routines in target environment are also segmented. Third, the proposed Marvel is adopted to cluster the motion path segments as well as RSS sequence segments respectively to construct the physical and signal logic graphs. Finally, by using the concept of backbone nodes diffusion mapping to establish the mapping relations between the physical and signal spaces, the pedestrian localization and the related motion analysis are conducted by the server. Furthermore, the extensive experimental results show that the proposed approach is capable of achieving higher localization accuracy compared with the current state-of-the-art approaches.

Privacy is a fundamental challenge for a variety of smart applications that depend on data aggregation and collaborative learning across different entities. In this paper, we propose a novel privacy-preserved architecture where clients can collaboratively train a deep model while preserving the privacy of each client’s data. Our main strategy is to carefully partition a deep neural network to two non-colluding parties. One party performs linear computations on encrypted data utilizing a less complex homomorphic cryptosystem, while the other executes non-polynomial computations in plaintext but in a privacy-preserved manner. We analyze security and compare the communication and computation complexity with the existing approaches. Our extensive experiments on different datasets demonstrate not only stable training without accuracy loss, but also 14 to 35 times speedup compared to the state-of-the-art system.