The training phase of deep neural networks requires substantial resources and as such is often performed on cloud servers. However, this raises privacy concerns when the training dataset contains sensitive content, e.g., facial or medical images. In this work, we propose a method to perform the training phase of a deep learning model on both an edge device and a cloud server that prevents sensitive content being transmitted to the cloud while retaining the desired information. The proposed privacy-preserving method uses adversarial early exits to suppress the sensitive content at the edge and transmits the task-relevant information to the cloud. This approach incorporates noise addition during the training phase to provide a differential privacy guarantee. We extensively test our method on different facial and medical datasets with diverse attributes using various deep learning architectures, showcasing its outstanding performance. We also demonstrate the effectiveness of privacy preservation through successful defenses against different white-box, deep and GAN-based reconstruction attacks. This approach is designed for resource-constrained edge devices, ensuring minimal memory usage and computational overhead.

Deep neural networks (DNNs) provide state-of-the-art accuracy for vision tasks, but they require significant resources for training. Thus, they are trained on cloud servers far from the edge devices that acquire the data. This issue increases communication cost, runtime, and privacy concerns. In this study, a novel hierarchical training method for DNNs is proposed that uses early exits in a divided architecture between edge and cloud workers to reduce the communication cost, training runtime, and privacy concerns. The method proposes a brand-new use case for early exits to separate the backward pass of neural networks between the edge and the cloud during the training phase. We address the issues of most available methods that, due to the sequential nature of the training phase, cannot train the levels of hierarchy simultaneously or they do it with the cost of compromising privacy. In contrast, our method can use both edge and cloud workers simultaneously, does not share the raw input data with the cloud, and does not require communication during the backward pass. Several simulations and on-device experiments for different neural network architectures demonstrate the effectiveness of this method. It is shown that the proposed method reduces the training runtime for VGG-16 and ResNet-18 architectures by 29% and 61% in CIFAR-10 classification and by 25% and 81% in Tiny ImageNet classification, respectively, when the communication with the cloud is done over a low bit rate channel. This gain in the runtime is achieved, while the accuracy drop is negligible. This method is advantageous for online learning of high-accuracy DNNs on sensor-holding low-resource devices such as mobile phones or robots as a part of an edge-cloud system, making them more flexible in facing new tasks and classes of data.

Preserving privacy is a growing concern in our society where cameras are ubiquitous. In this work, we propose a trainable image acquisition method that removes the sensitive information in the optical domain before it reaches the image sensor. The method benefits from a trainable optical convolution kernel, which transmits the desired information whilst filtering out the sensitive information, making it irretrievable against different privacy attacks in the digital domain. This is in contrast with the current digital privacy-preserving methods that are all vulnerable to direct access attacks. Also, in contrast with most of the previous optical privacy-preserving methods that cannot be trained, our method is data-driven and optimized for the specific application at hand. Moreover, there is no additional computation or power burden on the acquisition system since it works passively in the optical domain and can be even used in conjunction with other privacy-preserving techniques in the digital domain. We demonstrate our new, generic method in several scenarios such as smile or open-mouth detection as the desired attribute while the gender or wearing make-up is filtered out as the sensitive content. Through several experiments, we show that this method is able to reduce around$\mathbf {65}\%$of sensitive content while causing a negligible reduction in the desired information. Moreover, we tested our method by deep reconstruction attack and confirmed the ineffectiveness of this attack to reconstruct the original sensitive content. This new method has different use cases such as feedback systems for smart TV content or outdoor advertising.