Transformer-based Large Models (TLM), such as generative pre-trained models (GPT), have become increasingly popular for practical applications through Deep Learning as a Service (DLaaS). They have been extensively used in natural language processing and computer vision. However, concerns regarding potential private data leakage arise with this type of inference service. While some private inference techniques can protect privacy, they often introduce high latency and approximate replacements in the design protocols, resulting in changes to the model structure and decreased accuracy. In this research, we present SecureTLM, a private inference method based on secure multi-party computation (MPC) that does not require modifications to the underlying model structure. SecureTLM offers protocols for crucial computations in TLM, such as Multiplication, Softmax, GeLU, and LayerNorm, without altering the model structure. Experimental results demonstrate that SecureTLM ensures data privacy, maintains correctness, and achieves efficiency in private inference tasks.

Cover Caption: The cover image is based on the Research Article Active forgetting via influence estimation for neural networks by Xianjia Meng et al., https://doi.org/10.1002/int.22981.

The rapidly exploding of user data, especially applications of neural networks, involves analyzing data collected from individuals, which brings convenience to life. Meanwhile, privacy leakage in the applications as a potential threat needs to be addressed urgently. However, removing private information from models is difficult once the user's sensitive data enters machine learning models, particularly neural networks. Most of the previous amnestic methods based on retraining require full access to the training set of the target model and have limited improvements in computational resources and time improvement. In this paper, we propose Scrubber, which removes sensitive data from the original model via influence estimation to produce an unlearning model that is approximately indistinguishable from the retrained model. S crubber builds on the essential concept of influence function and reformulates the influence estimation as a closed-form update of forgetting. For learned models with strictly convex loss functions, our approach theoretically guarantees the effectiveness of forgetting while empirically demonstrating forgetting performance. For models with non-convex losses, we relax strictly convex assumptions by applying a damping term that allows us to make approximate estimates with negligible errors from the original assumption. Furthermore, experiments show that S crubber only causes less than 1% and 3% accuracy drop with more than 80% forgetting rate on average for logistic regression models and convolutional neural networks. The accuracy drop is reduced by 2%–3% compared to most state-of-the-art methods.