Haoran Li, Wei Fan, Yulin Chen, Cheng Jiayang, Tianshu Chu, Xuebing Zhou, Peizhao Hu, Yangqiu Song. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2025.

Vertical federated learning (VFL) is an increasingly popular, yet understudied, collaborative learning technique. In VFL, features and labels are distributed among different participants allowing for various innovative applications in business domains, e.g., online marketing. When deploying VFL, training data (labels and features) from each participant ought to be protected; however, very few studies have investigated the vulnerability of data protection in the VFL training stage. In this paper, we propose a posterior-difference-based data attack,VFLRecon, reconstructing labels and features to examine this problem. Our experiments show that standard VFL is highly vulnerable to serious privacy threats, with reconstruction achieving up to 92% label accuracy and 0.05 feature MSE, compared to our baseline with 55% label accuracy and 0.19 feature MSE. Even worse, this privacy risk remains during standard operations (e.g., encrypted aggregation) that appear to be safe. We also systematically analyze data leakage risks in the VFL training stage across diverse data modalities (i.e., tabular data and images), different training frameworks (i.e., with or without encryption techniques), and a wide range of training hyperparameters. To mitigate this risk, we design a novel defense mechanism,VFLDefender, dedicated to obfuscating the correlation between bottom model changes and labels (features) during training. The experimental results demonstrate thatVFLDefenderprevents reconstruction attacks during standard encryption operations (around 17% more effective than standard encryption operations).

In the era of big data, user data are often vertically partitioned and stored at different local parties. Exploring the data from all the local parties would enable data analysts to gain a better understanding of the user population from different perspectives. However, the publication of vertically-partitioned data faces a dilemma: on the one hand, the original data cannot be directly shared by local parties due to privacy concerns; on the other hand, independently privatizing the local datasets before publishing may break the potential correlation between the cross-party attributes and lead to a significant utility loss. Prior solutions compute the privatized multivariate distributions of different attribute sets for constructing a synthetic integrated dataset. However, these algorithms are only applicable for low-dimensional structured data and may suffer from large utility loss with the increase in data dimensionality. Following the idea of synthetic data generation, we propose VertiGAN, the first framework based on a generative adversarial network (GAN) for publishing vertically-partitioned data with privacy protection. The framework adopts a GAN model comprised of one multi-output global generator and multiple local discriminators. The generator is collaboratively trained by the server and local parties to learn the distribution of all parties' local data and is used to generate a high-utility synthetic integrated dataset on the server side. Additionally, we apply differential privacy (DP) during the training process to ensure strict privacy guarantees for the local data. We evaluate the framework's performance on a number of real-world datasets containing 68--1501 classification attributes and show that our framework is more capable of capturing joint distributions and cross-attribute correlations compared to statistics-based baseline algorithms. Moreover, with a privacy guarantee of epsilon=8, our framework achieves around a 2%~15% improvement in classification accuracy compared to the baseline algorithms. Extensive experimental results demonstrate the capability and efficiency of our framework in synthesizing vertically-partitioned data while striking a satisfactory utility-privacy balance.

Abstract Business intelligence and AI services often involve the collection of copious amounts of multidimensional personal data. Since these data usually contain sensitive information of individuals, the direct collection can lead to privacy violations. Local differential privacy (LDP) is currently considered a state-ofthe-art solution for privacy-preserving data collection. However, existing LDP algorithms are not applicable to high-dimensional data; not only because of the increase in computation and communication cost, but also poor data utility. In this paper, we aim at addressing thecurse-of-dimensionalityproblem in LDP-based high-dimensional data collection. Based on the idea of machine learning and data synthesis, we propose DP-Fed-Wae, an efficient privacy-preserving framework for collecting high-dimensional categorical data. With the combination of a generative autoencoder, federated learning, and differential privacy, our framework is capable of privately learning the statistical distributions of local data and generating high utility synthetic data on the server side without revealing users’ private information. We have evaluated the framework in terms of data utility and privacy protection on a number of real-world datasets containing 68–124 classification attributes. We show that our framework outperforms the LDP-based baseline algorithms in capturing joint distributions and correlations of attributes and generating high-utility synthetic data. With a local privacy guarantee ∈ = 8, the machine learning models trained with the synthetic data generated by the baseline algorithm cause an accuracy loss of 10% ~ 30%, whereas the accuracy loss is significantly reduced to less than 3% and at best even less than 1% with our framework. Extensive experimental results demonstrate the capability and efficiency of our framework in synthesizing high-dimensional data while striking a satisfactory utility-privacy balance.

Abstract Vertical federated learning (VFL), a variant of federated learning, has recently attracted increasing attention. An active party having the true labels jointly trains a model with other parties (referred to as passive parties ) in order to use more features to achieve higher model accuracy. During the prediction phase, all the parties collaboratively compute the predicted confidence scores of each target record and the results will be finally returned to the active party. However, a recent study by Luo et al . [28] pointed out that the active party can use these confidence scores to reconstruct passive-party features and cause severe privacy leakage. In this paper, we conduct a comprehensive analysis of privacy leakage in VFL frameworks during the prediction phase. Our study improves on previous work [28] regarding two aspects. We first design a general gradient-based reconstruction attack framework that can be flexibly applied to simple logistic regression models as well as multi-layer neural networks. Moreover, besides performing the attack under the white-box setting, we give the first attempt to conduct the attack under the black-box setting. Extensive experiments on a number of real-world datasets show that our proposed attack is effective under different settings and can achieve at best twice or thrice of a reduction of attack error compared to previous work [28]. We further analyze a list of potential mitigation approaches and compare their privacy-utility performances. Experimental results demonstrate that privacy leakage from the confidence scores is a substantial privacy risk in VFL frameworks during the prediction phase, which cannot be simply solved by crypto-based confidentiality approaches. On the other hand, processing the confidence scores with information compression and randomization approaches can provide strengthened privacy protection.

Federated Learning (FL) [ 31 ] is a decentralized learning mechanism that has attracted increasing attention due to its achievements in computational efficiency and privacy preservation. However, recent research highlights that the original FL framework may still reveal sensitive information of clients’ local data from the exchanged local updates and the global model parameters. Local Differential Privacy (LDP), as a rigorous definition of privacy, has been applied to Federated Learning to provide formal privacy guarantees and prevent potential privacy leakage. However, previous LDP-FL solutions suffer from considerable utility loss with an increase of model dimensionality. Recent work [ 29 ] proposed a two-stage framework that mitigates the dimension-dependency problem by first selecting one “important” dimension for each local update and then perturbing the dimension value to construct the sparse privatized update. However, the framework may still suffer from utility loss because of the insufficient per-stage privacy budget and slow model convergence. In this article, we propose an improved framework, SignDS-FL , which shares the concept of dimension selection with Reference [ 29 ], but saves the privacy cost for the value perturbation stage by assigning random sign values to the selected dimensions. Besides using the single-dimension selection algorithms in Reference [ 29 ], we propose an Exponential Mechanism-based Multi-Dimension Selection algorithm that further improves model convergence and accuracy. We evaluate the framework on a number of real-world datasets with both simple logistic regression models and deep neural networks. For training logistic regression models on structured datasets, our framework yields only a \( \sim \) 1%–2% accuracy loss in comparison to a \( \sim \) 5%–15% decrease of accuracy for the baseline methods. For training deep neural networks on image datasets, the accuracy loss of our framework is less than \( 8\% \) and at best only \( 2\% \) . Extensive experimental results show that our framework significantly outperforms the previous LDP-FL solutions and enjoys an advanced utility-privacy balance.

The neural network model is having a significant impact on many real-world applications. Unfortunately, the increasing popularity and complexity of these models also amplifies their security and privacy challenges, with privacy leakage from training data being one of the most prominent issues. In this context, prior studies proposed to analyze the abstraction behavior of neural network models, e.g., RNN, to understand their robustness. However, the existing research rarely addresses privacy breaches caused by memorization in neural language models. To fill this gap, we propose a novel approach, DeepMemory, that analyzes memorization behavior for a neural language model. We first construct a memorization-analysis-oriented model, taking both training data and a neural language model as input. We then build a semantic first-order Markov model to bind the constructed memorization-analysis-oriented model to the training data to analyze memorization distribution. Finally, we apply our approach to address data leakage issues associated with memorization and to assist in dememorization. We evaluate our approach on one of the most popular neural language models, the LSTM-based language model, with three public datasets, namely, WikiText-103, WMT2017, and IWSLT2016. We find that sentences in the studied datasets with low perplexity are more likely to be memorized. Our approach achieves an average AUC of 0.73 in automatically identifying data leakage issues during assessment. We also show that with the assistance of DeepMemory, data breaches due to memorization of neural language models can be successfully mitigated by mutating training data without reducing the performance of neural language models.

Fuzzy commitment is an efficient template protection algorithm that can improve security and safeguard privacy of biometrics. Existing theoretical security analysis has proved that although privacy leakage is unavoidable, perfect security from information-theoretical points of view is possible when bits extracted from biometric features are uniformly and independently distributed. Unfortunately, this strict condition is difficult to fulfill in practice. In many applications, dependency of binary features is ignored and security is thus suspected to be highly overestimated. This paper gives a comprehensive analysis on security and privacy of fuzzy commitment regarding empirical evaluation. The criteria representing requirements in practical applications are investigated and measured quantitatively in an existing protection system for 3D face recognition. The evaluation results show that a very significant reduction of security and enlargement of privacy leakage occur due to the dependency of biometric features. This work shows that in practice, one has to explicitly measure the security and privacy instead of trusting results under non-realistic assumptions.