Federated learning has emerged as a potential solution to overcome the bottleneck posed by the near exhaustion of public text data in training large language models. There are claims that the strategy of exchanging gradients allows using text data including private information. Although recent studies demonstrate that data can be reconstructed from gradients, the threat for text data seems relatively small due to its sensitivity to even a few token errors. However, we propose a novel attack method FET, indicating that it is possible to Fully Expose Text data from gradients. Unlike previous methods that optimize continuous embedding vectors, we directly search for a text sequence with gradients that match the known gradients. First, we infer the total number of tokens and the unique tokens in the target text data from the gradients of the embedding layer. Then we develop a discrete optimization algorithm, which globally explores the solution space and precisely refines the obtained solution, incorporating both global and local search strategies. We also find that gradients of the fully connected layer are dominant, providing sufficient guidance for the optimization process. Our experiments show a significant improvement in attack performance, with an average increase of 39% for TinyBERT-6, 20% for BERT-base and 15% for BERT-large in exact match rates across three datasets. These findings highlight serious privacy risks in text data, suggesting that using smaller models is not an effective privacy-preserving strategy.

In privacy-preserving machine learning with vertically distributed data, private entity alignment methods are used to securely match and utilize features of the same samples. However, existing methods not only risk exposing sample intersections and introducing unnecessary samples but also face a gap in adapting to multi-party scenarios. To address these limitations, we propose Peafowl, a novel multi-party private entity alignment protocol. Peafowlachieves entity alignment among multiple parties through a mapping from original datasets to intersections, termed permutation. This method mitigates intersection disclosure and sample redundancy concerns by avoiding direct use of the intersection. The proposed protocol leverages a cloud server that utilizes secret-shared shuffle to protect the privacy of the permutation, in case of colluding data providers reconstructing intersections. Further, by integrating a seed homomorphic pseudorandom generator, Peafowlavoids the intensive communication of secret sharing and achieves superior runtime performance. Additionally, an offline/online variant is introduced to ensure a linear growth in communication and computation complexity relative to the dataset size by pre-computing permutation calculations. Implemented on a real PPML framework, the protocol demonstrates practical efficiency in various multi-party settings. Experimental results indicate that Peafowl’s overhead is less than 1% of the total training cost, while the offline/online variant achieves approximately a 50% reduction in online runtime. Overall, Peafowloffers an efficient and straightforward solution for multi-party PPML, making it an attractive option for implementation and future improvements.