The widespread adoption of online Large Language Models (LLMs) raises considerable privacy concerns, as prompts may inadvertently contain sensitive information exposed to LLM service providers. Limited by high computational costs, reduced response utility, and excessive system modifications, previous works based on local deployment, embedding perturbation, and homomorphic encryption are not feasible for online prompt-based LLM services. To address these issues, we introduce ProSan (Prompt Privacy Sanitizer), an end-to-end method for prompt privacy protection that generates prompts with task-irrelevant privacy removed, while preserving both utility and readability. It can also be seamlessly integrated into the online LLM service pipeline. To achieve high utility and contextual privacy, ProSan flexibly adjusts its protection targets and strength based on the importance of the words and the privacy leakage risk of the prompts. Additionally, ProSan is capable of adapting to diverse computational resource conditions, ensuring privacy protection for low-resource users. Our experiments demonstrate that ProSan effectively removes sensitive information across various tasks, including question answering, text summarization, and code generation, with minimal reduction in task performance.

We focus on Text-to-SQL semantic parsing from the perspective of retrieval-augmented generation.Motivated by challenges related to the size of commercial database schemata and the deployability of business intelligence solutions, we propose ASTRES that dynamically retrieves input database information and uses abstract syntax trees to select few-shot examples for in-context learning.Furthermore, we investigate the extent to which an in-parallel semantic parser can be leveraged for generating approximated versions of the expected SQL queries, to support our retrieval.We take this approach to the extreme-we adapt a model consisting of less than 500M parameters, to act as an extremely efficient approximator, enhancing it with the ability to process schemata in a parallelised manner.We apply ASTRES to monolingual and crosslingual benchmarks for semantic parsing, showing improvements over state-of-the-art baselines.Comprehensive experiments highlight the contribution of modules involved in this retrieval-augmented generation setting, revealing interesting directions for future work.

Vertical federated learning (VFL) is being used more and more widely in industry. One of its most common application scenarios is a two-party setting: a participant (i.e., the host), who exclusively owns the labels but possesses insufficient number of features, wants to improve its model performance by combining features from another participant (i.e., the client) of a different business group. The best deep ML architecture suits for this scenario is considered to be Split Neural Network (SplitNN), in which each participant runs a self-defined bottom model to learn the hidden representations (i.e., the local embeddings) of its local data and then forwards them to the host, who runs a top model to aggregate both the local embeddings to produce the final predicts. In this paper, we assume the client is malicious and demonstrate that she/he could inject a stealthy backdoor into the top model during the training to misclassify any sample to a pre-selected target class with a high probability by just replacing its local embedding with a special trigger vector regardless of the host-side embedding. This task is non-trivial because existing data poison attacks for backdoor injection in traditional models usually require to modify the labels of a set of trigger-tagged samples of non-target classes, which is impossible here as the client has no rights to access or modify the labels exclusively owned by the host. Targeting this challenge, we propose a SplitNN-dedicated data poison attack which does not require to modify any labels but just replaces the local embeddings of a very small number of target-class samples with a carefully constructed trigger vector during training. The experiments on four datasets show that our attack can achieve an attack rate as high as 94%, while bringing negligible side-effects to the model accuracy. Moreover, it is stealthy enough to resist various anomaly detection methods.

Pavlos Vougiouklis, Nikos Papasarantopoulos, Danna Zheng, David Tuckey, Chenxin Diao, Zhili Shen, Jeff Pan. Proceedings of the 13th International Joint Conference on Natural Language Processing and the 3rd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers). 2023.