International audience

Offsite-tuning is a privacy-preserving method for tuning large language models (LLMs) by sharing a lossy compressed emulator from the LLM owners with data owners for downstream task tuning. This approach protects the privacy of both the model and data owners. However, current offsite tuning methods often suffer from adaptation degradation, high computational costs, and limited protection strength due to uniformly dropping LLM layers or relying on expensive knowledge distillation. To address these issues, we propose ScaleOT, a novel privacy-utility-scalable offsite-tuning framework that effectively balances privacy and utility. ScaleOT introduces a novel layerwise lossy compression algorithm that uses reinforcement learning to obtain the importance of each layer. It employs lightweight networks, termed harmonizers, to replace the raw LLM layers. By combining important original LLM layers and harmonizers in different ratios, ScaleOT generates emulators tailored for optimal performance with various model scales for enhanced privacy protection. Additionally, we present a rank reduction method to further compress the original LLM layers, significantly enhancing privacy with negligible impact on utility. Comprehensive experiments show that ScaleOT can achieve nearly lossless offsite tuning performance compared with full fine-tuning while obtaining better model privacy.

Kai Yao, Zhaorui Tan, Penglei Gao, Lichun Li, Kaixin Wu, Yinggui Wang, Yuan Zhao, Yixin Ji, Jianke Zhu, Wei Wang. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.

Gradient Boosting Decision Tree (GBDT) and its variants are widely used in industry. They have achieved remarkable success in numerous machine learning competitions and practical applications. Secure Multi-Party Computation (MPC) allows multiple data owners to compute a function jointly while keeping their input private. In this work, we present Gibbon, a secure two-party GBDT training framework on a vertically split dataset, where two data owners each hold different features of the same data samples. Compared with the state-of-the-art Squirrel (USENIX'Sec 2023), for most parameter settings, Gibbon achieves 2×-4× reduction in running time and 2×-3× reduction in communication.

Federated learning (FL) is a privacy-preserving machine learning paradigm that enables collaborative model training across multiple distributed clients without disclosing their raw data. Personalized federated learning (pFL) has gained increasing attention for its ability to address data heterogeneity. However, most existing pFL methods assume that each client's data follows a single distribution and learn one client-level personalized model for each client. This assumption often fails in practice, where a single client may possess data from multiple sources or domains, resulting in significant intra-client heterogeneity and suboptimal performance. To tackle this challenge, we propose pFedBayesPT, a fine-grained instance-wise pFL framework based on visual prompt tuning. Specifically, we formulate instance-wise prompt generation from a Bayesian perspective and model the prompt posterior as an implicit distribution to capture diverse visual semantics. We derive a variational training objective under the semi-implicit variational inference framework. Extensive experiments on benchmark datasets demonstrate that pFedBayesPT consistently outperforms existing pFL methods under both feature and label heterogeneity settings.

This paper studies MPC based decision graph inference (MDGI) where decision graphs (generalization of decision trees) are very popular machine learning models. In MDGI, a modeler holding a private model and a data owner holding private feature vectors jointly run model inference on each vector through an MPC protocol, which outputs the inference result without revealing the model or vector. Several noteworthy MDGI solutions have been proposed, but they remain unsatisfactory for large models due to the high communication cost of oblivious decision, the most complex component in MDGI. Oblivious decision securely evaluates binary tests (Boolean-valued functions) over features without revealing test type, parameter, feature index, or value. All constant-round oblivious decision protocols suffer from high communication costs due to bitwise encryption and transmission. Moreover, most support only one of the three common test types: threshold comparison, equality test, and set containment. We propose Homomorphic Lookup Table (HLT), a novel MPC technique for oblivious decision. HLT circumvents the bitwiseencryption and heavy-communication issue by adopting tablelookup-style computation and amortizing the cost of encrypted table transfer across many inferences. We carefully design the table structure and lookup rules, and integrate homomorphic encryption to optimize performance and support multiple test types. HLT achieves 78 × −4151× reduction in communication for oblivious decision, and is the first method to support all three common test types. Based on HLT, we design constant-round MDGI solutions for two widely used decision graphs: decision trees and scorecards. This is the first privacy-preserving solution for scorecards, and our decision tree solution reduces communication by 27 × −321×.

Vertical Federated Learning (VFL) is a solution increasingly used by companies with the same user group but differing features, enabling them to collaboratively train a machine learning model. VFL ensures that clients exchange intermediate results extracted by their local models, without sharing raw data. However, in practice, VFL encounters several challenges, such as computational and communication overhead, privacy leakage risk, and adversarial attack. Our study reveals that the usage of floating-point (FP) numbers is a common factor causing these issues, as they can be redundant and contain too much information. To address this, we propose a new architecture called rounding layer, which converts intermediate results to integers. Our theoretical analysis and empirical results demonstrate the benefits of the rounding layer in reducing computation and memory overhead, providing privacy protection, preserving model performance, and mitigating adversarial attacks. We hope this paper inspires further research into novel architectures to address practical issues in VFL.

We present an efficient zero-knowledge argument of knowledge system customized for the Paillier cryptosystem. Our system enjoys sublinear proof size, low verification cost, and acceptable proof generation effort, while also supporting batch proof generation/verification. Existing works specialized for Paillier cryptosystem feature linear proof size and verification time. Using existing sublinear argument systems for generic statements (e.g., zk-SNARK) results in unaffordable proof generation cost since it involves translating the relations to be proven into an inhibitive large Boolean or arithmetic circuit over a prime order field. Our system does not suffer from these limitations.The core of our argument systems is a constraint system defined over the ring of residue classes modulo a composite number, together with novel techniques tailored for arguing binary values in this setting. We then adapt the approach from Bootle et al. (EUROCRYPT 2016) to compile the constraint system into a sublinear argument system. Our constraint system is generic and can be used to express typical relations in Paillier cryptosystems including range proof, correctness proof, relationships between bits of plaintext, relationships of plaintexts among multiple ciphertexts, and more. Our argument supports batch proof generation and verification, with the amortized cost outperforming state-of-the-art protocol specialized for Paillier when the number of Paillier ciphertext is in the order of hundreds.We report an end-to-end prototype and conduct comprehensive experiments across multiple scenarios. Scenario 1 is Paillier with packing. When we pack 25.6K bits into 400 ciphertexts, a proof that all these ciphertexts are correctly computed is 17 times smaller and is 3 times faster to verify compared with the naive implementation: using 25.6K OR-proofs without packing. Furthermore, we can prove additional statements almost for free, e.g., one can prove that the sum of a subset of the witness bits is less than a threshold t. Another scenario is range proof. To prove that each plaintext in 200 Paillier ciphertexts is of size 256 bits, our proof size is 10 times smaller than the state-of-the-art. Our analysis suggests that our system is asymptotically more efficient than existing protocols, and is highly suitable for scenarios involving a large number (more than 100) of Paillier ciphertexts, which is often the case for data analytics applications.

Softmax and sigmoid, composing exponential functions (ex) and division (1/x), are activation functions often required in training. Secure computation on non-linear, unbounded 1/x and ex is already challenging, let alone their composition. Prior works aim to compute softmax by its exact formula via iteration (CrypTen, NeurIPS ’21) or with ASM approximation (Falcon, PoPETS ’21). They fall short in efficiency and/or accuracy. For sigmoid, existing solutions such as ABY2.0 (Usenix Security ’21) compute it via piecewise functions, incurring logarithmic communication rounds.

Branching program (BP) is a DAG-based non-uniform computational model for L/poly class. It has been widely used in formal verification, logic synthesis, and data analysis. As a special BP, a decision tree is a popular machine learning classifier for its effectiveness and simplicity. In this work, we propose a UC-secure efficient 3-party computation platform for outsourced branching program and/or decision tree evaluation. We construct a constant-round protocol and a linear-round protocol. In particular, the overall (online + offline) communication cost of our linear-round protocol is$O(d(\ell + \log m+\log n))$and its round complexity is$2d-1$, where$m$is the DAG size,$n$is the number of features,$\ell$is the feature length, and$d$is the longest path length. To enable efficient oblivious hopping among the DAG nodes, we propose a lightweight 1-out-of-$N$shared OT protocol with logarithmic communication in both online and offline phase. This partial result may be of independent interest to some other cryptographic protocols. Our benchmark shows, compared with the state-of-the-arts, the proposed constant-round protocol is up to 10X faster in the WAN setting, while the proposed linear-round protocol is up to 15X faster in the LAN setting.

With the pervasiveness of smart phones, location-based services (LBS) have received considerable attention and become more popular and vital recently. However, the use of LBS also poses a potential threat to user's location privacy. In this paper, aiming at spatial range query, a popular LBS providing information about points of interest (POIs) within a given distance, we present an efficient and privacy-preserving location-based query solution, called EPLQ. Specifically, to achieve privacy-preserving spatial range query, we propose the first predicate-only encryption scheme for inner product range (IPRE), which can be used to detect whether a position is within a given circular area in a privacy-preserving way. To reduce query latency, we further design a privacy-preserving tree index structure in EPLQ. Detailed security analysis confirms the security properties of EPLQ. In addition, extensive experiments are conducted, and the results demonstrate that EPLQ is very efficient in privacy-preserving spatial range query over outsourced encrypted data. In particular, for a mobile LBS user using an Android phone, around 0.9 s is needed to generate a query, and it also only requires a commodity workstation, which plays the role of the cloud in our experiments, a few seconds to search POIs.

Association rule mining and frequent itemset mining are two popular and widely studied data analysis techniques for a range of applications. In this paper, we focus on privacy-preserving mining on vertically partitioned databases. In such a scenario, data owners wish to learn the association rules or frequent itemsets from a collective data set and disclose as little information about their (sensitive) raw data as possible to other data owners and third parties. To ensure data privacy, we design an efficient homomorphic encryption scheme and a secure comparison scheme. We then propose a cloud-aided frequent itemset mining solution, which is used to build an association rule mining solution. Our solutions are designed for outsourced databases that allow multiple data owners to efficiently share their data securely without compromising on data privacy. Our solutions leak less information about the raw data than most existing solutions. In comparison to the only known solution achieving a similar privacy level as our proposed solutions, the performance of our proposed solutions is three to five orders of magnitude higher. Based on our experiment findings using different parameters and data sets, we demonstrate that the run time in each of our solutions is only one order higher than that in the best non-privacy-preserving data mining algorithms. Since both data and computing work are outsourced to the cloud servers, the resource consumption at the data owner end is very low.

Event triggered systems are feedback systems that sample the state when the novelty in that state exceeds a threshold. Prior work has demonstrated that event-triggered feedback may have inter-sampling intervals that are, on average, greater than the sampling periods found in comparably performing periodic sampled data systems. This fact has been used to justify the claim that event-triggered systems are more efficient in their use of communication or computational resources than periodic sampled data systems. If, however, one accounts for quantization effects and maximum acceptable delays, then it is quite possible that the actual bit-rates generated by event triggered systems may be greater than that of periodically triggered systems. This paper examines the bit-rates required to asymptotically stabilize nonlinear event triggered systems. An increasing upper bound on the stabilizing bit-rate with respect to the norm of the state is derived. This increasing upper bound on the stabilizing bit-rate reveals the efficient attentiveness property of event triggered systems, i.e. the farther the state is away from the origin, the higher the stabilizing bit-rate will be. Moreover, this paper presents the conditions under which the stabilizing bit-rates asymptotically go to 0.

This paper addresses two research challenges of P2PSIP: i) call setup latency ii) decentralized bootstrapping. To optimize the call setup latency of P2PSIP, we design an O(1) lookup protocol which completes the overlay routing within 3 hops most time. We also leverage the routing table of O(l) DHT to achieve decentralized bootstrapping. Experimental results indicate that 90% of bootstrapping latency is less than 446 ms even if there's no active node within LAN. Analytic and simulation results also demonstrate the feasibility and efficiency of our proposed schemes.

DHT (Distributed Hash Table) algorithms are very efficient for distributed data storage and retrieval. As one kind of P2P overlay, DHT overlay also has the advantages of high reliability, high scalability and low cost. DHT has not only been applied to form user nodes' overlays, but also been proposed to form DHT-based server farms, such as DHT-based SIP server farm, HSS server farm, DNS server farm, CDN server farm, etc. However, seldom DHT algorithms consider server farm's stringent requirement on system capacity, and only a handful of DHT algorithms take server into consideration. This paper presents our DHT algorithm called SFDHT for high throughput DHT server farm. Compared with existing DHTs, SFDHT considers the characters and requirements of DHT server farm and maximizes system capacity. SFDHT is a one-hop DHT with novel built-in load balancing solution. The proposed load balancing solution produces much less overhead that existing solutions do. Both theoretical analysis and simulation results show that SFDHT can reduce overhead, balance load and improve system capacity.

Peer-to-Peer SIP (P2PSIP) is proposed to provide fully distributed multimedia communication systems. This paper focuses on analysis of call setup delay "(CSD) of P2PSIP system to evaluate whether it could provide equivalent signaling performance to that of traditional SIP or telephone networks. Call setup delay is composed of lookup latency in DHT overlay and INVITE transaction latency, and mainly decided by Round Trip Time (RTT) experienced on hops and message exchange procedures of lookup and INVITE transactions. Simulation with active measurement RTT data showed that call setup delay ranges widely from under 100 ms to more than ten seconds, and lookup latency takes up more than 60% of call setup delay. It is concluded that P2PSIP system within limited geographic range could achieve acceptable signaling latency, and efficient latency optimization should be considered to ensure the signaling performance when P2PSIP system is used spread across the public Internet.

Session initiation protocol has been widely deployed to enable multimedia communication. To guarantee service reliability and scalability, service providers usually deploy SIP server farms. In this paper, we present a reliable and scalable DHT-based SIP server farm. In the proposed server farm, one-hop DHT is adopted and a new SIP event package called SIEP (server information event package) is designed. In addition, a novel load sharing and failover scheme is designed based on one-hop DHT and SIEP. Compared with multi-hop DHT, one-hop DHT produces less overhead in DHT-based SIP server farm. Using SIEP, DHT routing hop number can be further reduced to zero in registration and intra-domain call setup process. Performance analysis shows the whole message overhead (overhead caused by SIP and DHT messages) can further reduce about 25% by applying SIEP. It's proven that the proposed load sharing and failover scheme is effective, scalable and reliable even in dynamic environments. What's more, it doesn't increase the deployment and maintenance cost.

SIP (Session Initiation Protocol) is a signaling protocol widely used in multimedia communication. Recently, P2PSIP (Peer-to-Peer SIP), which combines DHT (distributed hash table) and SIP, has been proposed to overcome the drawbacks of traditional CS SIP (client/server architecture SIP). However, the introduction of DHT increases the registration overhead and, session setup overhead/latency. These problems become unbearable when P2PSIP overlay grows huge. To address these problems, we propose locality-aware P2PSIP in this paper. In the P2PSIP context, we design a locality-aware approach, which can be applied to most DHTs. This locality-aware approach reduces both DHT routing hop count and latency per DHT hop. Performance evaluation shows that registration overhead and session setup overhead/latency are reduced dramatically in locality-aware P2PSIP.

P2P SIP is proposed to leverage Peer-to-Peer computing to control multimedia sessions in a decentralized manner. The deployment and maintenance cost of P2PSIP is reduced compared to conventional SIP. In this paper, we propose a hierarchical P2PSIP system to address the connectivity and overhead problems which haven't been solved in the P2PSIP literature. The hierarchical P2PSIP system is implemented under Linux, which demonstrates the feasibility of the proposed scheme. Finally, exhaustive simulations are performed to evaluate the performance of various P2PSIP schemes. Results indicate that the hierarchical approach not only solves the connectivity problem caused by heterogeneous overlays, but also performs more efficiently than the flat scheme when the percentage of nodes in the upper level overlay is less than 10%.