Yu Lin, Ruining Yang, Yunlong Mao, Qizhi Zhang, Jue Hong, Quanwei Cai, Ye Wu, Huiqi Liu, Zhiyu Chen, Bing Duan, Sheng Zhong. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.

Pre-trained Language Models (PLM) have enabled a cost-effective approach to handling various downstream applications via Parameter-Efficient-Fine-Tuning (PEFT) techniques. In this context, service providers have introduced a popular fine-tuning-based product service known as Model-as-a-Service (MaaS). This service offers users access to extensive PLMs and training resources. With MaaS, users can fine-tune, deploy, and utilize their customized models seamlessly, leveraging a one-stop platform that allows them to work with their private datasets efficiently. However, this service paradigm has recently been exposed to the possibility of leaking user private data. To this end, we identify the data privacy leakage risks in MaaS-based PEFT and propose a Split-and-Privatize (SAP) framework, mitigating the privacy leakage by integrating split learning and differential privacy into MaaS PEFT. Furthermore, we propose Contributing-Token-Identification (CTI), a novel method to balance model utility degradation and privacy leakage. As a result, the proposed framework is comprehensively evaluated, demonstrating a 65% improvement in empirical privacy with only a 1% degradation in model performance on the Stanford Sentiment Treebank dataset, outperforming existing state-of-the-art baselines.

With the rapid advancement of the digital economy, data collaboration between organizations has become a well-established business model, driving the growth of various industries. However, privacy concerns make direct data sharing impractical. To address this, Two-Party Split Learning (a.k.a. Vertical Federated Learning (VFL)) has emerged as a promising solution for secure collaborative learning. Despite its advantages, this architecture still suffers from low computational resource utilization and training efficiency. Specifically, its synchronous dependency design increases training latency, while resource and data heterogeneity among participants further hinder efficient computation. To overcome these challenges, we propose \texttt{PubSub-VFL}, a novel VFL paradigm with a Publisher/Subscriber architecture optimized for two-party collaborative learning with high computational efficiency. \texttt{PubSub-VFL} leverages the decoupling capabilities of the Pub/Sub architecture and the data parallelism of the parameter server architecture to design a hierarchical asynchronous mechanism, reducing training latency and improving system efficiency. Additionally, to mitigate the training imbalance caused by resource and data heterogeneity, we formalize an optimization problem based on participants’ system profiles, enabling the selection of optimal hyperparameters while preserving privacy. We conduct a theoretical analysis to demonstrate that \texttt{PubSub-VFL} achieves stable convergence and is compatible with security protocols such as differential privacy. Extensive case studies on five benchmark datasets further validate its effectiveness, showing that \texttt{PubSub-VFL} compared to state-of-the-art baselines not only accelerates training by $2 \sim 7\times$ without compromising accuracy but also achieves computational resource utilization by up to 91.07\%.

Private large language model (LLM) inference based on cryptographic primitives offers a promising path towards privacy-preserving deep learning. However, existing frameworks only support dense LLMs like LLaMA-1 and struggle to scale to mixture-of-experts (MoE) architectures. The key challenge comes from securely evaluating the dynamic routing mechanism in MoE layers, which may reveal sensitive input information if not fully protected. In this paper, we propose CryptoMoE, the first framework that enables private, efficient, and accurate inference for MoE-based models. CryptoMoE balances expert loads to protect expert routing information and proposes novel protocols for secure expert dispatch and combine. CryptoMoE also develops a confidence-aware token selection strategy and a batch matrix multiplication protocol to improve accuracy and efficiency further. Extensive experiments on DeepSeekMoE-16.4B, OLMoE-6.9B, and QWenMoE-14.3B show that CryptoMoE achieves $2.8\sim3.5\times$ end-to-end latency reduction and $3\sim6\times$ communication reduction over a dense baseline with minimum accuracy loss. We also adapt CipherPrune (ICLR'25) for MoE inference and demonstrate CryptoMoE can reduce the communication by up to $4.3 \times$.

Designing an efficient and secure collaborative SQL analysis system that supports large-scale dataset inputs is a very challenging task. In this paper, we present FedQuery, an MPC-based solution for efficient and secure collaborative analysis that is able to handle billion-scale dataset inputs. FedQuery introduces novel designs in its system architecture, the underlying MPC primitives, and oblivious SQL operators as well as their combinations, significantly reducing communication and computation overhead. Comprehensive experiments on real-world datasets show that FedQuery achieves large performance improvements over state-of-the-art baselines at both the operator and query levels. Additionally, it can handle complex SQL queries on datasets up to ten billion entries in less than 14 hours.

With the rapidly-growing deployment of large language model (LLM) inference services, privacy concerns have arisen regarding to the user input data.Recent studies are exploring transforming user inputs to obfuscated embedded vectors, so that the data will not be eavesdropped by service provides.However, in this paper we show that again, without a solid and deliberate security design and analysis, such embedded vector obfuscation failed to protect users' privacy.We demonstrate the conclusion via conducting a novel inversion attack called Element-wise Differential Nearest Neighbor (EDNN) on the glide-reflection proposed in (Mishra et al., 2024), and the result showed that the original user input text can be 100% recovered from the obfuscated embedded vectors.We further analyze security requirements on embedding obfuscation and present several remedies to our proposed attack.

This paper presents a framework to support transparent, live migration of virtual GPU accelerators in a virtualized execution environment. Migration is a critical capability in such environments because it provides support for fault tolerance, on-demand system maintenance, resource management, and load balancing in the mapping of virtual to physical GPUs. Techniques to increase responsiveness and reduce migration overhead are explored. The system is evaluated by using four application kernels and is demonstrated to provide low migration overheads. Through transparent load balancing, our system provides a speedup of 1.7 to 1.9 for three of the four application kernels.

According to the pay-per-use model adopted in clouds, the more the resources consumed by an application running in a cloud computing environment, the greater the amount of money the owner of the corresponding application will be charged. Therefore, applying intelligent solutions to minimize the resource consumption is of great importance. Because centralized solutions are deemed unsuitable for large-distributed systems or large-scale applications, we propose a fully distributed algorithm (called DRA) to overcome the scalability issues. The aforementioned problem can be solved by identifying an assignment scheme between the interacting components of an application, such as processes and virtual machines, and the computing nodes of a cloud system, such that the total amount of resources consumed by the respective application is minimized. The decisions for the transition from one assignment scheme to another one are made in a dynamic way and based only on local information. It should be stressed that DRA achieves convergence and always results in the optimal solution. We also show, through an experimental evaluation, that DRA achieves up to 55% network cost reduction when compared to the most recent algorithm in the literature.

Broadcast is a fundamental operation in wireless ad hoc networks (WANETs). To design efficient broadcast protocols, one of the most important concerns is to reduce broadcast redundancy. In conventional WANETs where nodes are always active, due to the broadcast nature of wireless medium, minimizing broadcast redundancy is equivalent to finding a Minimum Connected Dominating Set (MCDS). However, this is not true for uncoordinated duty-cycled WANETs, where each node periodically switches between active and sleep states, and can only hear messages when it is active. In this paper, we investigate the minimum redundancy broadcast problem in uncoordinated duty-cycled WANETs. We first show that by modifying the conventional CDS-based approaches properly, a constant-approximation broadcast algorithm (MCA) can be obtained. We then propose a hierarchical CDS-based algorithm (HCA), improving the best known approximation ratio from 20 to 13.67. Both algorithms are distributed, and with low time and message complexities. Simulation results show that our algorithms achieve about 5%-30% performance improvement over the state-of-the art scheme.

Interactive wireless applications, like VoIP over wireless networks, desire high-quality links and smooth connectivity during user movement. In order to support seamless roaming in wireless networks, handoff optimization has attracted a lot of attention recently. Most existing approaches aim at reducing handoff latency in communication protocols. While these methods provide significant savings in handoff latency, frequent handoffs could still be crucial and problematic for interactive applications. In this paper, we propose a new perspective for handoff optimization by introducing navigation guidance to minimize the handoff frequency. We first formulate the navigation-driven handoff minimization problem, then propose an optimal algorithm and a localized algorithm to solve it. The optimal algorithm assumes global knowledge of AP locations and uses a navigation graph to find a minimal handoff frequency path. The localized algorithm, however, only uses neighbor AP locations for route selection, which is more practical in real applications. Implementation issues of the proposed algorithms are discussed and simulations based on real world AP deployment are used to evaluate their performance. Experiment results show that our algorithms reduce handoff frequency by at most 42% compared to existing strategies.

Broadcast is a fundamental operation of wireless ad hoc networks (WANET) and has been widely studied in the last decade. However, very few existing broadcasting strategies has considered the scenarios with sleeping schedule, which is a prevalent power-saving method in wireless networks. In this paper we study the sleeping schedule-aware minimum latency broadcast (MLB-SA) problem in WANETs and prove its NP-hardness. By constructing a shortest path tree (SPT) defined with the latency function on the network, we derive a lower bound on the broadcast latency theoretically. Following the top-down layered approach and using the D2-coloring solution, we proposed two progressively improved algorithms: the simple layered coloring algorithm (SLAC) and the enhanced layered coloring algorithm (ELAC) for the MLB-SA problem. The SLAC has an approximation ratio of O(Delta2+ 1) where Delta is maximum degree of the network, while the ELAC has constant approximation ratio of 24|T| + 1 where |T| is the number of timeslots in a scheduling period. The two algorithms have O(n2) and O(n3) time complexities respectively. The performance of the proposed algorithms are evaluated by simulations.

Sleep scheduling protocols are widely used in wireless sensor networks for saving energy in sensor nodes. However, without considering the special requirements of object tracking, conventional sleep scheduling protocols may lead to intolerable degradation of tracking qualities when they are used in object tracking applications. To handle this problem, sleep scheduling protocols tailed for object tracking have been proposed recently. For saving energy while maintaining satisfactory tracking qualities, these protocols pro-actively awaken sensors according to the prediction of objects' movement. Such sleep scheduling protocols are called the prediction-based sleep scheduling protocols. Most existing prediction-based sleep scheduling protocols require sensor nodes to know the locations of themselves, which may not always be available. In this paper we propose a Location-free Prediction-based Sleep Scheduling protocol (LPSS) for object tracking in sensor networks. LPSS guarantees the coverage level, an important tracking quality in most applications, which is defined as the number of sensors simultaneously detecting the object. In LPSS, when a sensor detects the object, it will emit a signal, namely the sensing stimulus. Sensors decide to wake up or not based on only the received sensing stimulus, the prediction models and the required coverage level, without the requirement of location information. We implement LPSS with two most popular prediction models: the Circle-based and the Probability-based prediction models. Experiment results show that LPSS not only provides qualified coverage levels, but also saves about 40% to 70% energy compared with existing location-free protocols. Moreover, the energy cost of LPSS is close to the ideal approach using accurate location information in terms of the number of awakened nodes.

Sensors' locations are important to many wireless sensor networks (WSNs). When WSNs are deployed in hostile environments, two issues about sensors' locations need to be considered. First, the attackers may attack the localization process to make the estimated locations incorrect. Second, since sensor nodes may be compromised, the base station may not trust the locations reported by sensor nodes. Researchers have proposed two techniques, secure localization and location verification, to solve the two issues respectively. In this paper we describe the attacks against localization and location verification, and survey the state of research of both secure localization and location verification.

Recently with the emergence of mobile sensor networks, localization for such networks has gained much attention, and many localization algorithms have been proposed. Among them the Sequential Monte Carlo (SMC) based algorithms are very popular because of their simplicity and efficiency. However, most current SMC-based localization algorithms implicitly assume there is no attacker in the network, which may not be true in real applications. The attackers, if any, may send false information by themselves or through compromised nodes to disturb the localization. In this paper, we present the design and evaluation of a Secure Monte Carlo Localization algorithm, SecMCL. SecMCL provides authentication to messages and employs a new sampling method to defeat attacks. Simulation results show that SecMCL greatly improves the localization accuracy of existing SMC-based localization method when there are attacks. Also, compared with existing SMC localization method, SecMCL incurs no communication cost (in terms of number of messages) and achieves the same localization accuracy when there is no attack.

Self-organization is the key to implement self-calibration, autonomously coordination and P2P communication in sensor networks. Current researches mainly solve this problem in an inappropriate pre-planned manner. The swarm intelligence of the ant colony algorithm (ACA) provides a novel and efficient method for self-organization. For the similarity of sensor network and ant colony, we argue that the sensor networks will benefit from the bio-inspired self-organization by applying the ACA in optimization, structure formation and task/resource allocation. In this paper we outline the current researches on the ACA in sensor network first, then propose the potential applications and research issues. General design of the ACA in sensor network and some challenges are presented as well.

As a fundamental operation of wireless ad hoc networks (WANET), broadcast has been widely studied in the past ten years. However, most existing broadcasting strategies assumed non-sleeping wireless devices. Little attention has been paid to broadcast in WANETs with sleeping schedule, which is a promising power-saving method in wireless networks. In this paper we study the sleeping schedule aware minimum transmission broadcast problem in WANETs (MTB-SA problem) and prove its NP-hardness. Both centralized and distributed approximation algorithms are presented to solve the problem. The centralized algorithm SchmM-Cent has an approximation ratio of 3(ln¿+1) and time complexity of O(n^3). The distributed algorithm SchmM-Dist has a constant approximation ratio of at most 20, while time and message complexity are both O(n). In addition, we provide theoretical analysis and simulations to evaluate the performance of the approximation algorithms.