Yu Lin, Ruining Yang, Yunlong Mao, Qizhi Zhang, Jue Hong, Quanwei Cai, Ye Wu, Huiqi Liu, Zhiyu Chen, Bing Duan, Sheng Zhong. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.

Low-precision training has emerged as a powerful technique for reducing computational and storage costs in Deep Neural Network (DNN) model training, enabling on-chip training or fine-tuning on edge devices. However, existing low-precision training methods often require higher bit-widths to maintain accuracy as model sizes increase. In this paper, we introduce an outlier-aware quantization strategy for low-precision training. While traditional value-aware quantization methods require costly online distribution statistics operations on computational data, impeding the efficiency gains of low-precision training, our approach addresses this challenge through a novel Learnable Threshold based Outlier-Aware Quantization (LT-OAQ) training framework. This method concurrently updates outlier thresholds and model weights through gradient descent, eliminating the need for costly data-statistics operations. To efficiently support the LT-OAQ training framework, we designed a hardware accelerator based on the systolic array architecture. This accelerator introduces a processing element (PE) fusion mechanism that dynamically fuses adjacent PEs into clusters to support outlier computations, optimizing the mapping of outlier computation tasks, enabling mixed-precision training, and implementing online quantization. Our approach maintains model accuracy while significantly reducing computational complexity and storage resource requirements. Experimental results demonstrate that our design achieves a 2.9 ×speedup in performance and a 2.17 ×reduction in energy consumption compared to state-of-the-art low-precision accelerators.

Pre-trained Language Models (PLM) have enabled a cost-effective approach to handling various downstream applications via Parameter-Efficient-Fine-Tuning (PEFT) techniques. In this context, service providers have introduced a popular fine-tuning-based product service known as Model-as-a-Service (MaaS). This service offers users access to extensive PLMs and training resources. With MaaS, users can fine-tune, deploy, and utilize their customized models seamlessly, leveraging a one-stop platform that allows them to work with their private datasets efficiently. However, this service paradigm has recently been exposed to the possibility of leaking user private data. To this end, we identify the data privacy leakage risks in MaaS-based PEFT and propose a Split-and-Privatize (SAP) framework, mitigating the privacy leakage by integrating split learning and differential privacy into MaaS PEFT. Furthermore, we propose Contributing-Token-Identification (CTI), a novel method to balance model utility degradation and privacy leakage. As a result, the proposed framework is comprehensively evaluated, demonstrating a 65% improvement in empirical privacy with only a 1% degradation in model performance on the Stanford Sentiment Treebank dataset, outperforming existing state-of-the-art baselines.

With the development of cryptographic tools such as Fully Homomorphic Encryption (FHE) and secure Multiparty Computation (MPC), privacy-preserving Machine Learning as a Service (MLaaS) has gained attractiveness for its security when it comes to utilizing cross-domain data. However, cryptographic tools are characterized by huge overhead, which results in the MLaaS quality being unbearably degraded, especially for latency-sensitive MLaaS applications. In this paper, we focus on the problem of low-latency inference associated with MLaaS and propose CrossNet, a Privacy-preserving Neural Network Inference (PPNI) framework based on FHE, for applications with limited client-side computational and communication resources. CrossNet performs model transformations on neural networks so that they can be evaluated in an FHE-friendly manner. Model transformation introduces limited interactions between client and server, thus restricting inference latency. In addition, CrossNet includes a series of layer constructions where elaborate encoding forms and computational orders are designed to further reduce the overhead of transformed layers. CrossNet outperforms the existing FHE-based frameworks by 4x efficiency and reduces nearly 30% inference latency on ResNet-50 in a resource-limited setting.

Conventional crowdsourcing platforms primarily rely on a central server as the broker for information exchange. Although many efforts have been made, centralized platforms are still vulnerable to underlying security issues, such as an untrusted central server and single-point failure. Fortunately, blockchain has emerged as an alternative infrastructure for building crowdsourcing platforms. Many excellent designs of blockchain-based decentralized crowdsourcing (BDCS) solutions have been proposed. Benefiting from blockchain, BDCS can provide fascinating features, like tampering resistance and anonymity. However, a new attack surface appears in BDCS. Recently, a new attack against BDCS named solution probing attack has been identified. The solution-probing adversary can take advantage of the anonymity of BDCS to probe valid solutions using a generative model. Due to the transparency of blockchain transactions, the probing attack is effective even if solutions are encrypted. Nevertheless, we find transaction-mixing techniques effective in defending against probing attacks. In this paper, we introduce the solution probing attack and an improved variant, which can attack coin mixing-based BDCS. We evaluate probing attacks on large-scale crowdsourcing tasks. Experimental results show that the adversary is capable of deceiving BDCS with a limited number of probing, even if the BDCS is protected by solution encryption and coin mixing techniques.

Federated learning (FL) is a promising approach for participants’ collaborative learning tasks with cross-silo data. Participants benefit from FL since heterogeneous data can contribute to the generalization of the global model while keeping private data locally. However, practical issues of FL, such as security and fairness, keep emerging, impeding its further development. One of the most threatening security issues is the poisoning attack, corrupting the global model by an adversary’s will. Recent studies have demonstrated that elaborate model poisoning attacks can breach the existing Byzantine-robust FL solutions. Although various defenses have been proposed to mitigate poisoning attacks, participants will sacrifice learning performance and fairness due to strict regulations. Considering that the importance of fairness is no less than security, it is crucial to explore alternative solutions that can secure FL while ensuring both robustness and fairness. This paper introduces a robust and fair model aggregation solution, Romoa-AFL, for cross-silo FL in an agnostic data setting. Unlike a previous study named Romoa and other similarity-based solutions, Romoa-AFL ensures robustness against poisoning attacks and learning fairness in agnostic FL, which has no assumptions of participants’ data distributions and the server’s auxiliary dataset.

Adversarial examples that can fool neural network classifiers have attracted much attention. Existing approaches to detect adversarial examples leverage a supervised scheme in generating attacks (either targeted or non-targeted) for training the detectors, which means the detectors are geared to the attacks chosen at the training time and could be circumvented if the adversary does not act as expected. In this paper, we borrow ideas from cryptography and present a novel approach called pseudorandom classifier. In a nutshell, a pseudorandom classifier is a classifier equipped with a mapping to encode the category labels into random multi-bit labels, and a keyed pseudorandom injective function to transform the input to the classifier. The multi-bit labels enable attack-independent and probabilistic detection if the input sample is adversarial. The pseudorandom injection makes the existing white-box adversarial example generation methods, largely based on back-propagation, no longer applicable. We empirically evaluate our method on MNIST, CIFAR10, Imagenette, CIFAR100, and GTSRB. The results suggest that its performance against adversarial examples is comparable to the state-of-the-art.

Many mobile apps access users’ trajectories to provide critical services (e.g., trip tracking). Unfortunately, in such apps, malicious users may upload fake trajectories to cheat providers for illegal benefits. There are few works in the literature that delicately study trajectory forgery problems. In this paper, we first take the perspective of attackers and consider how they fabricate vivid trajectories confronting a strict provider. In particular, we use the technique of adversarial examples in deep learning to propose a trajectory forgery method, which produces fake trajectories satisfying two conditions: (1) having the motion characteristics indistinguishable from those of real ones, and (2) matching a reasonable walking, cycling, or driving route when being projected to the map. We show through experiments that they can hardly be detected by mainstream trajectory service providers, even after being equipped with machine learning-based approaches. Therefore, we further present a dedicated countermeasure by validating the reasonability of reported received signal strength indicator (RSSI) data of WiFi access points (APs) nearby every location. It can well deal with the most challenging replay scenario, which can hardly be handled by existing WiFi-based location verification methods. We conduct extensive real-world experiments in three local commercial areas covering walking, cycling, and driving scenarios. Results demonstrate the high detection accuracy of this method.

Vast data and computing resources are commonly needed to train deep neural networks, causing an unaffordable price for individual users. Motivated by the increasing demands of deep learning applications, sharing well-trained models becomes popular. The owner of a pre-trained model can share it by publishing the model directly or providing a prediction interface. Either way, individual users can benefit from deep learning without much cost, and computing resources can be saved. However, recent studies of machine learning security have identified severe threats to these model publishing approaches. This paper will focus on the privacy leakage issue of publishing well-trained deep neural network models. To tackle this problem, we propose a series of secure model publishing solutions based on training task parallelism. Specifically, we show how to estimate private model parameters through parallel model training and generate new model parameters in a privacy-preserving manner to replace the original ones for publishing. Based on data parallelism and parameter generating techniques, we design another two solutions concentrating on model quality and parameter privacy, respectively. Through privacy leakage analysis and experimental attack evaluation, we conclude that deep neural network models published with our solutions can provide on-demand model quality guarantees and resist membership inference attacks.

As one of a few critical technologies to cloud storage service, deduplication allows cloud servers to save storage space by deleting redundant file copies. However, it often leaks side channel information regarding whether an uploading file gets deduplicated or not. Exploiting this information, adversaries can easily launch a template side-channel attack and severely harm cloud users' privacy. To thwart this kind of attack, we resort to the k-anonymity privacy concept to design secure threshold deduplication protocols. Specifically, we have devised a novel cryptographic primitive called “dispersed convergent encryption” (DCE) scheme, and proposed two different constructions of it. With these DCE schemes, we successfully construct secure threshold deduplication protocols that do not rely on any trusted third party. Our protocols not only support confidentiality protections and ownership verifications, but also enjoy formal security guarantee against template side-channel attacks even when the cloud server could be a “covert adversary” who may violate the predefined threshold and perform deduplication covertly. Experimental evaluations show our protocols enjoy very good performance in practice.

Deep neural networks (DNNs) have brought significant performance improvements to various real-life applications. However, a DNN training task commonly requires intensive computing resources and a huge data collection, which makes it hard for personal devices to carry out the entire training, especially for mobile devices. The federated learning concept has eased this situation. However, it is still an open problem for individuals to train their own DNN models at an affordable price. In this article, we propose an alternative DNN training strategy for resource-limited users. With the help of an untrusted server, end users can offload their DNN training tasks to the server in a privacy-preserving manner. To this end, we study the possibility of the separation of a DNN. Then we design a differentially private activation algorithm for end users to ensure the privacy of the offloading after model separation. Furthermore, to meet the rising demand for federated learning, we extend the offloading solution to parallel DNN models training with a secure model weights aggregation scheme for the privacy concern. Experimental results prove the feasibility of computation offloading solutions for DNN models in both solo and parallel modes.

Machine learning as a service has emerged recently to relieve tensions between heavy deep learning tasks and increasing application demands. A deep learning service provider could help its clients to benefit from deep learning techniques at an affordable price instead of huge resource consumption. However, the service provider may have serious concerns about model privacy when a deep neural network model is published. Previous model publishing solutions mainly depend on additional artificial noise. By adding elaborated noises to parameters or gradients during the training phase, strong privacy guarantees like differential privacy could be achieved. However, this kind of approach cannot give guarantees on some other aspects, such as the quality of the disturbingly trained model and the convergence of the modified learning algorithm. In this paper, we propose an alternative private deep neural network model publishing solution, which caused no interference in the original training phase. We provide privacy, convergence and quality guarantees for the published model at the same time. Furthermore, our solution can achieve a smaller privacy budget when compared with artificial noise based training solutions proposed in previous works. Specifically, our solution gives an acceptable test accuracy with privacy budget ϵ = 1. Meanwhile, membership inference attack accuracy will be deceased from nearly 90% to around 60% across all classes.

Multi-User MIMO (MU-MIMO) has attracted much attention due to its significant advantage of increasing the utilization ratio of wireless channels. However, Frequency-Division Duplex (FDD) systems are vulnerable to eavesdropping, since the explicit CSI feedback can be manipulated. In this paper, we show that Time-Division Duplex (TDD) systems are insecure as well. In particular, we show that it is possible to eavesdrop on other users' downloads by tuning training sequences. In order to defend MU-MIMO against such threats, we propose a secure CSI estimation scheme, which can provide correct estimates of CSI when adversarial users are in presence. We prove that our scheme is secure against training sequence based eavesdropping attack. We have implemented our scheme for TDD MU-MIMO systems and performed a series of experiments. Results demonstrate that our secure CSI estimation scheme is highly effective in protecting TDD MIMO networks against eavesdropping attack. Furthermore, we extend our scheme to support massive MU-MIMO networks, with a carefully redesigned uplink protocol and optimized power allocation to achieve higher spectral efficiency. To be more practical, we also take mismatch channel issue into our consideration. An enhancement scheme is proposed and we show that our scheme with enhancement is secure and correct under mismatch channel.

Collaborative spectrum sensing has become increasingly popular in cognitive radio networks to enable unlicensed secondary users to coexist with the licensed primary users and share spectrum without interference. Despite its promise in performance enhancement, collaborative sensing is still facing a lot of security challenges. The problem of revealing secondary users' location information through sensing reports has been reported recently. Unlike any existing work, in this paper we not only address the location privacy issue in the collaborative sensing to be against semi-honest adversaries, but also take malicious adversaries into consideration. We propose efficient schemes to protect secondary users' reports from being revealed in the aggregation process at the fusion center. We rigorously prove that our privacy-preserving collaborative sensing schemes are secure against attacks from both the fusion center and secondary users. We also evaluate our schemes extensively and verify its efficiency and feasibility.

Multi-User MIMO has attracted much attention due to its significant advantage of increasing the utilization ratio of wireless channels. Recently a serious eavesdropping attack, which exploits the CSI feedback of the FDD system, is discovered in MU-MIMO networks. In this paper, we firstly show a similar eavesdropping attack for the TDD system is also possible by proposing a novel, feasible attack approach. Following it, a malicious user can eavesdrop on other users' downloads by transforming training sequences. To prevent this attack, we propose a secure CSI estimation scheme for instantaneous CSI. Furthermore, we extend this scheme to achieve adaptive security when CSI is relatively statistical. We have implemented our scheme for both uplink and downlink of MU-MIMO and performed a series of experiments. Results show that our secure CSI estimation scheme is highly effective in preventing downlink leakage against malicious users.

Being an important application of spectrum sharing in cellular networks, mobile traffic offloading, which advocates third-party owners of network resource on unlicensed/licensed spectrum to share their spectrum and provide data offloading services, is considered a promising solution to severe spectrum shortage faced by cellular network service providers. In this paper, we consider a general mobile traffic offloading system that adopts the widely used Gale-Shapley algorithm to optimize its mobile phone users (MUs) to offloading stations allocation plan. We notice that without careful protection, such a system could cause serious threat to MUs' location privacy, and thus design effective countermeasures based on the powerful state-of-the-art differential privacy concept. Specifically, we have proposed two joint differentially private Gale-Shapley mechanisms with strong privacy protections for mobile traffic offloading systems. The first mechanism is able to protect each user's location privacy even when all other users collude against this user assuming the system administrator can be trusted. The second mechanism is able to achieve the same privacy guarantee against colluding users, and moreover against an untrusted semi-honest system administrator. We perform extensive experiments to evaluate our mechanisms, and the results show that our mechanisms have good efficiency, accuracy, and privacy protection.

To put mobile sensing into large-scale deployments, we have to take care of sensing participants' incentives and privacy first. In this paper, we study how to protect the sensing participants' privacy in the mobile sensing market where multiple sensing jobs reside in one consolidated place. Our problem is highly challenging due to the facts that incentives are introduced and we consider both the sensing job owner and the market administrator could invade the sensing participants' privacy. We propose two privacy-preserving market mechanisms that are able to protect the sensing participants' privacy to solve our problem. Experiments also demonstrate that our mechanisms have good efficiency.

Collaborative sensing has become increasingly popular in cognitive radio networks to enable unlicensed secondary users to coexist with the licensed primary users and share spectrum without interference. Despite its promise in performance enhancement, collaborative sensing is still facing a lot of security challenges. The problem of revealing secondary users' location information through sensing reports has been reported recently. Unlike any existing work, in this paper we not only address the location privacy issues in the collaborative sensing process against semi-honest adversaries, but also take the malicious adversaries into consideration. We propose efficient schemes to protect secondary users' report from being revealed in the report aggregation process at the fusion center. We rigorously prove that our privacy-preserving collaborative sensing schemes are secure against the fusion center and the secondary users in semi-honest model. We also evaluate our scheme extensively and verify its efficiency.

Wireless security has been an active research area since the last decade. A lot of studies of wireless security use cryptographic tools, but traditional cryptographic tools are normally based on computational assumptions, which may turn out to be invalid in the future. Consequently, it is very desirable to build cryptographic tools that do not rely on computational assumptions. In this paper, we focus on a crucial cryptographic tool, namely 1-out-of-2 oblivious transfer. This tool plays a central role in cryptography because we can build a cryptographic protocol for any polynomial-time computable function using this tool. We present a novel 1-out-of-2 oblivious transfer protocol based on wireless channel characteristics, which does not rely on any computational assumption. We also illustrate the potential broad applications of this protocol by giving two applications, one on private communications and the other on privacy preserving password verification. We have fully implemented this protocol on wireless devices and conducted experiments in real environments to evaluate the protocol. Our experimental results demonstrate that it has reasonable efficiency.