Rui Yin, Tianxu Han, Naen Xu, Changjiang Li, Ping He, Chunyi Zhou, Jun Wang, Zhihui Fu, Tianyu Du, Jinbao Li, Shouling Ji. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

Traditional sequential recommendation (SR) models learn low-dimensional item ID embeddings from user-item interactions, often overlooking textual information such as item titles or descriptions. Recent advances in Large Language Models (LLMs) have inspired a surge of research that encodes item textual information with high-dimensional semantic embeddings, and designs transformation methods to inject such embeddings into SR models. These embedding transformation strategies can be categorized into two types, both of which exhibits notable drawbacks: 1) adapter-based methods suffer from pronounced dimension collapse, concentrating information into a few dominant dimensions; 2) SVD-based methods are rigid and manual, considering only a few principal spectral components while discarding rich information in the remaining spectrum.

The emergence of generative models enables the creation of texts and images tailored to users' preferences. Existing personalized generative models have two critical limitations: lacking a dedicated paradigm for accurate preference modeling, and generating unimodal content despite real-world multimodal-driven user interactions. Therefore, we propose personalized multimodal generation, which captures modal-specific preferences via a dedicated preference model from multimodal interactions, and then feeds them into downstream generators for personalized multimodal content. However, this task presents two challenges: (1) Gap between continuous preferences from dedicated modeling and discrete token inputs intrinsic to generator architectures; (2) Potential inconsistency between generated images and texts. To tackle these, we present a two-stage framework called Discrete Preference learning for Personalized Multimodal Generation (DPPMG). In the first stage, to accurately learn discrete modal-specific preferences, we introduce a modal-specific graph neural network (a dedicated preference model) to learn users' modal-specific preferences, which preferences are then quantized into discrete preference tokens. In the second stage, the discrete modal-specific preference tokens are injected into downstream text and image generators. To further enhance cross-modal consistency while preserving personalization, we design a cross-modal consistent and personalized reward to fine-tune token-associated parameters. Extensive experiments on two real-world datasets demonstrate the effectiveness of our model in generating personalized and consistent multimodal content.

Click-through rate (CTR) prediction is a fundamental task in modern recommender systems. In recent years, the integration of large language models (LLMs) has been shown to effectively enhance the performance of traditional CTR methods. However, existing LLM-enhanced methods often require extensive processing of detailed textual descriptions for large-scale instances or user/item entities, leading to substantial computational overhead. To address this challenge, this work introduces LLaCTR, a novel and lightweight LLM-enhanced CTR method that employs a field-level enhancement paradigm. Specifically, LLaCTR first utilizes LLMs to distill crucial and lightweight semantic knowledge from small-scale feature fields through self-supervised field-feature fine-tuning. Subsequently, it leverages this field-level semantic knowledge to enhance both feature representation and feature interactions. In our experiments, we integrate LLaCTR with six representative CTR models across four datasets, demonstrating its superior performance in terms of both effectiveness and efficiency compared to existing LLM-enhanced methods. Our code is available at https://github.com/istarryn/LLaCTR.

The rapid advancement of multimodal large language models has enabled agents to operate mobile devices by directly interacting with graphical user interfaces, opening new possibilities for mobile automation. However, real-world mobile tasks are often complex and allow for multiple valid solutions. This contradicts current mobile agent evaluation standards: offline static benchmarks can only validate a single predefined ''golden path'', while online dynamic testing is constrained by the complexity and non-reproducibility of real devices, making both approaches inadequate for comprehensively assessing agent capabilities. To bridge the gap between offline and online evaluation and enhance testing stability, this paper introduces a novel graph-structured benchmarking framework. By modeling the finite states observed during real-device interactions, it achieves static simulation of dynamic behaviors. Building on this, we develop ColorBench, a benchmark focused on complex long-horizon tasks. It supports evaluation of multiple valid solutions, subtask completion rate statistics, and atomic-level capability analysis. ColorBench contains 175 tasks (74 single-app, 101 cross-app) with an average length of over 13 steps. Each task includes at least two correct paths and several typical error paths, enabling quasi-dynamic interaction.

Federated recommender systems enable collaborative model training while keeping user interaction data local and sharing only essential model parameters, thereby mitigating privacy risks. However, existing methods overlook a critical issue, i.e., the stable learning of a generalized item embedding throughout the federated recommender system training process. Item embedding plays a central role in facilitating knowledge sharing across clients. Yet, under the cross-device setting, local data distributions exhibit significant heterogeneity and sparsity, exacerbating the difficulty of learning generalized embeddings. These factors make the stable learning of generalized item embeddings both indispensable for effective federated recommendation and inherently difficult to achieve. To fill this gap, we propose a new federated recommendation framework, named Federated Recommendation with Generalized Embedding Learning (FedRecGEL). We reformulate the federated recommendation problem from an item-centered perspective and cast it as a multi-task learning problem, aiming to learn generalized embeddings throughout the training procedure. Based on theoretical analysis, we employ sharpness-aware minimization to address the generalization problem, thereby stabilizing the training process and enhancing recommendation performance. Extensive experiments on four datasets demonstrate the effectiveness of FedRecGEL in significantly improving federated recommendation performance. Our code is available at https://github.com/anonymifish/FedRecGEL.

With the rapidly growing demand for collaborative data analysis, Jaccard Coefficient (JC) computation over multisets has been widely adopted in data deduplication to enhance large-scale data processing efficiency, but meanwhile incurs some security issues such as the leakage of input data sets. Thus, Secure Jaccard Coefficient Computation over Multisets (SCJM) schemes have been proposed. However, existing solutions that indirectly compute JC fail to protect the privacy of intersection and union cardinalities, incur high computational overhead, and rely on approximation techniques that cannot support high-precision analysis or simulation-based security proofs. To address these problems, we propose a secure and efficient protocol to accurately compute JC over multisets. Specifically, the protocol computes JC for small-scale data domains using secure oblivious ratio computation, ensuring that under the semi-honest adversarial model, the intersection and union cardinalities remain concealed from all parties, including the decryption key holder, during and after the computation. And it achieves a linear computational cost ofO(em-1) without accuracy loss, whereem-1denotes the maximum number of repetitions of an element in the multiset. The protocol can also be extended to compute the cardinalities of intersection and union. To further enhance efficiency, we introduce a cloud-assisted encryption scheme, which improves computational efficiency by 25.5% to 30.4% compared with the non-cloud-assisted scheme. Additionally, we provide a secure proof of the proposed protocol in the ideal-real paradigm. Experimental results show efficiency advantages of our protocol over the state-of-the-art solution.

Sequential recommenders generate recommendations based on users’ historical interaction sequences. However, in practice, these sequences are often contaminated by noisy interactions, which can arise from various factors such as clickbait, the influence of prominently positioned items, or accidental interactions. Such noise can significantly degrade recommendation performance. Accurately identifying such noisy interactions without additional information is particularly challenging due to the absence of explicit supervisory signals indicating noise. Large Language Models (LLMs), equipped with extensive open knowledge and semantic reasoning abilities, offer a promising avenue to bridge this information gap. However, employing LLMs for denoising in sequential recommendation presents notable challenges: (1) Direct application of pretrained LLMs may not be competent for the denoising task, frequently generating nonsensical responses; (2) Fine-tuning on the denoising task can partially mitigate the issue of generating nonsensical outputs. However, even after fine-tuning, the reliability of LLM outputs remains questionable, especially given the complexity of the denoising task and the inherent hallucination issue of LLMs. To tackle these challenges, we propose LLM4DSR, a tailored approach for denoising sequential recommendation using LLMs. We constructed a self-supervised fine-tuning task to activate LLMs’ capabilities to identify noisy items and suggest replacements. Furthermore, we developed an uncertainty estimation module that ensures only high-confidence responses are utilized for sequence corrections. Remarkably, LLM4DSR is model-agnostic, allowing corrected sequences to be flexibly applied across various recommendation models. To the best of our knowledge, this is the first work that employs LLMs for sequential recommendation denoising while addressing the unique challenges of adapting LLMs to this task. Extensive experiments conducted on three real-world datasets across two noise settings validate the effectiveness of LLM4DSR, demonstrating an average improvement of 12.9% in NDCG@20. The code is available at https://github.com/WANGBohaO-jpg/LLM4DSR .

Model extraction attack shows promising performance in revealing sequential recommendation (SeqRec) robustness, e.g., as an upstream task of transfer-based attack to provide optimization feedback for downstream attacks. However, existing work either heavily relies on impractical prior knowledge or has impressive attack performance. In this paper, we focus on data-free model extraction attack on SeqRec, which aims to efficiently train a surrogate model that closely imitates the target model in a practical setting. Conducting such an attack is challenging. First, imitating sequential training data for accurate model extraction is hard without prior knowledge. Second, limited queries for the target model require the attack to be efficient. To address these challenges, we propose a novel adversarial framework Sim4Rec which includes two modules, i.e., controllable sequence generation and reinforced adversarial distillation. The former allows a sequential generator to produce synthetic data similar to training data through pre-training with controllable generated samples. The latter efficiently extracts the target model via reinforced adversarial knowledge distillation. Extensive experiments demonstrate the advancement of Sim4Rec.

Federated graph learning (FGL) has emerged as a promising approach to enable collaborative training of graph models while preserving data privacy. However, current FGL methods overlook the out-of-distribution (OOD) shifts that occur in real-world scenarios. The distribution shifts between training and testing datasets in each client impact the FGL performance. To address this issue, we propose federated graph OOD generalization framework FedGOG, which includes two modules, i.e., diffusion data exploration (DDE) and latent embedding decorrelation (LED). In DDE, all clients jointly train score models to accurately estimate the global graph data distribution and sufficiently explore sample space using score-based graph diffusion with conditional generation. In LED, each client models a global invariant GNN and a personalized spurious GNN. LED aims to decorrelate spuriousness from invariant relationships by minimizing the mutual information between two categories of latent embeddings from different GNN models. Extensive experiments on six benchmark datasets demonstrate the superiority of FedGOG.

Personalized recommender systems are critical for enhancing user engagement across a range of digital platforms. However, conventional approaches rely heavily on centralized data collection, raising significant privacy concerns. Federated recommender systems (PFRS) address these concerns by decentralizing model training, ensuring user data privacy. Despite the progress, existing methods still struggle with capturing the multi-faceted nature of user and transferring global knowledge effectively. In this work, we propose FedMUR, a novel federated recommendation framework that models user representation as a Gaussian mixture distribution, capturing users' multi-faceted characteristics. Each Gaussian component corresponds to a distinct interest facet, with adaptive mixture weights representing the user's preference intensity toward each facet. To facilitate knowledge transfer, FedMUR constructs global consistent prototypes that encode shared behavioral trends across users via popularity-weighted optimal transport. These prototypes enhance local models by injecting global shared patterns into personalized representation learning. Extensive experiments across several real-world datasets demonstrate that FedMUR significantly outperforms existing state-of-the-art federated recommendation systems.

Fine-tuning large language models (LLMs) is challenged by the presence of noisy data and the high computational cost when training on large-scale datasets. While data selection has emerged as a promising approach to reduce training cost and improve data quality, existing methods often rely on static heuristics or manual metrics. These approaches struggle to adapt to the model’s evolving capabilities during training, as its understanding of tasks improves. As the model becomes more powerful, its requirements for data that can enhance performance also change, making it crucial to incorporate this dynamic into the data selection process. Moreover, ensuring data diversity throughout different stages of training is essential for preventing redundancy, reducing overfitting. To address these issues, we propose DSP, a Diversity-Aware Self-Paced data selection framework that evolves with the model. DSP progressively selects training samples based on the model’s own outputs and incorporates a diversity-aware mechanism to enhance generalization and mitigate overfitting. Unlike prior static or rule-based strategies, DSP adaptively adjusts to the model’s internal feedback and training stage. Experiments on two public benchmarks demonstrate that DSP consistently outperforms static and heuristic-based baselines across multiple datasets and backbone models. Our findings highlight the critical role of dynamic, diversity-aware data selection in effective LLM fine-tuning.

Industrial recommender systems usually train models incrementally to grasp recent interests of users. However, a fundamental issue of these incremental updated models is their tendency to overfit current data while neglecting past information. Specifically, we have observed that the data distribution of real systems exhibits periodic drifts, leading to periodic fluctuations of prediction bias. To alleviate the above bias fluctuations while minimizing the loss of recent interests, we propose TPIA, a Training-free approach for Periodic Interest Augmentation in incremental recommendation. Specifically, after the latest model is trained, we first calculate the importance score of each model in the previous period. Then, we merge these models based on the importance scores. To minimize information loss due to interference of parameters during model merging, we further develop a method for trimming redundant and abnormal parameters. Offline experiments on both public and private datasets demonstrate the effectiveness of TPIA. It has also been deployed on a large-scale industrial recommender system, and has shown a notable 1.61% increase in CVR and a 1.97% increase in CPM, along with enhanced stability in prediction bias.

Large language models (LLMs), known for their comprehension capabilities and extensive knowledge, have been increasingly applied to recommendation systems (RS).Given the fundamental gap between the mechanism of LLMs and the requirement of RS, researchers have focused on fine-tuning LLMs with recommendationspecific data to enhance their performance.Language Modeling Loss (LML), originally designed for language generation tasks, is commonly adopted.However, we identify two critical limitations of LML: 1) it exhibits significant divergence from the recommendation objective; 2) it erroneously treats all fictitious item descriptions as negative samples, introducing misleading training signals.To address these limitations, we propose a novel Masked Softmax Loss (MSL) tailored for fine-tuning LLMs on recommendation.MSL improves LML by identifying and masking invalid tokens that could lead to fictitious item descriptions during loss computation.This strategy can effectively avoid the interference from erroneous

Federated Recommendation (FedRec) enables joint training across a large number of clients without centralizing user interaction data. However, existing FedRec methods overlook two key challenges, i.e. (1) sufficiently explore the global item embedding space, and (2) effectively achieve local and global collaboration. The former is caused by client sparsity, which leads to suboptimal item embeddings and subsequently impacts the global item embedding in both the dimension and sample views. The latter arises from the lack of modeling the relative importance of local and global contributions to personalized user preferences. To address the above challenges, we propose FedIAR which contains two modules, i.e., item embedding dual-view exploration and adaptive local-global fusion. The first module enhances the global item embedding by reducing redundancy in the dimension view and capturing latent item relationships in the sample view, improving representational capacity. The second module enables the adaptive fusion of local and global item embeddings based on the user preference representation, achieving personalized optimum for recommendation. Extensive experiments on six datasets demonstrate the effectiveness of FedIAR in improving federated recommendation performance.

Customer lifetime value (LTV) is crucial to companies who are intending to adopt personalized promoting strategies to optimize the profits. However, LTV prediction in the scenario of online App advertising usually suffers from label sparsity issue, towards which existing methods designed complex model structures but ignored the information contained in intermediate user behaviors. Moreover, previous works mainly focus on fitting the overall LTV distribution, overlooking the fact that LTV in online App advertising consists of sources with diverse data distributions and thus resulting in sub-optimal solutions. In this paper, we propose a novel Progressive Tasks guided Multi-Source Network (PTMSN) to tackle the aforementioned problems. Specifically, a Cascaded Sub-task Module (CSM) is introduced to alleviate data sparsity by modeling reliance between explicit interactions and implicit monetization. In addition, as the overall LTV is assembled from multiple sources, we propose a divide-and-conquer scheme named Multi-source Integrating Module (MIM) to disentangle the original single target into several source distributions and model in a fine-grained manner. Extensive offline experiments on real-world industrial datasets compared to state-of-the-art baseline models validate the effectiveness of our approach. PTMSN has been successfully deployed in industrial online advertising system, serving various business scenarios and acquiring 2.97% absolute ROI gains.

Federated graph learning involves training graph neural networks distributively on local graphs and aggregating model parameters in a central server. However, existing methods fail to effectively capture and leverage the inherent global structures, hindering local structural modeling. To address this, we propose Federated Graph Factorization (FedGF), which enhances structural knowledge via privacy-preserving graph factorization. Specifically, FedGF includes three modules, i.e., global structure reconstruction (GSR), local structure exploration (LSE), and global-local structure alignment (GLSA). Firstly, GSR factorizes client graphs into a series of learnable graph atoms and conducts reconstruction to capture the globally shared structure. Then, LSE explores the local structure, mining potential but unrevealed connections within client subgraphs. GLSA further aligns the global and local structure to alternatively refine the graph atoms and GNN model, enhancing the overall structural modeling. Extensive experiments on six datasets consistently validate the effectiveness of \modelname.

Cross-Domain Recommendation (CDR) has been widely investi- gated for solving long-standing data sparsity problem via knowl- edge sharing across domains. In this paper, we focus on the Multi- Modal Cross-Domain Recommendation (MMCDR) problem where different items have multi-modal information while few users are overlapped across domains. MMCDR is particularly challenging in two aspects: fully exploiting diverse multi-modal information within each domain and leveraging useful knowledge transfer across domains. However, previous methods fail to cluster items with similar characteristics while filtering out inherit noises within different modalities, hurdling the model performance. What is worse, conventional CDR models primarily rely on overlapped users for domain adaptation, making them ill-equipped to handle scenarios where the majority of users are non-overlapped. To fill this gap, we propose Joint Similarity Item Exploration and Overlapped User Guidance (SIEOUG) for solving the MMCDR problem. SIEOUG first proposes similarity item exploration module, which not only obtains pair-wise and group-wise item-item graph knowledge, but also reduces irrelevant noise for multi-modal modeling. Then SIEOUG proposes user-item collaborative filtering module to aggregate user/item embeddings with the attention mechanism for collaborative filtering. Finally SIEOUG proposes overlapped user guidance module with optimal user matching for knowledge sharing across domains. Our empirical study on Amazon dataset with several different tasks demonstrates that SIEOUG significantly outperforms the state-of-the-art models under the MMCDR setting.

With the growing privacy concerns in recommender systems, recommendation unlearning is getting increasing attention. Existing studies predominantly use training data, i.e., model inputs, as unlearning target. However, attackers can extract private information from the model even if it has not been explicitly encountered during training. We name this unseen information as attribute and treat it as unlearning target. To protect the sensitive attribute of users, Attribute Unlearning (AU) aims to make target attributes indistinguishable. In this article, we focus on a strict but practical setting of AU, namely Post-Training Attribute Unlearning (PoT-AU), where unlearning can only be performed after the training of the recommendation model is completed. To address the PoT-AU problem in recommender systems, we propose a two-component loss function. The first component is distinguishability loss, where we design a distribution-based measurement to make attribute labels indistinguishable from attackers. We further extend this measurement to handle multi-class attribute cases with efficient computational overhead. The second component is regularization loss, where we explore a function-space measurement that effectively maintains recommendation performance compared to parameter-space regularization. We use stochastic gradient descent algorithm to optimize our proposed loss. Extensive experiments on four real-world datasets demonstrate the effectiveness of our proposed methods.

Large language models (LLMs) have become instrumental in advancing software engineering (SE) tasks, showcasing their efficacy in code understanding and beyond. AI code models have demonstrated their value not only in code generation but also in defect detection, enhancing security measures and improving overall software quality. They are emerging as crucial tools for both software development and maintaining software quality. Like traditional SE tools, open source collaboration is key in realizing the excellent products. However, with AI models, the essential need is in data. The collaboration of these AI-based SE models hinges on maximizing the sources of high-quality data. However, data, especially of high quality, often hold commercial or sensitive value, making them less accessible for open source AI-based SE projects. This reality presents a significant barrier to the development and enhancement of AI-based SE tools within the SE community. Therefore, researchers need to find solutions for enabling open source AI-based SE models to tap into resources by different organizations. Addressing this challenge, our position article investigates one solution to facilitate access to diverse organizational resources for open source AI models, ensuring that privacy and commercial sensitivities are respected. We introduce a governance framework centered on federated learning (FL), designed to foster the joint development and maintenance of open source AI code models while safeguarding data privacy and security. Additionally, we present guidelines for developers on AI-based SE tool collaboration, covering data requirements, model architecture, updating strategies, and version control. Given the significant influence of data characteristics on FL, our research examines the effect of code data heterogeneity on FL performance. We consider six different scenarios of data distributions and include four code models. We also include four most common FL algorithms. Our experimental findings highlight the potential for employing FL in the collaborative development and maintenance of AI-based SE models. We also discuss the key issues to be addressed in the co-construction process and future research directions.

The demand for mobile terminals to participate in data services is increasingly vital. The General Data Protection Regulation (GDPR) has established several principled requirements for data services. Existing studies focusing on data service put emphasis on data privacy and accessibility. However, they face challenges in achieving data forgetability and portability on mobile devices under GDPR and lack consideration of usage control. In this article, we propose ADSS, an app-level data service scheme for mobile devices that can beavailable-but-invisibleand guarantee fine-grained usage control. ADSS addresses the challenges by executing the logic of data usage in the Trusted Execution Environment (TEE) and managing the TEE states (i.e., data usage states) in the blockchain smart contracts. It not only satisfies the requirements of GDPR, ensuring strong security and confidentiality guarantees, but also enables the functionality of “pay-per-use”. We implement a prototype of the ADSS framework based on ARM Trustzone and conduct experimental evaluations. The results demonstrate that our scheme brings high efficiency compared with other data service schemes and exhibits feasibility on mobile-grade devices.

Cross-Domain Recommendation (CDR) have received widespread attention due to their ability to utilize rich information across domains. However, most existing CDR methods assume an ideal static condition that is not practical in industrial recommendation systems (RS). Therefore, simply applying existing CDR methods in the industrial RS environment may lead to low effectiveness and efficiency. To fill this gap, we propose DIIT, an end-to-end Domain-Invariant Information Transfer method for industrial cross-domain recommendation. Specifically, We first simulate the industrial RS environment that maintains respective models in multiple domains, each of them is trained in the incremental mode. Then, for improving the effectiveness, we design two extractors to fully extract domain-invariant information from the latest source domain models at the domain level and the representation level respectively. Finally, for improving the efficiency, we design a migrator to transfer the extracted information to the latest target domain model, which only need the target domain model for inference. Experiments conducted on one production dataset and two public datasets verify the effectiveness and efficiency of DIIT.

Large language models (LLMs) have achieved unprecedented success in the field of natural language processing. However, the black-box nature of their internal mechanisms has brought many concerns about their trustworthiness and interpretability. Recent research has discovered a class of abnormal tokens in the model's vocabulary space and named them "glitch tokens". Those tokens, once included in the input, may induce the model to produce incorrect, irrelevant, or even harmful results, drastically undermining the reliability and practicality of LLMs.

Federated learning (FL) is a promising machine learning paradigm that collaborates with client models to capture global knowledge. However, deploying FL models in real-world scenarios remains unreliable due to the coexistence of in-distribution data and unexpected out-of-distribution (OOD) data, such as covariate-shift and semantic-shift data. Current FL researches typically address either covariate-shift data through OOD generalization or semantic-shift data via OOD detection, overlooking the simultaneous occurrence of various OOD shifts. In this work, we propose FOOGD, a method that estimates the probability density of each client and obtains reliable global distribution as guidance for the subsequent FL process. Firstly, SM3D in FOOGD estimates score model for arbitrary distributions without prior constraints, and detects semantic-shift data powerfully. Then SAG in FOOGD provides invariant yet diverse knowledge for both local covariate-shift generalization and client performance generalization. In empirical validations, FOOGD significantly enjoys three main advantages: (1) reliably estimating non-normalized decentralized distributions, (2) detecting semantic shift data via score values, and (3) generalizing to covariate-shift data by regularizing feature extractor. The project is open in https://github.com/XeniaLLL/FOOGD-main.git.

Owing to their powerful semantic reasoning capabilities, Large Language Models (LLMs) have been effectively utilized as recommenders, achieving impressive performance. However, the high inference latency of LLMs significantly restricts their practical deployment. To address this issue, this work investigates knowledge distillation from cumbersome LLM-based recommendation models to lightweight conventional sequential models. It encounters three challenges: 1) the teacher’s knowledge may not always be reliable; 2) the capacity gap between the teacher and student makes it difficult for the student to assimilate the teacher’s knowledge; 3) divergence in semantic space poses a challenge to distill the knowledge from embeddings.

The decentralized Federated Learning (FL) paradigm built upon blockchain architectures leverages distributed node clusters to replace the single server for executing FL model aggregation. This paradigm tackles the vulnerability of the centralized malicious server in vanilla FL and inherits the trustfulness and robustness offered by blockchain. However, existing blockchain-enabled schemes face challenges related to inadequate confidentiality on models and limited computational resources of blockchains. In this paper, we present Voltran, an innovative hybrid platform designed to achieve trust, confidentiality, and robustness for FL based on the combination of the Trusted Execution Environment (TEE) and blockchain technology. We offload the FL aggregation computation into TEE to provide an isolated, trusted and customizable off-chain execution and then guarantee the authenticity and verifiability of aggregation results on the blockchain. Moreover, we provide strong scalability on multiple FL scenarios by introducing a multi-SGX parallel execution strategy to amortize the large-scale FL workload. We implement a prototype of Voltran and conduct a comprehensive performance evaluation. Extensive experimental results demonstrate that Voltran incurs minimal additional overhead while guaranteeing trust, confidentiality, and authenticity, and it significantly brings a significant speed-up compared to state-of-the-art ciphertext aggregation schemes.

Federated learning breaks through the barrier of data owners by allowing them to collaboratively train a federated machine learning model without compromising the privacy of their own data. However, Federation Learning also faces the threat of poisoning attacks, especially from the client model updates, which may impair the accuracy of the global model. To defend against the poisoning attacks, previous work aims to identify the malicious updates in high dimensional spaces. However, we find that the distances in high dimensional spaces cannot identify the changes in a small subset of dimensions, and the small changes may affect the global models severely. Based on this finding, we propose an untargeted poisoning attack under the federated learning setting via the partial perturbations on a small subset of the carefully selected model parameters, and present two attack object selection strategies. We experimentally demonstrate that the proposed attack scheme achieves high attack success rate on five state-of-the-art defense schemes. Furthermore, the proposed attack scheme remains effective at low malicious client ratios and still circumvents three defense schemes with a malicious client ratio as low as 2%.

Speaker verification has been widely and successfully adopted in many mission-critical fields for user identification. Similar to other classical machine learning tasks, the training performance of speaker verification relies heavily on the number and diversity of data. Therefore third-party data (e.g. data from public dataset benchmarks, the Internet or other cooperating companies) need to be included during the training process. This raises two major questions: How to use multi-party data while ensuring data security and privacy? Whether adopting entrusted third-party data can threaten the security of the speaker verification model? In this paper, we first demonstrate that federated learning (FL) provides an alternative way for training the speaker verification model without collecting data from multi-parties together. We then validate that it is possible to perform backdoor attack under a looser threat assumption, namely poisoning partial speakers instead of all of the speakers. Specifically, we study the security of speaker verification models in FL for the first time. During the training process of FL, we make full use of the advantages of FL, and design a two stage training strategy. Besides, we propose Global Spectral Cluster (GSC) method to alleviate insufficient trigger generalization problem, which cased by the constrain that the attacker can only reach and poison its own data. We also adopt Personalized Federated Aggregation (PFAgg) to avoid modeling pollution in other parties, enhancing the invisibility of backdoor attack in FL. Experimental results on the TIMIT dataset show that our proposed framework can not only achieve satisfying attack results, but also have an acceptable error rate.

Federated Learning (FL) has emerged as a revolutionary paradigm within the domain of machine learning, offering the capability to facilitate collaborative model training across a network of decentralized devices while meticulously upholding the tenets of data privacy. Nevertheless, a prominent challenge prevails in the realm of safeguarding the commendable performance of the model, while optimizing communication efficiency as well as the computational cost. This paper tackles this challenge by introducing an innovative framework for federated learning that harmoniously integrates principles from both Low-Rank and Sparse decomposition, denoted as FedLRS. We treat low-rank layers as discrete neural network layers and utilize decomposed weights to represent sparse features. This design enabling training without the requirement for reconstructing the original parameters. To enhance stability and accelerate convergence speed, we employ Spectral Initialization and Intermediate Normalization techniques. Furthermore, we impose a sparse constraint grounded in Reconstruction-Based principles to effectively harness sparsity. Through meticulous experimentation and analysis conducted on benchmark datasets such as CIFAR10 and CIFAR100, the results prominently demonstrate that FedLRS not only achieves remarkable performance but also showcases efficiency in terms of communication and computation costs, consequently outperforming existing methodologies in the field.

Federated learning (FL) collaboratively models user data in a decentralized way. However, in the real world, non-identical and independent data distributions (non-IID) among clients hinder the performance of FL due to three issues, i.e., (1) the class statistics shifting, (2) the insufficient hierarchical information utilization, and (3) the inconsistency in aggregating clients. To address the above issues, we propose HyperFed which contains three main modules, i.e., hyperbolic prototype Tammes initialization (HPTI), hyperbolic prototype learning (HPL), and consistent aggregation (CA). Firstly, HPTI in the server constructs uniformly distributed and fixed class prototypes, and shares them with clients to match class statistics, further guiding consistent feature representation for local clients. Secondly, HPL in each client captures the hierarchical information in local data with the supervision of shared class prototypes in the hyperbolic model space. Additionally, CA in the server mitigates the impact of the inconsistent deviations from clients to server. Extensive studies of four datasets prove that HyperFed is effective in enhancing the performance of FL under the non-IID setting.

Recommender systems are typically biased toward a small group of users, leading to severe unfairness in recommendation performance, i.e., User-Oriented Fairness (UOF) issue. The existing research on UOF is limited and fails to deal with the root cause of the UOF issue: the learning process between advantaged and disadvantaged users is unfair. To tackle this issue, we propose an In-processing User Constrained Dominant Sets (In-UCDS) framework, which is a general framework that can be applied to any backbone recommendation model to achieve user-oriented fairness. We split In-UCDS into two stages, i.e., the UCDS modeling stage and the in-processing training stage. In the In-UCDS modeling stage, for each disadvantaged user, we extract a constrained dominant set (a user cluster) containing some advantaged users that are similar to it. In the in-processing training stage, we move the representations of disadvantaged users closer to their corresponding cluster by calculating a fairness loss. By combining the fairness loss with the original backbone model loss, we address the UOF issue and maintain the overall recommendation performance simultaneously. Comprehensive experiments on three real-world datasets demonstrate that In-UCDS outperforms the state-of-the-art methods, leading to a fairer model with better overall recommendation performance.

With the growing privacy concerns in recommender systems, recommendation unlearning, i.e., forgetting the impact of specific learned targets, is getting increasing attention. Existing studies predominantly use training data, i.e., model inputs, as the unlearning target. However, we find that attackers can extract private information, i.e., gender, race, and age, from a trained model even if it has not been explicitly encountered during training. We name this unseen information as attribute and treat it as the unlearning target. To protect the sensitive attribute of users, Attribute Unlearning (AU) aims to degrade attacking performance and make target attributes indistinguishable. In this paper, we focus on a strict but practical setting of AU, namely Post-Training Attribute Unlearning (PoT-AU), where unlearning can only be performed after the training of the recommendation model is completed. To address the PoT-AU problem in recommender systems, we design a two-component loss function that consists of i) distinguishability loss: making attribute labels indistinguishable from attackers, and ii) regularization loss: preventing drastic changes in the model that result in a negative impact on recommendation performance. Specifically, we investigate two types of distinguishability measurements, i.e., user-to-user and distribution-to-distribution. We use the stochastic gradient descent algorithm to optimize our proposed loss. Extensive experiments on three real-world datasets demonstrate the effectiveness of our proposed methods.

Cross-Domain Recommendation (CDR) has been popularly studied for solving the data sparsity problem via leveraging rich knowledge from the auxiliary domain. Most of the current CDR models assume that user-item ratings/reviews are both accessible during the training procedure. However, it is impractical nowadays due to the strict data privacy protection policy. In this paper, we focus on the Privacy-Preserving Cross-Domain Recommendation problem (PPCDR). Although some previous approaches have investigated the problems, they always fail to effectively utilize the rating and review for user preference modeling. What is worse, they separate the processes of privacy data modeling and user-item collaborative filtering, leading to suboptimal solutions. To fill this gap, we propose the Differentially Private Sparse Mapping Recommendation model (DPSMRec), an end-to-end cross-domain recommendation framework for solving the PPCDR problem. DPSMRec includes three main modules, i.e., source embedding generation module, target rating prediction module, and differentially private sparse mapping module. Specifically, the source embedding generation module and target rating prediction module are set to exploit the ratings and review in each domain. The differentially private sparse mapping module aims to transfer knowledge among the overlapped users via Wasserstein distance privately. To obtain more accurate mapping solutions, we further propose differentially private enhanced sparse optimal transport via fused Gromov-Wasserstein distance which can consider both structure and semantic information among the users across domains. Our empirical study on Amazon and Douban datasets demonstrates that DPSMRec significantly outperforms the state-of-the-art models under the PPCDR setting.

With growing concerns regarding privacy in machine learning models, regulations have committed to granting individuals the right to be forgotten while mandating companies to develop non-discriminatory machine learning systems, thereby fueling the study of the machine unlearning problem. Our attention is directed toward a practical unlearning scenario, i.e., recommendation unlearning. As the state-of-the-art framework, i.e., RecEraser, naturally achieves full unlearning completeness, our objective is to enhance it in terms of model utility and unlearning efficiency. In this paper, we rethink RecEraser from an ensemble-based perspective and focus on its three potential losses, i.e., redundancy, relevance, and combination. Under the theoretical guidance of the above three losses, we propose a new framework named UltraRE, which simplifies and powers RecEraser for recommendation tasks. Specifically, for redundancy loss, we incorporate transport weights in the clustering algorithm to optimize the equilibrium between collaboration and balance while enhancing efficiency; for relevance loss, we ensure that sub-models reach convergence on their respective group data; for combination loss, we simplify the combination estimator without compromising its efficacy. Extensive experiments on three real-world datasets demonstrate the effectiveness of UltraRE.

The technique of packing multiple values into one message without losing homomorphic computation properties is the main workhorse that drives many exciting advances in applying lattice-based homomorphic encryption schemes to privacy-preserving Machine-Learning-as-a-Service (MLaaS). However, this technique does not directly work for the classic Paillier homomorphic encryption scheme, limiting the use of the Paillier scheme in the privacy-preserving MLaaS. To enrich the applications of Paillier in privacy-preserving MLaaS, we present a set of new methods for efficient linear computations over packed values under the Paillier scheme, such as vector multiplication, matrix multiplication, and convolutional calculation between ciphertexts and plaintexts. Different from the packing methods of lattice-based schemes, the Paillier packing method naturally allows higher packing capability for values in lower bit-length. This property can significantly benefit privacy-preserving MLaaS, as the values of user inputs and parameters of machine learning models are often quantized into low bits (e.g., 1-8 bits). We conduct comparisons based on different linear computation tasks, the proposed methods under the Paillier scheme clearly outperform the state-of-the-art in terms of communication and computational efficiency, especially in realistic scenarios. For example, compared to one of the recent arts CrypTFlow2 [1], the communication cost of our solution can be 21.7× smaller at best. Thanks to the reduction of communication cost, the runtime can be 2.46× faster than CrypTFlow2 at the median-country-speed of current global mobile broadband.

With the explosive growth of user data, recommenders have become increasingly complicated. State-of-the-art algorithms often have high computational complexity and heavily use non-linear transformations. This fact makes the privacy-preserving problem more challenging, despite the significant advances in cryptography. To alleviate this problem, we propose a privacy-friendly recommender, CryptoRec. It only relies on additions and multiplications, which are efficiently supported by most cryptographic primitives. Different from others, in CryptoRec, the parameter space only contains item features (user features can be directly computed from the item features). This property allows CryptoRec to, (1) naturally achieve transferability if two datasets share the same item entries, which can benefit differential privacy protection; (2) directly estimate the preference of new users whose data is not included in the training set, drastically improving recommendation efficiency. We first evaluate CryptoRec on three real-world datasets. The evaluation results show that the accuracy is competitive with state-of-the-art. Then, we build differential privacy into CryptoRec and leverage its transferability property to reduce the overall privacy loss. Lastly, we demonstrate the simplicity and efficiency of using CryptoRec to construct secure recommendation protocols based on homomorphic encryption schemes. Our results show that CryptoRec outperforms existing solutions in terms of both accuracy and efficiency.

Automatically detecting the positions of key-points (e.g., facial key-points or finger key-points) in an image is an essential problem in many applications, such as driver's gaze detection and drowsiness detection in automated driving systems. With the recent advances of Deep Neural Networks (DNNs), Key-Points detection DNNs (KP-DNNs) have been increasingly employed for that purpose. Nevertheless, KP-DNN testing and validation have remained a challenging problem because KP-DNNs predict many independent key-points at the same time---where each individual key-point may be critical in the targeted application---and images can vary a great deal according to many factors.

Nowadays, recommender system is an indispensable tool in many information services, and a large number of algorithms have been designed and implemented. However, fed with very large datasets, state-of-the-art recommendation algorithms often face an efficiency bottleneck, i.e., it takes huge amount of computing resources to train a recommendation model. In order to satisfy the needs of privacy-savvy users who do not want to disclose their information to the service provider, the complexity of most existing solutions becomes prohibitive. As such, it is an interesting research question to design simple and efficient recommendation algorithms that achieve reasonable accuracy and facilitate privacy protection at the same time. In this paper, we propose an efficient recommendation algorithm, named CryptoRec, which has two nice properties: (1) can estimate a new user's preferences by directly using a model pre-learned from an expert dataset, and the new user's data is not required to train the model; (2) can compute recommendations with only addition and multiplication operations. As to the evaluation, we first test the recommendation accuracy on three real-world datasets and show that CryptoRec is competitive with state-of-the-art recommenders. Then, we evaluate the performance of the privacy-preserving variants of CryptoRec and show that predictions can be computed in seconds on a PC. In contrast, existing solutions will need tens or hundreds of hours on more powerful computers.

Machine-Learning-as-a-Service has become increasingly popular, with Recommendation-as-a-Service as one of the representative examples. In such services, providing privacy protection for the users is an important topic. Reviewing privacy-preserving solutions which were proposed in the past decade, privacy and machine learning are often seen as two competing goals at stake. Though improving cryptographic primitives (e.g., secure multi-party computation (SMC) or homomorphic encryption (HE)) or devising sophisticated secure protocols has made a remarkable achievement, but in conjunction with state-of-the-art recommender systems often yields far-from-practical solutions. We tackle this problem from the direction of machine learning. We aim to design crypto-friendly recommendation algorithms, thus to obtain efficient solutions by directly using existing cryptographic tools. In particular, we propose an HE-friendly recommender system, refer to as CryptoRec, which (1) decouples user features from latent feature space, avoiding training the recommendation model on encrypted data; (2) only relies on addition and multiplication operations, making the model straightforwardly compatible with HE schemes. The properties turn recommendation-computations into a simple matrix-multiplication operation. To further improve efficiency, we introduce a sparse-quantization-reuse method which reduces the recommendation-computation time by $9\times$ (compared to using CryptoRec directly), without compromising the accuracy. We demonstrate the efficiency and accuracy of CryptoRec on three real-world datasets. CryptoRec allows a server to estimate a user's preferences on thousands of items within a few seconds on a single PC, with the user's data homomorphically encrypted, while its prediction accuracy is still competitive with state-of-the-art recommender systems computing over clear data. Our solution enables Recommendation-as-a-Service on large datasets in a nearly real-time (seconds) level.

Privacy-preserving recommender systems have been an active research topic for many years. However, until today, it is still a challenge to design an efficient solution without involving a fully trusted third party or multiple semi-trusted third parties. The key obstacle is the large underlying user populations (i.e., huge input size) in the systems. In this paper, we revisit the concept of friendship-based recommender systems, proposed by Jeckmans et al. and Tang and Wang. These solutions are very promising because recommendations are computed based on inputs from a very small subset of the overall user population (precisely, a user's friends and some randomly chosen strangers). We first clarify the single prediction protocol and Top-n protocol by Tang and Wang, by correcting some flaws and improving the efficiency of the single prediction protocol. We then design a decentralized single protocol by getting rid of the semi-honest service provider. In order to validate the designed protocols, we crawl Twitter and construct two datasets (FMT and 10-FMT) which are equipped with auxiliary friendship information. Based on 10-FMT and MovieLens 100k dataset with simulated friendships, we show that even if our protocols use a very small subset of the datasets, their accuracy can still be equal to or better than some baseline algorithm. Based on these datasets, we further demonstrate that the outputs of our protocols leak very small amount of information of the inputs, and the leakage decreases when the input size increases. We finally show that he single prediction protocol is quite efficient but the Top-n is not. However, we observe that the efficiency of the Top-n protocol can be dramatically improved if we slightly relax the desired security guarantee.